2026-06-10 00:02:37.360826 | Job console starting 2026-06-10 00:02:37.491791 | Updating git repos 2026-06-10 00:02:37.521990 | Cloning repos into workspace 2026-06-10 00:02:38.067402 | Restoring repo states 2026-06-10 00:02:38.083325 | Merging changes 2026-06-10 00:02:38.083346 | Checking out repos 2026-06-10 00:02:38.320630 | Preparing playbooks 2026-06-10 00:02:40.617965 | Running Ansible setup 2026-06-10 00:02:44.506804 | PRE-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/pre.yaml@main] 2026-06-10 00:02:45.219674 | 2026-06-10 00:02:45.219831 | PLAY [localhost] 2026-06-10 00:02:45.227469 | 2026-06-10 00:02:45.227547 | TASK [Gathering Facts] 2026-06-10 00:02:46.379840 | localhost | ok 2026-06-10 00:02:46.386549 | 2026-06-10 00:02:46.386767 | TASK [Setup log path fact] 2026-06-10 00:02:46.406611 | localhost | ok 2026-06-10 00:02:46.418507 | 2026-06-10 00:02:46.418699 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 00:02:46.448042 | localhost | ok 2026-06-10 00:02:46.460565 | 2026-06-10 00:02:46.460760 | TASK [emit-job-header : Print job information] 2026-06-10 00:02:46.501125 | # Job Information 2026-06-10 00:02:46.501378 | Ansible Version: 2.16.18 2026-06-10 00:02:46.501412 | Job: sf-operator-minikube-integration 2026-06-10 00:02:46.501433 | Pipeline: periodic 2026-06-10 00:02:46.501452 | Executor: 38.102.83.40 2026-06-10 00:02:46.501471 | Triggered by: https://softwarefactory-project.io/r/plugins/gitiles/software-factory/sf-operator/+/None^!/ 2026-06-10 00:02:46.501494 | Log URL (when completed): https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/logs/5d7/sf/5d71f650632045e78ef50fac9860d220/ 2026-06-10 00:02:46.501514 | Event ID: 2333b81a76ed4179814fd4bfda749aac 2026-06-10 00:02:46.504371 | 2026-06-10 00:02:46.504556 | LOOP [emit-job-header : Print node information] 2026-06-10 00:02:46.622675 | localhost | ok: 2026-06-10 00:02:46.623089 | localhost | # Node Information 2026-06-10 00:02:46.623131 | localhost | Inventory Hostname: controller 2026-06-10 00:02:46.623157 | localhost | Hostname: np0000069206 2026-06-10 00:02:46.623181 | localhost | Username: zuul-worker 2026-06-10 00:02:46.623205 | localhost | Distro: CentOS 9 2026-06-10 00:02:46.623228 | localhost | Provider: vexxhost-nodepool-sf 2026-06-10 00:02:46.623249 | localhost | Region: RegionOne 2026-06-10 00:02:46.623270 | localhost | Label: cloud-centos-9-big 2026-06-10 00:02:46.623291 | localhost | Product Name: OpenStack Nova 2026-06-10 00:02:46.623313 | localhost | Interface IP: 38.102.83.130 2026-06-10 00:02:46.640078 | 2026-06-10 00:02:46.640228 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-10 00:02:47.124124 | localhost -> localhost | changed 2026-06-10 00:02:47.130975 | 2026-06-10 00:02:47.131093 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-06-10 00:02:48.373892 | localhost -> localhost | changed 2026-06-10 00:02:48.382569 | 2026-06-10 00:02:48.382970 | PLAY [all] 2026-06-10 00:02:48.394717 | 2026-06-10 00:02:48.394919 | TASK [include_role : start-zuul-console] 2026-06-10 00:02:48.416301 | controller | ok 2026-06-10 00:02:48.431128 | 2026-06-10 00:02:48.431327 | TASK [start-zuul-console : Start zuul_console daemon.] 2026-06-10 00:02:48.857931 | controller | ok 2026-06-10 00:02:48.865810 | 2026-06-10 00:02:48.865975 | TASK [include_role : validate-host] 2026-06-10 00:02:48.896385 | controller | ok 2026-06-10 00:02:48.915304 | 2026-06-10 00:02:48.915628 | TASK [validate-host : Define zuul_info_dir fact] 2026-06-10 00:02:48.953959 | controller | ok 2026-06-10 00:02:48.958402 | 2026-06-10 00:02:48.958520 | TASK [validate-host : Ensure Zuul Ansible directory exists] 2026-06-10 00:02:49.258795 | controller -> localhost | ok 2026-06-10 00:02:49.266359 | 2026-06-10 00:02:49.266519 | TASK [validate-host : Collect information about the host] 2026-06-10 00:02:50.011654 | controller | ok 2026-06-10 00:02:50.077112 | 2026-06-10 00:02:50.077298 | TASK [validate-host : Sanitize hostname] 2026-06-10 00:02:50.137310 | controller | ok 2026-06-10 00:02:50.142998 | 2026-06-10 00:02:50.143132 | TASK [validate-host : Write out all ansible variables/facts known for each host] 2026-06-10 00:02:50.700003 | controller -> localhost | changed 2026-06-10 00:02:50.705509 | 2026-06-10 00:02:50.705750 | TASK [validate-host : Collect information about zuul worker] 2026-06-10 00:02:51.145430 | controller | ok 2026-06-10 00:02:51.150517 | 2026-06-10 00:02:51.150697 | TASK [validate-host : Write out all zuul information for each host] 2026-06-10 00:02:51.655999 | controller -> localhost | changed 2026-06-10 00:02:51.664181 | 2026-06-10 00:02:51.664274 | TASK [include_role : prepare-workspace] 2026-06-10 00:02:51.692753 | controller | ok 2026-06-10 00:02:51.710939 | 2026-06-10 00:02:51.711104 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-06-10 00:02:51.977156 | controller | ok 2026-06-10 00:02:51.986680 | 2026-06-10 00:02:51.986829 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-06-10 00:02:53.139667 | controller | Output suppressed because no_log was given 2026-06-10 00:02:53.148053 | 2026-06-10 00:02:53.148193 | TASK [include_role : add-build-sshkey] 2026-06-10 00:02:53.179207 | controller | ok 2026-06-10 00:02:53.207542 | 2026-06-10 00:02:53.207725 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-06-10 00:02:53.466901 | controller -> localhost | ok 2026-06-10 00:02:53.473012 | 2026-06-10 00:02:53.473141 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-06-10 00:02:53.503790 | controller | ok 2026-06-10 00:02:53.523713 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-06-10 00:02:53.530289 | 2026-06-10 00:02:53.530396 | TASK [add-build-sshkey : Create Temp SSH key] 2026-06-10 00:02:54.529257 | controller -> localhost | Generating public/private rsa key pair. 2026-06-10 00:02:54.529472 | controller -> localhost | Your identification has been saved in /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/work/5d71f650632045e78ef50fac9860d220_id_rsa 2026-06-10 00:02:54.529500 | controller -> localhost | Your public key has been saved in /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/work/5d71f650632045e78ef50fac9860d220_id_rsa.pub 2026-06-10 00:02:54.529521 | controller -> localhost | The key fingerprint is: 2026-06-10 00:02:54.529542 | controller -> localhost | SHA256:Qe4atqqBu3U8AMbZu8tNxC/eCW/hy7O8GSD6WDg9ZyQ zuul-build-sshkey 2026-06-10 00:02:54.529561 | controller -> localhost | The key's randomart image is: 2026-06-10 00:02:54.529601 | controller -> localhost | +---[RSA 3072]----+ 2026-06-10 00:02:54.529622 | controller -> localhost | | . | 2026-06-10 00:02:54.529641 | controller -> localhost | |. o o | 2026-06-10 00:02:54.529660 | controller -> localhost | |.+ . o | 2026-06-10 00:02:54.529678 | controller -> localhost | |.. o . . | 2026-06-10 00:02:54.529696 | controller -> localhost | | .E =o S | 2026-06-10 00:02:54.529715 | controller -> localhost | | .+o*.o= | 2026-06-10 00:02:54.529732 | controller -> localhost | |.+o=+*+o. | 2026-06-10 00:02:54.529751 | controller -> localhost | | o*oOoB++ | 2026-06-10 00:02:54.529768 | controller -> localhost | |oo.=.o.@= | 2026-06-10 00:02:54.529786 | controller -> localhost | +----[SHA256]-----+ 2026-06-10 00:02:54.529837 | controller -> localhost | ok: Runtime: 0:00:00.627501 2026-06-10 00:02:54.533656 | 2026-06-10 00:02:54.533744 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-06-10 00:02:54.572509 | controller | ok 2026-06-10 00:02:54.588530 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-06-10 00:02:54.600891 | 2026-06-10 00:02:54.601095 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-06-10 00:02:54.627736 | controller | skipping: Conditional result was False 2026-06-10 00:02:54.635055 | 2026-06-10 00:02:54.635152 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-06-10 00:02:55.088614 | controller | changed 2026-06-10 00:02:55.093806 | 2026-06-10 00:02:55.093912 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-06-10 00:02:55.361041 | controller | ok 2026-06-10 00:02:55.369779 | 2026-06-10 00:02:55.369989 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-06-10 00:02:56.156072 | controller | changed 2026-06-10 00:02:56.160919 | 2026-06-10 00:02:56.161003 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-06-10 00:02:57.017542 | controller | changed 2026-06-10 00:02:57.026283 | 2026-06-10 00:02:57.026534 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-06-10 00:02:57.063420 | controller | skipping: Conditional result was False 2026-06-10 00:02:57.071832 | 2026-06-10 00:02:57.071968 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-06-10 00:02:57.516970 | controller -> localhost | changed 2026-06-10 00:02:57.527543 | 2026-06-10 00:02:57.527784 | TASK [add-build-sshkey : Add back temp key] 2026-06-10 00:02:57.888231 | controller -> localhost | Identity added: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/work/5d71f650632045e78ef50fac9860d220_id_rsa (zuul-build-sshkey) 2026-06-10 00:02:57.889115 | controller -> localhost | ok: Runtime: 0:00:00.038160 2026-06-10 00:02:57.897299 | 2026-06-10 00:02:57.897476 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-06-10 00:02:58.238201 | controller | ok 2026-06-10 00:02:58.242830 | 2026-06-10 00:02:58.242958 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-06-10 00:02:58.270332 | controller | skipping: Conditional result was False 2026-06-10 00:02:58.280827 | 2026-06-10 00:02:58.280977 | TASK [include_role : prepare-workspace-openshift] 2026-06-10 00:02:58.308031 | controller | skipping: Conditional result was False 2026-06-10 00:02:58.312988 | 2026-06-10 00:02:58.313103 | TASK [include_role : remove-zuul-sshkey] 2026-06-10 00:02:58.338464 | controller | skipping: Conditional result was False 2026-06-10 00:02:58.343981 | 2026-06-10 00:02:58.344079 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-06-10 00:02:58.611624 | controller | ok: "logs" 2026-06-10 00:02:58.612058 | controller | ok: All items complete 2026-06-10 00:02:58.612116 | 2026-06-10 00:02:58.826009 | controller | ok: "artifacts" 2026-06-10 00:02:59.019496 | controller | ok: "docs" 2026-06-10 00:02:59.025903 | 2026-06-10 00:02:59.026046 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-06-10 00:02:59.240250 | controller | changed: "logs" 2026-06-10 00:02:59.427546 | controller | changed: "artifacts" 2026-06-10 00:02:59.662963 | controller | changed: "docs" 2026-06-10 00:02:59.687373 | 2026-06-10 00:02:59.687566 | LOOP [Tune dnf options] 2026-06-10 00:03:00.159216 | controller | changed: option added 2026-06-10 00:03:00.159527 | controller | changed: All items complete 2026-06-10 00:03:00.159564 | 2026-06-10 00:03:00.376854 | controller | changed: option added 2026-06-10 00:03:00.607986 | controller | changed: option added 2026-06-10 00:03:00.616785 | 2026-06-10 00:03:00.617019 | PLAY RECAP 2026-06-10 00:03:00.617094 | controller | ok: 24 changed: 11 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-06-10 00:03:00.617133 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 00:03:00.617159 | 2026-06-10 00:03:00.829808 | PRE-RUN END RESULT_NORMAL: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/pre.yaml@main] 2026-06-10 00:03:00.832465 | PRE-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/rhel_subscription/register.yaml@main] 2026-06-10 00:03:01.537222 | 2026-06-10 00:03:01.537408 | PLAY [all] 2026-06-10 00:03:01.548568 | 2026-06-10 00:03:01.548774 | TASK [Register system] 2026-06-10 00:03:01.603753 | 2026-06-10 00:03:01.604143 | TASK [Get minor release for EUS activation] 2026-06-10 00:03:01.672013 | controller | skipping: Conditional result was False 2026-06-10 00:03:01.679182 | 2026-06-10 00:03:01.679429 | LOOP [Manage Extended Update Support repos] 2026-06-10 00:03:01.735033 | controller | skipping: Conditional result was False 2026-06-10 00:03:01.739039 | controller | skipping: Conditional result was False 2026-06-10 00:03:01.757291 | 2026-06-10 00:03:01.757538 | TASK [Upgrade all packages] 2026-06-10 00:03:01.805722 | controller | skipping: Conditional result was False 2026-06-10 00:03:01.817194 | 2026-06-10 00:03:01.817442 | TASK [Reboot to use new kernel] 2026-06-10 00:03:01.865840 | controller | skipping: Conditional result was False 2026-06-10 00:03:01.868155 | 2026-06-10 00:03:01.868268 | PLAY RECAP 2026-06-10 00:03:01.868420 | controller | ok: 0 changed: 0 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-06-10 00:03:01.868500 | 2026-06-10 00:03:02.086336 | PRE-RUN END RESULT_NORMAL: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/rhel_subscription/register.yaml@main] 2026-06-10 00:03:02.089629 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-minikube.yaml@master] 2026-06-10 00:03:02.837122 | 2026-06-10 00:03:02.837266 | PLAY [controller] 2026-06-10 00:03:02.848857 | 2026-06-10 00:03:02.849003 | TASK [Create ramdisk folder] 2026-06-10 00:03:03.311456 | controller | changed 2026-06-10 00:03:03.320260 | 2026-06-10 00:03:03.320558 | TASK [Installing packages] 2026-06-10 00:04:13.594704 | controller | changed 2026-06-10 00:04:13.601421 | 2026-06-10 00:04:13.601552 | TASK [Set pids_limit in containers.conf] 2026-06-10 00:04:14.618638 | controller | changed 2026-06-10 00:04:14.625165 | 2026-06-10 00:04:14.625309 | TASK [Installing Minikube] 2026-06-10 00:04:18.511947 | controller | changed: OK (132766301 bytes) 2026-06-10 00:04:18.519786 | 2026-06-10 00:04:18.519899 | TASK [Start minikube] 2026-06-09 20:04:19.037006 | controller | * minikube v1.36.0 on Centos 9 (kvm/amd64) 2026-06-09 20:04:19.042056 | controller | Using Kubernetes 1.32.5 since patch version was unspecified 2026-06-09 20:04:19.362532 | controller | * minikube 1.38.1 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.38.1 2026-06-09 20:04:19.366797 | controller | * To disable this notice, run: 'minikube config set WantUpdateNotification false' 2026-06-09 20:04:19.366830 | controller | 2026-06-09 20:04:19.383686 | controller | * Using the podman driver based on user configuration 2026-06-09 20:04:20.204365 | controller | * Using Podman driver with root privileges 2026-06-09 20:04:20.208350 | controller | * Starting "minikube" primary control-plane node in "minikube" cluster 2026-06-09 20:04:20.213845 | controller | * Pulling base image v0.0.47 ... 2026-06-09 20:04:20.341046 | controller | * Downloading Kubernetes v1.32.5 preload ... 2026-06-09 20:04:33.826644 | controller | > gcr.io/k8s-minikube/kicbase...: 0 B [_______________________] ?% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 0 B [_______________________] ?% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB [>_] 0.00% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB 0.00% 2.68 KiB p/ > gcr.io/k8s-minikube/kicbase...: 225.70 KiB / 502.26 MiB 0.04% 2.68 KiB > gcr.io/k8s-minikube/kicbase...: 1.89 MiB / 502.26 MiB 0.38% 2.68 KiB p/ > gcr.io/k8s-minikube/kicbase...: 11.77 MiB / 502.26 MiB 2.34% 1.27 MiB p > gcr.io/k8s-minikube/kicbase...: 23.60 MiB / 502.26 MiB 4.70% 1.27 MiB p > gcr.io/k8s-minikube/kicbase...: 35.53 MiB / 502.26 MiB 7.07% 1.27 MiB p > gcr.io/k8s-minikube/kicbase...: 47.49 MiB / 502.26 MiB 9.45% 5.03 MiB p > gcr.io/k8s-minikube/kicbase...: 59.32 MiB / 502.26 MiB 11.81% 5.03 MiB > gcr.io/k8s-minikube/kicbase...: 71.32 MiB / 502.26 MiB 14.20% 5.03 MiB > gcr.io/k8s-minikube/kicbase...: 83.37 MiB / 502.26 MiB 16.60% 8.56 MiB > gcr.io/k8s-minikube/kicbase...: 95.27 MiB / 502.26 MiB 18.97% 8.56 MiB > gcr.io/k8s-minikube/kicbase...: 104.57 MiB / 502.26 MiB 20.82% 8.56 MiB > gcr.io/k8s-minikube/kicbase...: 116.67 MiB / 502.26 MiB 23.23% 11.58 Mi > gcr.io/k8s-minikube/kicbase...: 128.48 MiB / 502.26 MiB 25.58% 11.58 Mi > gcr.io/k8s-minikube/kicbase...: 140.51 MiB / 502.26 MiB 27.98% 11.58 Mi > gcr.io/k8s-minikube/kicbase...: 152.51 MiB / 502.26 MiB 30.36% 14.70 Mi > gcr.io/k8s-minikube/kicbase...: 164.60 MiB / 502.26 MiB 32.77% 14.70 Mi > gcr.io/k8s-minikube/kicbase...: 168.00 MiB / 502.26 MiB 33.45% 14.70 Mi > gcr.io/k8s-minikube/kicbase...: 173.91 MiB / 502.26 MiB 34.63% 16.05 Mi > gcr.io/k8s-minikube/kicbase...: 185.60 MiB / 502.26 MiB 36.95% 16.05 Mi > gcr.io/k8s-minikube/kicbase...: 192.00 MiB / 502.26 MiB 38.23% 16.05 Mi > gcr.io/k8s-minikube/kicbase...: 192.00 MiB / 502.26 MiB 38.23% 16.96 Mi > gcr.io/k8s-minikube/kicbase...: 201.19 MiB / 502.26 MiB 40.06% 16.96 Mi > gcr.io/k8s-minikube/kicbase...: 213.13 MiB / 502.26 MiB 42.43% 16.96 Mi > gcr.io/k8s-minikube/kicbase...: 225.14 MiB / 502.26 MiB 44.83% 19.43 Mi > gcr.io/k8s-minikube/kicbase...: 237.17 MiB / 502.26 MiB 47.22% 19.43 Mi > gcr.io/k8s-minikube/kicbase...: 249.22 MiB / 502.26 MiB 49.62% 19.43 Mi > gcr.io/k8s-minikube/kicbase...: 261.24 MiB / 502.26 MiB 52.01% 22.06 Mi > gcr.io/k8s-minikube/kicbase...: 273.24 MiB / 502.26 MiB 54.40% 22.06 Mi > gcr.io/k8s-minikube/kicbase...: 285.33 MiB / 502.26 MiB 56.81% 22.06 Mi > gcr.io/k8s-minikube/kicbase...: 297.27 MiB / 502.26 MiB 59.19% 24.51 Mi > gcr.io/k8s-minikube/kicbase...: 305.31 MiB / 502.26 MiB 60.79% 24.51 Mi > gcr.io/k8s-minikube/kicbase...: 317.17 MiB / 502.26 MiB 63.15% 24.51 Mi > gcr.io/k8s-minikube/kicbase...: 328.99 MiB / 502.26 MiB 65.50% 26.34 Mi > gcr.io/k8s-minikube/kicbase...: 340.92 MiB / 502.26 MiB 67.88% 26.34 Mi > gcr.io/k8s-minikube/kicbase...: 352.91 MiB / 502.26 MiB 70.26% 26.34 Mi > gcr.io/k8s-minikube/kicbase...: 362.22 MiB / 502.26 MiB 72.12% 28.21 Mi > gcr.io/k8s-minikube/kicbase...: 369.47 MiB / 502.26 MiB 73.56% 28.21 Mi > gcr.io/k8s-minikube/kicbase...: 381.49 MiB / 502.26 MiB 75.95% 28.21 Mi > gcr.io/k8s-minikube/kicbase...: 392.00 MiB / 502.26 MiB 78.05% 29.59 Mi > gcr.io/k8s-minikube/kicbase...: 400.00 MiB / 502.26 MiB 79.64% 29.59 Mi > gcr.io/k8s-minikube/kicbase...: 400.00 MiB / 502.26 MiB 79.64% 29.59 Mi > gcr.io/k8s-minikube/kicbase...: 400.00 MiB / 502.26 MiB 79.64% 28.54 Mi > gcr.io/k8s-minikube/kicbase...: 408.19 MiB / 502.26 MiB 81.27% 28.54 Mi > gcr.io/k8s-minikube/kicbase...: 416.00 MiB / 502.26 MiB 82.83% 28.54 Mi > gcr.io/k8s-minikube/kicbase...: 424.00 MiB / 502.26 MiB 84.42% 29.28 Mi > gcr.io/k8s-minikube/kicbase...: 432.00 MiB / 502.26 MiB 86.01% 29.28 Mi > gcr.io/k8s-minikube/kicbase...: 440.00 MiB / 502.26 MiB 87.60% 29.28 Mi > gcr.io/k8s-minikube/kicbase...: 440.00 MiB / 502.26 MiB 87.60% 29.11 Mi > gcr.io/k8s-minikube/kicbase...: 448.00 MiB / 502.26 MiB 89.20% 29.11 Mi > gcr.io/k8s-minikube/kicbase...: 453.53 MiB / 502.26 MiB 90.30% 29.11 Mi > gcr.io/k8s-minikube/kicbase...: 456.00 MiB / 502.26 MiB 90.79% 28.95 Mi > gcr.io/k8s-minikube/kicbase...: 459.67 MiB / 502.26 MiB 91.52% 28.95 Mi > gcr.io/k8s-minikube/kicbase...: 464.00 MiB / 502.26 MiB 92.38% 28.95 Mi > gcr.io/k8s-minikube/kicbase...: 469.61 MiB / 502.26 MiB 93.50% 28.55 Mi > gcr.io/k8s-minikube/kicbase...: 472.00 MiB / 502.26 MiB 93.98% 28.55 Mi > gcr.io/k8s-minikube/kicbase...: 476.74 MiB / 502.26 MiB 94.92% 28.55 Mi > gcr.io/k8s-minikube/kicbase...: 484.77 MiB / 502.26 MiB 96.52% 28.34 Mi > gcr.io/k8s-minikube/kicbase...: 490.08 MiB / 502.26 MiB 97.57% 28.34 Mi > gcr.io/k8s-minikube/kicbase...: 500.27 MiB / 502.26 MiB 99.60% 28.34 Mi > gcr.io/k8s-minikube/kicbase...: 502.26 MiB / 502.26 MiB 100.00% 40.37 ME0609 20:04:33.826526 12596 cache.go:225] Error downloading kic artifacts: not yet implemented, see issue #8426 2026-06-09 20:04:33.832870 | controller | * Creating podman container (CPUs=no-limit, Memory=no-limit) ... 2026-06-09 20:05:10.747899 | controller | * Preparing Kubernetes v1.32.5 on CRI-O 1.24.6 ... 2026-06-09 20:05:10.951288 | controller | - kubelet.housekeeping-interval=5m 2026-06-09 20:05:21.725032 | controller | - Generating certificates and keys ... 2026-06-09 20:05:21.729311 | controller | - Booting up control plane ... 2026-06-09 20:05:21.735887 | controller | - Configuring RBAC rules ... 2026-06-09 20:05:21.742087 | controller | * Configuring CNI (Container Networking Interface) ... 2026-06-09 20:05:22.256046 | controller | * Verifying Kubernetes components... 2026-06-09 20:05:22.338339 | controller | - Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3 2026-06-09 20:05:22.346445 | controller | - Using image gcr.io/k8s-minikube/storage-provisioner:v5 2026-06-09 20:05:22.357855 | controller | - Using image registry.k8s.io/ingress-nginx/controller:v1.12.2 2026-06-09 20:05:22.360048 | controller | - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.3 2026-06-09 20:05:22.361563 | controller | - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.3 2026-06-09 20:05:23.542209 | controller | * Verifying ingress addon... 2026-06-09 20:06:20.554306 | controller | * Enabled addons: ingress-dns, storage-provisioner, default-storageclass, ingress 2026-06-09 20:06:20.564553 | controller | * kubectl not found. If you need it, try: 'minikube kubectl -- get pods -A' 2026-06-09 20:06:20.566531 | controller | * Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default 2026-06-10 00:06:20.695359 | controller | ok: Runtime: 0:02:01.628689 2026-06-10 00:06:20.700967 | 2026-06-10 00:06:20.701066 | TASK [Get minikube deployment ip] 2026-06-09 20:06:21.203772 | controller | 192.168.49.2 2026-06-10 00:06:21.243711 | controller | ok: Runtime: 0:00:00.298301 2026-06-10 00:06:21.248192 | 2026-06-10 00:06:21.248292 | TASK [Get default interface] 2026-06-09 20:06:21.500741 | controller | eth0 2026-06-10 00:06:21.783919 | controller | ok: Runtime: 0:00:00.008552 2026-06-10 00:06:21.788959 | 2026-06-10 00:06:21.789044 | TASK [Test minikube] 2026-06-09 20:06:34.144928 | controller | NAMESPACE NAME READY STATUS RESTARTS AGE 2026-06-09 20:06:34.145026 | controller | ingress-nginx ingress-nginx-admission-create-gqrrr 0/1 Completed 0 68s 2026-06-09 20:06:34.145042 | controller | ingress-nginx ingress-nginx-admission-patch-ftshk 0/1 Completed 2 68s 2026-06-09 20:06:34.145060 | controller | ingress-nginx ingress-nginx-controller-86649c845b-wnd44 1/1 Running 0 68s 2026-06-09 20:06:34.145071 | controller | kube-system coredns-668d6bf9bc-5m5hh 1/1 Running 0 68s 2026-06-09 20:06:34.145080 | controller | kube-system etcd-minikube 1/1 Running 0 73s 2026-06-09 20:06:34.145089 | controller | kube-system kindnet-twq7n 1/1 Running 0 68s 2026-06-09 20:06:34.145097 | controller | kube-system kube-apiserver-minikube 1/1 Running 0 73s 2026-06-09 20:06:34.145105 | controller | kube-system kube-controller-manager-minikube 1/1 Running 0 73s 2026-06-09 20:06:34.145113 | controller | kube-system kube-ingress-dns-minikube 0/1 ImagePullBackOff 0 72s 2026-06-09 20:06:34.145127 | controller | kube-system kube-proxy-92vnl 1/1 Running 0 68s 2026-06-09 20:06:34.145135 | controller | kube-system kube-scheduler-minikube 1/1 Running 0 73s 2026-06-09 20:06:34.145143 | controller | kube-system storage-provisioner 1/1 Running 0 72s 2026-06-10 00:06:34.341227 | controller | ok: Runtime: 0:00:12.155777 2026-06-10 00:06:34.344325 | 2026-06-10 00:06:34.344424 | PLAY RECAP 2026-06-10 00:06:34.344507 | controller | ok: 8 changed: 8 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 00:06:34.344551 | 2026-06-10 00:06:34.804157 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-minikube.yaml@master] 2026-06-10 00:06:34.808653 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/prepare-minikube.yaml@master] 2026-06-10 00:06:35.506454 | 2026-06-10 00:06:35.506642 | PLAY [controller] 2026-06-10 00:06:35.519617 | 2026-06-10 00:06:35.519710 | TASK [Crete sf namespace] 2026-06-09 20:06:36.024059 | controller | namespace/sf created 2026-06-10 00:06:36.067007 | controller | ok: Runtime: 0:00:00.137087 2026-06-10 00:06:36.071203 | 2026-06-10 00:06:36.071290 | TASK [Set sf as the default namespace] 2026-06-09 20:06:36.406042 | controller | Context "minikube" modified. 2026-06-10 00:06:36.605338 | controller | ok: Runtime: 0:00:00.082890 2026-06-10 00:06:36.607164 | 2026-06-10 00:06:36.607220 | PLAY RECAP 2026-06-10 00:06:36.607276 | controller | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 00:06:36.607304 | 2026-06-10 00:06:36.819867 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/prepare-minikube.yaml@master] 2026-06-10 00:06:36.821016 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-test-dependencies.yaml@master] 2026-06-10 00:06:37.551241 | 2026-06-10 00:06:37.551439 | PLAY [controller] 2026-06-10 00:06:37.565340 | 2026-06-10 00:06:37.565497 | TASK [install-golang : Check Go version] 2026-06-10 00:06:38.091948 | controller | ok: ERROR (ignored) 2026-06-10 00:06:38.092279 | controller | { 2026-06-10 00:06:38.092327 | controller | "failed_when_result": false, 2026-06-10 00:06:38.092379 | controller | "msg": "[Errno 2] No such file or directory: b'go'", 2026-06-10 00:06:38.092439 | controller | "rc": 2 2026-06-10 00:06:38.092468 | controller | } 2026-06-10 00:06:38.099991 | 2026-06-10 00:06:38.100142 | TASK [install-golang : Ensure a clean install] 2026-06-10 00:06:38.528907 | controller | ok 2026-06-10 00:06:38.536485 | 2026-06-10 00:06:38.536810 | TASK [install-golang : Extract Go archive] 2026-06-10 00:06:53.852770 | controller | changed 2026-06-10 00:06:53.856744 | 2026-06-10 00:06:53.856814 | LOOP [install-golang : Create symlinks for Go binary] 2026-06-10 00:06:54.134509 | controller | changed: "go" 2026-06-10 00:06:54.381726 | controller | changed: "gofmt" 2026-06-10 00:06:54.396256 | 2026-06-10 00:06:54.396409 | TASK [Installing packages] 2026-06-10 00:06:55.455540 | controller | ok: Nothing to do 2026-06-10 00:06:55.461089 | 2026-06-10 00:06:55.461197 | TASK [Install pip packages] 2026-06-10 00:06:59.775103 | controller | changed 2026-06-10 00:06:59.777250 | 2026-06-10 00:06:59.777308 | PLAY RECAP 2026-06-10 00:06:59.777369 | controller | ok: 6 changed: 3 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 00:06:59.777399 | 2026-06-10 00:06:59.957032 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-test-dependencies.yaml@master] 2026-06-10 00:06:59.958560 | RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-deploy.yaml@master] 2026-06-10 00:07:00.685314 | 2026-06-10 00:07:00.685518 | PLAY [controller] 2026-06-10 00:07:00.696307 | 2026-06-10 00:07:00.696407 | TASK [Run ./hack/deploy.sh] 2026-06-09 20:07:01.125146 | controller | [+] Installing kubectl 2026-06-09 20:07:01.244915 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-09 20:07:01.244988 | controller | Dload Upload Total Spent Left Speed 2026-06-09 20:07:02.065621 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 56.7M 0 540k 0 0 4704k 0 0:00:12 --:--:-- 0:00:12 4663k 100 56.7M 100 56.7M 0 0 69.2M 0 --:--:-- --:--:-- --:--:-- 69.1M 2026-06-09 20:07:02.187985 | controller | go: downloading go1.25.11 (linux/amd64) 2026-06-09 20:07:10.340802 | controller | Error from server (NotFound): pods "gerrit-0" not found 2026-06-09 20:07:10.344049 | controller | [+] Setting up gerrit 2026-06-09 20:07:10.376924 | controller | go: downloading k8s.io/client-go v0.30.2 2026-06-09 20:07:10.378577 | controller | go: downloading github.com/spf13/cobra v1.8.1 2026-06-09 20:07:10.399272 | controller | go: downloading github.com/openshift/api v0.0.0-20240715171821-e9f09d21bcb5 2026-06-09 20:07:10.400723 | controller | go: downloading go.uber.org/zap v1.27.0 2026-06-09 20:07:10.401420 | controller | go: downloading k8s.io/api v0.30.2 2026-06-09 20:07:10.402324 | controller | go: downloading k8s.io/apimachinery v0.30.2 2026-06-09 20:07:11.543317 | controller | go: downloading k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 2026-06-09 20:07:11.547512 | controller | go: downloading sigs.k8s.io/controller-runtime v0.18.4 2026-06-09 20:07:11.814833 | controller | go: downloading gopkg.in/yaml.v3 v3.0.1 2026-06-09 20:07:11.835920 | controller | go: downloading sigs.k8s.io/yaml v1.4.0 2026-06-09 20:07:11.863158 | controller | go: downloading github.com/fatih/color v1.17.0 2026-06-09 20:07:11.863901 | controller | go: downloading github.com/go-sql-driver/mysql v1.8.1 2026-06-09 20:07:11.932956 | controller | go: downloading gopkg.in/ini.v1 v1.67.0 2026-06-09 20:07:11.934604 | controller | go: downloading k8s.io/kubectl v0.30.2 2026-06-09 20:07:11.966187 | controller | go: downloading github.com/spf13/pflag v1.0.5 2026-06-09 20:07:11.968267 | controller | go: downloading go.uber.org/multierr v1.11.0 2026-06-09 20:07:11.996498 | controller | go: downloading github.com/gogo/protobuf v1.3.2 2026-06-09 20:07:12.007030 | controller | go: downloading github.com/google/gofuzz v1.2.0 2026-06-09 20:07:12.018326 | controller | go: downloading k8s.io/klog/v2 v2.130.1 2026-06-09 20:07:12.090893 | controller | go: downloading github.com/go-logr/logr v1.4.2 2026-06-09 20:07:12.105629 | controller | go: downloading github.com/go-logr/zapr v1.3.0 2026-06-09 20:07:12.118401 | controller | go: downloading golang.org/x/exp v0.0.0-20240716175740-e3f259677ff7 2026-06-09 20:07:12.156806 | controller | go: downloading gopkg.in/yaml.v2 v2.4.0 2026-06-09 20:07:12.186166 | controller | go: downloading github.com/google/uuid v1.6.0 2026-06-09 20:07:12.243826 | controller | go: downloading golang.org/x/crypto v0.52.0 2026-06-09 20:07:12.246783 | controller | go: downloading k8s.io/apiextensions-apiserver v0.30.2 2026-06-09 20:07:12.385792 | controller | go: downloading github.com/imdario/mergo v0.3.16 2026-06-09 20:07:12.388271 | controller | go: downloading golang.org/x/term v0.43.0 2026-06-09 20:07:12.532016 | controller | go: downloading golang.org/x/net v0.55.0 2026-06-09 20:07:12.586037 | controller | go: downloading gopkg.in/inf.v0 v0.9.1 2026-06-09 20:07:12.586087 | controller | go: downloading github.com/gorilla/websocket v1.5.1 2026-06-09 20:07:12.595484 | controller | go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.4.1 2026-06-09 20:07:12.596098 | controller | go: downloading github.com/evanphx/json-patch/v5 v5.9.0 2026-06-09 20:07:12.628763 | controller | go: downloading github.com/mattn/go-colorable v0.1.13 2026-06-09 20:07:12.683492 | controller | go: downloading github.com/evanphx/json-patch v5.9.0+incompatible 2026-06-09 20:07:12.721682 | controller | go: downloading github.com/mattn/go-isatty v0.0.20 2026-06-09 20:07:12.733480 | controller | go: downloading filippo.io/edwards25519 v1.1.1 2026-06-09 20:07:12.765874 | controller | go: downloading golang.org/x/sys v0.45.0 2026-06-09 20:07:12.787568 | controller | go: downloading golang.org/x/oauth2 v0.27.0 2026-06-09 20:07:12.788636 | controller | go: downloading golang.org/x/time v0.5.0 2026-06-09 20:07:12.818589 | controller | go: downloading github.com/golang/protobuf v1.5.4 2026-06-09 20:07:12.856132 | controller | go: downloading github.com/google/gnostic-models v0.6.8 2026-06-09 20:07:12.875070 | controller | go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd 2026-06-09 20:07:12.928710 | controller | go: downloading k8s.io/kube-openapi v0.0.0-20240430033511-f0e62f92d13f 2026-06-09 20:07:12.983036 | controller | go: downloading gomodules.xyz/jsonpatch/v2 v2.4.0 2026-06-09 20:07:13.015656 | controller | go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da 2026-06-09 20:07:13.036625 | controller | go: downloading github.com/prometheus/client_golang v1.18.0 2026-06-09 20:07:13.118900 | controller | go: downloading github.com/json-iterator/go v1.1.12 2026-06-09 20:07:13.140883 | controller | go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc 2026-06-09 20:07:13.142658 | controller | go: downloading github.com/pkg/errors v0.9.1 2026-06-09 20:07:13.281648 | controller | go: downloading github.com/moby/spdystream v0.5.1 2026-06-09 20:07:13.344745 | controller | go: downloading google.golang.org/protobuf v1.34.1 2026-06-09 20:07:13.823279 | controller | go: downloading github.com/google/go-cmp v0.6.0 2026-06-09 20:07:13.823651 | controller | go: downloading github.com/prometheus/client_model v0.6.1 2026-06-09 20:07:13.823671 | controller | go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd 2026-06-09 20:07:13.823686 | controller | go: downloading github.com/modern-go/reflect2 v1.0.2 2026-06-09 20:07:13.823715 | controller | go: downloading github.com/prometheus/common v0.47.0 2026-06-09 20:07:13.823971 | controller | go: downloading github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f 2026-06-09 20:07:13.824155 | controller | go: downloading github.com/fsnotify/fsnotify v1.7.0 2026-06-09 20:07:13.829452 | controller | go: downloading github.com/go-openapi/jsonreference v0.21.0 2026-06-09 20:07:13.965762 | controller | go: downloading github.com/go-openapi/swag v0.23.0 2026-06-09 20:07:13.967345 | controller | go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 2026-06-09 20:07:13.969562 | controller | go: downloading github.com/beorn7/perks v1.0.1 2026-06-09 20:07:13.971197 | controller | go: downloading github.com/cespare/xxhash/v2 v2.3.0 2026-06-09 20:07:13.988355 | controller | go: downloading github.com/prometheus/procfs v0.15.0 2026-06-09 20:07:14.001190 | controller | go: downloading golang.org/x/text v0.37.0 2026-06-09 20:07:14.105363 | controller | go: downloading github.com/go-openapi/jsonpointer v0.21.0 2026-06-09 20:07:14.113689 | controller | go: downloading github.com/emicklei/go-restful/v3 v3.12.0 2026-06-09 20:07:14.117290 | controller | go: downloading github.com/mailru/easyjson v0.7.7 2026-06-09 20:07:14.260371 | controller | go: downloading github.com/josharian/intern v1.0.0 2026-06-09 20:07:49.006215 | controller | 2026-06-09T20:07:49-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:07:49.006286 | controller | 2026-06-09 20:07:49.022447 | controller | 2026-06-09T20:07:49-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:07:49.022523 | controller | 2026-06-09 20:07:49.153875 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "admin-ssh-key" in sf 2026-06-09 20:07:49.158149 | controller | 2026-06-09T20:07:49-04:00 INFO "admin-ssh-key" created 2026-06-09 20:07:49.652643 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "zuul-ssh-key" in sf 2026-06-09 20:07:49.658469 | controller | 2026-06-09T20:07:49-04:00 INFO "zuul-ssh-key" created 2026-06-09 20:07:49.661427 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "gerrit-admin-api-key" in sf 2026-06-09 20:07:49.665148 | controller | 2026-06-09T20:07:49-04:00 INFO "gerrit-admin-api-key" created 2026-06-09 20:07:49.667370 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "zuul-gerrit-api-key" in sf 2026-06-09 20:07:49.670846 | controller | 2026-06-09T20:07:49-04:00 INFO "zuul-gerrit-api-key" created 2026-06-09 20:07:49.672641 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "gerrit-httpd" in sf 2026-06-09 20:07:49.679130 | controller | 2026-06-09T20:07:49-04:00 INFO "gerrit-httpd" created 2026-06-09 20:07:49.682683 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "gerrit-sshd" in sf 2026-06-09 20:07:49.691991 | controller | 2026-06-09T20:07:49-04:00 INFO "gerrit-sshd" created 2026-06-09 20:07:49.694780 | controller | 2026-06-09T20:07:49-04:00 INFO managesf-resources 2026-06-09 20:07:49.694859 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "managesf-resources-config-map" in sf 2026-06-09 20:07:49.697207 | controller | 2026-06-09T20:07:49-04:00 INFO "managesf-resources-config-map" created 2026-06-09 20:07:49.699567 | controller | 2026-06-09T20:07:49-04:00 INFO managesf-resources-tooling 2026-06-09 20:07:49.699631 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "managesf-resources-tooling-config-map" in sf 2026-06-09 20:07:49.701949 | controller | 2026-06-09T20:07:49-04:00 INFO "managesf-resources-tooling-config-map" created 2026-06-09 20:07:49.707411 | controller | 2026-06-09T20:07:49-04:00 INFO Creating "gerrit" in sf 2026-06-09 20:07:49.718071 | controller | 2026-06-09T20:07:49-04:00 INFO "gerrit" created 2026-06-09 20:07:49.720482 | controller | 2026-06-09T20:07:49-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-09 20:07:59.732563 | controller | 2026-06-09T20:07:59-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-09 20:08:09.744070 | controller | 2026-06-09T20:08:09-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-09 20:08:19.749850 | controller | 2026-06-09T20:08:19-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-09 20:08:29.751502 | controller | 2026-06-09T20:08:29-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-09 20:08:39.756042 | controller | 2026-06-09T20:08:39-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-09 20:08:49.760538 | controller | 2026-06-09T20:08:49-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-09 20:08:59.773811 | controller | 2026-06-09T20:08:59-04:00 INFO Creating "post-init" in sf 2026-06-09 20:08:59.777653 | controller | 2026-06-09T20:08:59-04:00 INFO "post-init" created 2026-06-09 20:09:19.869665 | controller | 2026-06-09T20:09:19-04:00 INFO Creating "self-signed-cert" in sf 2026-06-09 20:09:19.873773 | controller | 2026-06-09T20:09:19-04:00 INFO "self-signed-cert" created 2026-06-09 20:09:19.878010 | controller | 2026-06-09T20:09:19-04:00 INFO Creating "gerrit-ingress" in sf 2026-06-09 20:09:19.893484 | controller | 2026-06-09T20:09:19-04:00 INFO "gerrit-ingress" created 2026-06-09 20:09:19.899518 | controller | 2026-06-09T20:09:19-04:00 INFO Creating "sf-ingress" in sf 2026-06-09 20:09:19.907848 | controller | 2026-06-09T20:09:19-04:00 INFO "sf-ingress" created 2026-06-09 20:09:19.911329 | controller | 2026-06-09T20:09:19-04:00 INFO Waiting for gerrit-ingress ... [attempt 1/25] 2026-06-09 20:09:26.924046 | controller | 2026-06-09T20:09:26-04:00 INFO Updating /etc/hosts 2026-06-09 20:09:26.924216 | controller | 2026-06-09T20:09:26-04:00 INFO Making sure Gerrit is up and ready... 2026-06-09 20:09:26.924267 | controller | 2026-06-09T20:09:26-04:00 INFO Querying Gerrit projects endpoint... [attempt 1/10] 2026-06-09 20:09:26.980454 | controller | 2026-06-09T20:09:26-04:00 INFO Gerrit is up and available 2026-06-09 20:09:26.980527 | controller | 2026-06-09T20:09:26-04:00 INFO Cloning config... 2026-06-09 20:09:26.983991 | controller | 2026-06-09T20:09:26-04:00 INFO Cloning repo https://admin:079edf87-291e-4fd3-8b5d-5ef0b88841a2@gerrit.sfop.me/a/config in deploy/config 2026-06-09 20:09:27.216595 | controller | 2026-06-09T20:09:27-04:00 INFO Configuring local repository for commits... 2026-06-09 20:09:27.231762 | controller | 2026-06-09T20:09:27-04:00 INFO Cloning demo-tenant-config... 2026-06-09 20:09:27.235187 | controller | 2026-06-09T20:09:27-04:00 INFO Cloning repo https://admin:079edf87-291e-4fd3-8b5d-5ef0b88841a2@gerrit.sfop.me/a/demo-tenant-config in deploy/demo-tenant-config 2026-06-09 20:09:27.344576 | controller | 2026-06-09T20:09:27-04:00 INFO Configuring local repository for commits... 2026-06-09 20:09:27.359470 | controller | 2026-06-09T20:09:27-04:00 INFO Cloning demo-project... 2026-06-09 20:09:27.362579 | controller | 2026-06-09T20:09:27-04:00 INFO Cloning repo https://admin:079edf87-291e-4fd3-8b5d-5ef0b88841a2@gerrit.sfop.me/a/demo-project in deploy/demo-project 2026-06-09 20:09:27.469185 | controller | 2026-06-09T20:09:27-04:00 INFO Configuring local repository for commits... 2026-06-09 20:09:27.482859 | controller | 2026-06-09T20:09:27-04:00 INFO Initialize demo-tenant's pipelines and jobs... 2026-06-09 20:09:27.494209 | controller | 2026-06-09T20:09:27-04:00 INFO Pushing local repo state to origin... 2026-06-09 20:09:27.703867 | controller | 2026-06-09T20:09:27-04:00 INFO Creating or updating tenant demo-tenant... 2026-06-09 20:09:27.712307 | controller | 2026-06-09T20:09:27-04:00 INFO Pushing local repo state to origin... 2026-06-09 20:09:27.867779 | controller | 2026-06-09T20:09:27-04:00 INFO Initialize demo-project's pipelines and jobs... 2026-06-09 20:09:27.879538 | controller | 2026-06-09T20:09:27-04:00 INFO Pushing local repo state to origin... 2026-06-09 20:09:28.072387 | controller | 2026-06-09T20:09:28-04:00 INFO Applying CRDs (did you run "make manifests" first?) 2026-06-09 20:09:28.432193 | controller | [+] Deploying sf-operator 2026-06-09 20:09:28.790100 | controller | 2026-06-09T20:09:28-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:09:28.790152 | controller | 2026-06-09 20:09:28.799919 | controller | 2026-06-09T20:09:28-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:09:28.799967 | controller | 2026-06-09 20:09:28.819026 | controller | 2026-06-09T20:09:28-04:00 INFO Deleting object, name: zookeeper 2026-06-09 20:09:28.822689 | controller | 2026-06-09T20:09:28-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:09:28.826505 | controller | 2026-06-09T20:09:28-04:00 INFO Creating secret, name: zuul-keystore-password 2026-06-09 20:09:28.826547 | controller | 2026-06-09T20:09:28-04:00 INFO Creating object, name: zuul-keystore-password 2026-06-09 20:09:28.829542 | controller | 2026-06-09T20:09:28-04:00 INFO Creating secret, name: zuul-auth-secret 2026-06-09 20:09:28.829567 | controller | 2026-06-09T20:09:28-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-09 20:09:28.832375 | controller | 2026-06-09T20:09:28-04:00 INFO Creating config map name: zuul-scheduler-tooling-config-map 2026-06-09 20:09:28.832391 | controller | 2026-06-09T20:09:28-04:00 INFO Creating object, name: zuul-scheduler-tooling-config-map 2026-06-09 20:09:28.837794 | controller | 2026-06-09T20:09:28-04:00 INFO Creating config map name: git-server-pi-config-map 2026-06-09 20:09:28.837813 | controller | 2026-06-09T20:09:28-04:00 INFO Creating object, name: git-server-pi-config-map 2026-06-09 20:09:28.842258 | controller | 2026-06-09T20:09:28-04:00 INFO Creating object, name: git-server 2026-06-09 20:09:28.846865 | controller | 2026-06-09T20:09:28-04:00 INFO Creating secret, name: mariadb-root-password 2026-06-09 20:09:28.846885 | controller | 2026-06-09T20:09:28-04:00 INFO Creating object, name: mariadb-root-password 2026-06-09 20:09:28.850735 | controller | 2026-06-09T20:09:28-04:00 INFO Creating secret, name: mariadb-config-secrets 2026-06-09 20:09:28.850757 | controller | 2026-06-09T20:09:28-04:00 INFO Creating object, name: mariadb-config-secrets 2026-06-09 20:09:29.214690 | controller | 2026-06-09T20:09:29-04:00 INFO Creating secret, name: mariadb-initdb-secrets 2026-06-09 20:09:29.214767 | controller | 2026-06-09T20:09:29-04:00 INFO Creating object, name: mariadb-initdb-secrets 2026-06-09 20:09:29.419737 | controller | 2026-06-09T20:09:29-04:00 INFO Creating object, name: mariadb 2026-06-09 20:09:29.615947 | controller | 2026-06-09T20:09:29-04:00 INFO Creating ssh key, name: logserver-keys 2026-06-09 20:09:32.147375 | controller | 2026-06-09T20:09:32-04:00 INFO Creating object, name: logserver-keys 2026-06-09 20:09:32.152724 | controller | 2026-06-09T20:09:32-04:00 INFO Creating ssh key, name: logserver-uploader-keys 2026-06-09 20:09:32.317385 | controller | 2026-06-09T20:09:32-04:00 INFO Creating object, name: logserver-uploader-keys 2026-06-09 20:09:32.325456 | controller | 2026-06-09T20:09:32-04:00 INFO Creating ssh key, name: logserver-uploader-spare-keys 2026-06-09 20:09:33.605281 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: logserver-uploader-spare-keys 2026-06-09 20:09:33.609818 | controller | 2026-06-09T20:09:33-04:00 INFO Creating config map name: logserver-config-map 2026-06-09 20:09:33.609857 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: logserver-config-map 2026-06-09 20:09:33.613682 | controller | 2026-06-09T20:09:33-04:00 INFO Creating service, name: logserver 2026-06-09 20:09:33.613712 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: logserver 2026-06-09 20:09:33.622008 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: logserver 2026-06-09 20:09:33.628414 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: gateway 2026-06-09 20:09:33.635172 | controller | 2026-06-09T20:09:33-04:00 INFO Creating config map name: gateway-config-map 2026-06-09 20:09:33.635244 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: gateway-config-map 2026-06-09 20:09:33.639106 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: gateway 2026-06-09 20:09:33.647293 | controller | 2026-06-09T20:09:33-04:00 INFO Creating service, name: hound-search 2026-06-09 20:09:33.647340 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: hound-search 2026-06-09 20:09:33.657350 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: hound-search 2026-06-09 20:09:33.663036 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: logjuicer-pvc 2026-06-09 20:09:33.670282 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: logjuicer 2026-06-09 20:09:33.688279 | controller | 2026-06-09T20:09:33-04:00 INFO Creating object, name: logjuicer 2026-06-09 20:09:33.694484 | controller | 2026-06-09T20:09:33-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-09 20:09:33.696403 | controller | 2026-06-09T20:09:33-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-09 20:09:34.827515 | controller | 2026-06-09T20:09:34-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-09 20:09:35.841617 | controller | 2026-06-09T20:09:35-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-09 20:09:35.846952 | controller | 2026-06-09T20:09:35-04:00 INFO Creating config map name: zookeeper-pi-config-map 2026-06-09 20:09:35.846990 | controller | 2026-06-09T20:09:35-04:00 INFO Creating object, name: zookeeper-pi-config-map 2026-06-09 20:09:35.852791 | controller | 2026-06-09T20:09:35-04:00 INFO Creating service, name: zookeeper-headless 2026-06-09 20:09:35.852856 | controller | 2026-06-09T20:09:35-04:00 INFO Creating object, name: zookeeper-headless 2026-06-09 20:09:35.858020 | controller | 2026-06-09T20:09:35-04:00 INFO Creating service, name: zookeeper 2026-06-09 20:09:35.858058 | controller | 2026-06-09T20:09:35-04:00 INFO Creating object, name: zookeeper 2026-06-09 20:09:35.865549 | controller | 2026-06-09T20:09:35-04:00 INFO Creating object, name: zookeeper 2026-06-09 20:09:35.868800 | controller | 2026-06-09T20:09:35-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-09 20:09:35.868833 | controller | 2026-06-09T20:09:35-04:00 INFO 2026-06-09 20:09:35.868843 | controller | - Config: Waiting ... 2026-06-09 20:09:35.868851 | controller | - Gateway: Waiting ... 2026-06-09 20:09:35.868859 | controller | - GitServer: Waiting ... 2026-06-09 20:09:35.868866 | controller | - HoundSearch: Waiting ... 2026-06-09 20:09:35.868874 | controller | - LogJuicer: Waiting ... 2026-06-09 20:09:35.868882 | controller | - Logserver: Waiting ... 2026-06-09 20:09:35.868889 | controller | - MariaDB: Waiting ... 2026-06-09 20:09:35.868897 | controller | - Zookeeper: Waiting ... 2026-06-09 20:09:35.868904 | controller | 2026-06-09 20:09:35.868914 | controller | 2026-06-09T20:09:35-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:09:40.877553 | controller | 2026-06-09T20:09:40-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:09:40.890550 | controller | 2026-06-09T20:09:40-04:00 INFO Creating service, name: git-server 2026-06-09 20:09:40.890587 | controller | 2026-06-09T20:09:40-04:00 INFO Creating object, name: git-server 2026-06-09 20:09:40.898065 | controller | 2026-06-09T20:09:40-04:00 INFO Creating service, name: git-server-rw 2026-06-09 20:09:40.898132 | controller | 2026-06-09T20:09:40-04:00 INFO Creating object, name: git-server-rw 2026-06-09 20:09:40.907503 | controller | 2026-06-09T20:09:40-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:09:40.907568 | controller | 2026-06-09T20:09:40-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:09:45.915530 | controller | 2026-06-09T20:09:45-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:09:45.915591 | controller | 2026-06-09T20:09:45-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:09:50.926162 | controller | 2026-06-09T20:09:50-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:09:50.926220 | controller | 2026-06-09T20:09:50-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:09:55.934426 | controller | 2026-06-09T20:09:55-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:09:55.947290 | controller | 2026-06-09T20:09:55-04:00 INFO Creating service, name: mariadb 2026-06-09 20:09:55.947361 | controller | 2026-06-09T20:09:55-04:00 INFO Creating object, name: mariadb 2026-06-09 20:09:55.954922 | controller | 2026-06-09T20:09:55-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:09:55.954981 | controller | 2026-06-09T20:09:55-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 20:10:01.045856 | controller | 2026-06-09T20:10:01-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:10:01.045943 | controller | 2026-06-09T20:10:01-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 20:10:06.056333 | controller | 2026-06-09T20:10:06-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:10:06.056430 | controller | 2026-06-09T20:10:06-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 20:10:11.063802 | controller | 2026-06-09T20:10:11-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:10:11.063901 | controller | 2026-06-09T20:10:11-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 20:10:16.075735 | controller | 2026-06-09T20:10:16-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:10:16.075810 | controller | 2026-06-09T20:10:16-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 20:10:21.086964 | controller | 2026-06-09T20:10:21-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:10:21.087092 | controller | 2026-06-09T20:10:21-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 20:10:26.094457 | controller | 2026-06-09T20:10:26-04:00 INFO Statefulset "mariadb" not ready after 30s 2026-06-09 20:10:26.120476 | controller | 2026-06-09T20:10:26-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:10:26.120557 | controller | 2026-06-09T20:10:26-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:10:31.130171 | controller | 2026-06-09T20:10:31-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:10:31.130284 | controller | 2026-06-09T20:10:31-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:10:36.145215 | controller | 2026-06-09T20:10:36-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:10:36.150368 | controller | 2026-06-09T20:10:36-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:10:36.247839 | controller | 2026-06-09T20:10:36-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:10:36.254708 | controller | 2026-06-09T20:10:36-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:10:36.254725 | controller | 2026-06-09T20:10:36-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:10:36.260179 | controller | 2026-06-09T20:10:36-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:10:36.260194 | controller | 2026-06-09T20:10:36-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:10:41.265607 | controller | 2026-06-09T20:10:41-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:10:41.265697 | controller | 2026-06-09T20:10:41-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:10:46.276196 | controller | 2026-06-09T20:10:46-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:10:46.276311 | controller | 2026-06-09T20:10:46-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:10:51.286261 | controller | 2026-06-09T20:10:51-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:10:51.286345 | controller | 2026-06-09T20:10:51-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:10:56.293894 | controller | 2026-06-09T20:10:56-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:10:56.293992 | controller | 2026-06-09T20:10:56-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:01.301617 | controller | 2026-06-09T20:11:01-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:01.301712 | controller | 2026-06-09T20:11:01-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:06.309824 | controller | 2026-06-09T20:11:06-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-09 20:11:06.320449 | controller | 2026-06-09T20:11:06-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:11:06.320500 | controller | 2026-06-09T20:11:06-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:11:06.339562 | controller | 2026-06-09T20:11:06-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:11:06.342836 | controller | 2026-06-09T20:11:06-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 20:11:06.342875 | controller | 2026-06-09T20:11:06-04:00 INFO 2026-06-09 20:11:06.342886 | controller | - Config: Waiting ... 2026-06-09 20:11:06.342895 | controller | - Gateway: OK 2026-06-09 20:11:06.342903 | controller | - GitServer: OK 2026-06-09 20:11:06.342911 | controller | - HoundSearch: Waiting ... 2026-06-09 20:11:06.342919 | controller | - LogJuicer: OK 2026-06-09 20:11:06.342927 | controller | - Logserver: OK 2026-06-09 20:11:06.342935 | controller | - MariaDB: Waiting ... 2026-06-09 20:11:06.342943 | controller | - Zookeeper: OK 2026-06-09 20:11:06.342951 | controller | 2026-06-09 20:11:06.342964 | controller | 2026-06-09T20:11:06-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:11:11.349137 | controller | 2026-06-09T20:11:11-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:11:11.381110 | controller | 2026-06-09T20:11:11-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:11:11.400366 | controller | 2026-06-09T20:11:11-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:11:11.404807 | controller | 2026-06-09T20:11:11-04:00 INFO Creating object, name: zuul-db-connection 2026-06-09 20:11:11.412549 | controller | 2026-06-09T20:11:11-04:00 INFO Creating object, name: zuul-db-provision 2026-06-09 20:11:12.158536 | controller | 2026-06-09T20:11:12-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:11:12.176258 | controller | 2026-06-09T20:11:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:11:12.176327 | controller | 2026-06-09T20:11:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:11:12.190298 | controller | 2026-06-09T20:11:12-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:12.190347 | controller | 2026-06-09T20:11:12-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:17.198827 | controller | 2026-06-09T20:11:17-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:17.198898 | controller | 2026-06-09T20:11:17-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:22.209149 | controller | 2026-06-09T20:11:22-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:22.209272 | controller | 2026-06-09T20:11:22-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:27.217851 | controller | 2026-06-09T20:11:27-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:27.217960 | controller | 2026-06-09T20:11:27-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:32.228460 | controller | 2026-06-09T20:11:32-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:32.228544 | controller | 2026-06-09T20:11:32-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:37.235291 | controller | 2026-06-09T20:11:37-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:37.235367 | controller | 2026-06-09T20:11:37-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:42.241005 | controller | 2026-06-09T20:11:42-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-09 20:11:42.250079 | controller | 2026-06-09T20:11:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:11:42.250121 | controller | 2026-06-09T20:11:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:11:42.267116 | controller | 2026-06-09T20:11:42-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:11:42.270252 | controller | 2026-06-09T20:11:42-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 20:11:42.270311 | controller | 2026-06-09T20:11:42-04:00 INFO 2026-06-09 20:11:42.270323 | controller | - Config: Waiting ... 2026-06-09 20:11:42.270332 | controller | - Gateway: OK 2026-06-09 20:11:42.270340 | controller | - GitServer: OK 2026-06-09 20:11:42.270347 | controller | - HoundSearch: Waiting ... 2026-06-09 20:11:42.270355 | controller | - LogJuicer: OK 2026-06-09 20:11:42.270363 | controller | - Logserver: OK 2026-06-09 20:11:42.270370 | controller | - MariaDB: Waiting ... 2026-06-09 20:11:42.270378 | controller | - Zookeeper: OK 2026-06-09 20:11:42.270386 | controller | 2026-06-09 20:11:42.270394 | controller | 2026-06-09T20:11:42-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:11:47.286361 | controller | 2026-06-09T20:11:47-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:11:47.315756 | controller | 2026-06-09T20:11:47-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:11:47.347372 | controller | 2026-06-09T20:11:47-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:11:47.888291 | controller | 2026-06-09T20:11:47-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:11:47.899195 | controller | 2026-06-09T20:11:47-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:11:47.899256 | controller | 2026-06-09T20:11:47-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:11:47.908656 | controller | 2026-06-09T20:11:47-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:47.908692 | controller | 2026-06-09T20:11:47-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:52.918277 | controller | 2026-06-09T20:11:52-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:52.918373 | controller | 2026-06-09T20:11:52-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:11:57.927666 | controller | 2026-06-09T20:11:57-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:11:57.927741 | controller | 2026-06-09T20:11:57-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:02.937651 | controller | 2026-06-09T20:12:02-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:02.937716 | controller | 2026-06-09T20:12:02-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:07.947161 | controller | 2026-06-09T20:12:07-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:07.947299 | controller | 2026-06-09T20:12:07-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:12.956709 | controller | 2026-06-09T20:12:12-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:12.956782 | controller | 2026-06-09T20:12:12-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:17.959917 | controller | 2026-06-09T20:12:17-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-09 20:12:17.970215 | controller | 2026-06-09T20:12:17-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:12:17.970356 | controller | 2026-06-09T20:12:17-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:12:17.986163 | controller | 2026-06-09T20:12:17-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:12:17.988674 | controller | 2026-06-09T20:12:17-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:12:17.989773 | controller | 2026-06-09T20:12:17-04:00 INFO Creating ssh key, name: zuul-spare-ssh-key 2026-06-09 20:12:18.790295 | controller | 2026-06-09T20:12:18-04:00 INFO Creating object, name: zuul-spare-ssh-key 2026-06-09 20:12:18.800070 | controller | 2026-06-09T20:12:18-04:00 INFO Creating config map name: np-statsd-config-map 2026-06-09 20:12:18.800122 | controller | 2026-06-09T20:12:18-04:00 INFO Creating object, name: np-statsd-config-map 2026-06-09 20:12:18.806317 | controller | 2026-06-09T20:12:18-04:00 INFO Creating config map name: nodepool-tooling-config-map 2026-06-09 20:12:18.806408 | controller | 2026-06-09T20:12:18-04:00 INFO Creating object, name: nodepool-tooling-config-map 2026-06-09 20:12:18.813667 | controller | 2026-06-09T20:12:18-04:00 INFO Creating config map name: nodepool-launcher-extra-config-config-map 2026-06-09 20:12:18.813747 | controller | 2026-06-09T20:12:18-04:00 INFO Creating object, name: nodepool-launcher-extra-config-config-map 2026-06-09 20:12:18.829930 | controller | 2026-06-09T20:12:18-04:00 INFO Creating object, name: nodepool-launcher 2026-06-09 20:12:18.842865 | controller | 2026-06-09T20:12:18-04:00 INFO Creating ssh key, name: nodepool-builder-ssh-key 2026-06-09 20:12:19.362545 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: nodepool-builder-ssh-key 2026-06-09 20:12:19.371208 | controller | 2026-06-09T20:12:19-04:00 INFO Creating config map name: nodepool-builder-extra-config-config-map 2026-06-09 20:12:19.371259 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: nodepool-builder-extra-config-config-map 2026-06-09 20:12:19.382878 | controller | 2026-06-09T20:12:19-04:00 INFO Creating service, name: nodepool-builder 2026-06-09 20:12:19.382931 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: nodepool-builder 2026-06-09 20:12:19.393394 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: nodepool-builder 2026-06-09 20:12:19.413611 | controller | 2026-06-09T20:12:19-04:00 INFO Creating secret, name: zuul-config 2026-06-09 20:12:19.413679 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: zuul-config 2026-06-09 20:12:19.419497 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:12:19.429543 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: zuul-executor-headless 2026-06-09 20:12:19.441716 | controller | 2026-06-09T20:12:19-04:00 INFO Creating config map name: zuul-extra-config-map 2026-06-09 20:12:19.441793 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: zuul-extra-config-map 2026-06-09 20:12:19.801622 | controller | 2026-06-09T20:12:19-04:00 INFO Creating config map name: zuul-statsd-config-map 2026-06-09 20:12:19.801700 | controller | 2026-06-09T20:12:19-04:00 INFO Creating object, name: zuul-statsd-config-map 2026-06-09 20:12:20.200514 | controller | 2026-06-09T20:12:20-04:00 INFO Creating config map name: zuul-logging-config-map 2026-06-09 20:12:20.200600 | controller | 2026-06-09T20:12:20-04:00 INFO Creating object, name: zuul-logging-config-map 2026-06-09 20:12:20.404479 | controller | 2026-06-09T20:12:20-04:00 INFO Creating object, name: zuul-scheduler 2026-06-09 20:12:21.203553 | controller | 2026-06-09T20:12:21-04:00 INFO Creating object, name: zuul-executor 2026-06-09 20:12:22.003169 | controller | 2026-06-09T20:12:22-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:12:22.803322 | controller | 2026-06-09T20:12:22-04:00 INFO Creating object, name: zuul-merger 2026-06-09 20:12:22.808847 | controller | 2026-06-09T20:12:22-04:00 INFO 2026-06-09 20:12:22.808886 | controller | - Config: Waiting ... 2026-06-09 20:12:22.808897 | controller | - Gateway: OK 2026-06-09 20:12:22.808907 | controller | - GitServer: OK 2026-06-09 20:12:22.808916 | controller | - HoundSearch: Waiting ... 2026-06-09 20:12:22.808924 | controller | - LogJuicer: OK 2026-06-09 20:12:22.808933 | controller | - Logserver: OK 2026-06-09 20:12:22.808941 | controller | - MariaDB: OK 2026-06-09 20:12:22.808951 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:12:22.808973 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:12:22.808982 | controller | - Zookeeper: OK 2026-06-09 20:12:22.808990 | controller | - Zuul: Waiting ... 2026-06-09 20:12:22.808999 | controller | - |_ Config Secrets: OK 2026-06-09 20:12:22.809008 | controller | - |_ Executor: Waiting ... 2026-06-09 20:12:22.809016 | controller | - |_ Merger: Waiting ... 2026-06-09 20:12:22.809025 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:12:22.809033 | controller | - |_ Web: Waiting ... 2026-06-09 20:12:22.809042 | controller | 2026-06-09 20:12:22.809051 | controller | 2026-06-09T20:12:22-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-09 20:12:27.817766 | controller | 2026-06-09T20:12:27-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:12:27.840687 | controller | 2026-06-09T20:12:27-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:12:27.855527 | controller | 2026-06-09T20:12:27-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:12:28.426295 | controller | 2026-06-09T20:12:28-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:12:28.437404 | controller | 2026-06-09T20:12:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:12:28.437456 | controller | 2026-06-09T20:12:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:12:28.444455 | controller | 2026-06-09T20:12:28-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:28.444486 | controller | 2026-06-09T20:12:28-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:33.450629 | controller | 2026-06-09T20:12:33-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:33.450699 | controller | 2026-06-09T20:12:33-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:38.458552 | controller | 2026-06-09T20:12:38-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:38.458635 | controller | 2026-06-09T20:12:38-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:43.468356 | controller | 2026-06-09T20:12:43-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:43.468409 | controller | 2026-06-09T20:12:43-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:48.482086 | controller | 2026-06-09T20:12:48-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:48.482207 | controller | 2026-06-09T20:12:48-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:53.490636 | controller | 2026-06-09T20:12:53-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:12:53.491005 | controller | 2026-06-09T20:12:53-04:00 INFO statefulset rollout not ok, name: hound-search 2026-06-09 20:12:58.499730 | controller | 2026-06-09T20:12:58-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-09 20:12:58.510280 | controller | 2026-06-09T20:12:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:12:58.510374 | controller | 2026-06-09T20:12:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:12:58.524084 | controller | 2026-06-09T20:12:58-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:12:58.526604 | controller | 2026-06-09T20:12:58-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:12:58.542103 | controller | 2026-06-09T20:12:58-04:00 INFO Creating object, name: nodepool-launcher 2026-06-09 20:12:58.547275 | controller | 2026-06-09T20:12:58-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:12:58.547324 | controller | 2026-06-09T20:12:58-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:12:58.559391 | controller | 2026-06-09T20:12:58-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:12:58.559438 | controller | 2026-06-09T20:12:58-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:12:59.115761 | controller | 2026-06-09T20:12:59-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:12:59.115840 | controller | 2026-06-09T20:12:59-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:13:04.123885 | controller | 2026-06-09T20:13:04-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:13:04.123964 | controller | 2026-06-09T20:13:04-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:13:09.134700 | controller | 2026-06-09T20:13:09-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:13:09.134760 | controller | 2026-06-09T20:13:09-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:13:14.146668 | controller | 2026-06-09T20:13:14-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:13:14.146733 | controller | 2026-06-09T20:13:14-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:13:19.158188 | controller | 2026-06-09T20:13:19-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:13:19.158299 | controller | 2026-06-09T20:13:19-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:13:24.168279 | controller | 2026-06-09T20:13:24-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:13:24.168411 | controller | 2026-06-09T20:13:24-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:13:29.172941 | controller | 2026-06-09T20:13:29-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:13:29.195087 | controller | 2026-06-09T20:13:29-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:13:29.195210 | controller | 2026-06-09T20:13:29-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:13:34.202760 | controller | 2026-06-09T20:13:34-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:13:34.202906 | controller | 2026-06-09T20:13:34-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:13:39.217589 | controller | 2026-06-09T20:13:39-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:13:39.217739 | controller | 2026-06-09T20:13:39-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:13:44.227900 | controller | 2026-06-09T20:13:44-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:13:44.228451 | controller | 2026-06-09T20:13:44-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:13:49.250925 | controller | 2026-06-09T20:13:49-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:13:49.251009 | controller | 2026-06-09T20:13:49-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:13:54.257151 | controller | 2026-06-09T20:13:54-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:13:54.257291 | controller | 2026-06-09T20:13:54-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:13:59.263761 | controller | 2026-06-09T20:13:59-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:13:59.264323 | controller | 2026-06-09T20:13:59-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:14:04.271999 | controller | 2026-06-09T20:14:04-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:14:04.272106 | controller | 2026-06-09T20:14:04-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:14:09.282037 | controller | 2026-06-09T20:14:09-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:14:09.282103 | controller | 2026-06-09T20:14:09-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 20:14:14.288662 | controller | 2026-06-09T20:14:14-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:14:14.300739 | controller | 2026-06-09T20:14:14-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:14:14.300801 | controller | 2026-06-09T20:14:14-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:14:14.312632 | controller | 2026-06-09T20:14:14-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:14:14.312684 | controller | 2026-06-09T20:14:14-04:00 INFO statefulset rollout not ok, name: zuul-merger 2026-06-09 20:14:19.316825 | controller | 2026-06-09T20:14:19-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:14:19.318616 | controller | 2026-06-09T20:14:19-04:00 INFO 2026-06-09 20:14:19.318632 | controller | - Config: Waiting ... 2026-06-09 20:14:19.318638 | controller | - Gateway: OK 2026-06-09 20:14:19.318643 | controller | - GitServer: OK 2026-06-09 20:14:19.318648 | controller | - HoundSearch: Waiting ... 2026-06-09 20:14:19.318653 | controller | - LogJuicer: OK 2026-06-09 20:14:19.318658 | controller | - Logserver: OK 2026-06-09 20:14:19.318663 | controller | - MariaDB: OK 2026-06-09 20:14:19.318668 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:14:19.318673 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:14:19.318677 | controller | - Zookeeper: OK 2026-06-09 20:14:19.318682 | controller | - Zuul: Waiting ... 2026-06-09 20:14:19.318687 | controller | - |_ Config Secrets: OK 2026-06-09 20:14:19.318692 | controller | - |_ Executor: OK 2026-06-09 20:14:19.318697 | controller | - |_ Merger: OK 2026-06-09 20:14:19.318702 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:14:19.318707 | controller | - |_ Web: Waiting ... 2026-06-09 20:14:19.318712 | controller | 2026-06-09 20:14:19.318717 | controller | 2026-06-09T20:14:19-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-09 20:14:24.327925 | controller | 2026-06-09T20:14:24-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:14:24.357567 | controller | 2026-06-09T20:14:24-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:14:24.376336 | controller | 2026-06-09T20:14:24-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:14:24.935779 | controller | 2026-06-09T20:14:24-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:14:24.947660 | controller | 2026-06-09T20:14:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:14:24.947720 | controller | 2026-06-09T20:14:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:14:24.958273 | controller | 2026-06-09T20:14:24-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:14:24.972095 | controller | 2026-06-09T20:14:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:14:24.972177 | controller | 2026-06-09T20:14:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:14:25.341404 | controller | 2026-06-09T20:14:25-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:14:25.344073 | controller | 2026-06-09T20:14:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:14:25.744010 | controller | 2026-06-09T20:14:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:14:25.744077 | controller | 2026-06-09T20:14:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:14:25.935813 | controller | 2026-06-09T20:14:25-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:14:26.753444 | controller | 2026-06-09T20:14:26-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:14:26.758327 | controller | 2026-06-09T20:14:26-04:00 INFO Creating object, name: zuul-weeder 2026-06-09 20:14:26.767302 | controller | 2026-06-09T20:14:26-04:00 INFO Creating object, name: zuul-weeder 2026-06-09 20:14:26.937621 | controller | 2026-06-09T20:14:26-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:14:27.733006 | controller | 2026-06-09T20:14:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:14:27.733086 | controller | 2026-06-09T20:14:27-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:14:28.540258 | controller | 2026-06-09T20:14:28-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:14:28.543894 | controller | 2026-06-09T20:14:28-04:00 INFO 2026-06-09 20:14:28.543940 | controller | - Config: Waiting ... 2026-06-09 20:14:28.543955 | controller | - Gateway: OK 2026-06-09 20:14:28.543968 | controller | - GitServer: OK 2026-06-09 20:14:28.543979 | controller | - HoundSearch: OK 2026-06-09 20:14:28.543991 | controller | - LogJuicer: OK 2026-06-09 20:14:28.544002 | controller | - Logserver: OK 2026-06-09 20:14:28.544013 | controller | - MariaDB: OK 2026-06-09 20:14:28.544024 | controller | - NodePoolBuilder: OK 2026-06-09 20:14:28.544035 | controller | - NodePoolLauncher: OK 2026-06-09 20:14:28.544046 | controller | - Zookeeper: OK 2026-06-09 20:14:28.544057 | controller | - Zuul: Waiting ... 2026-06-09 20:14:28.544098 | controller | - |_ Config Secrets: OK 2026-06-09 20:14:28.544111 | controller | - |_ Executor: OK 2026-06-09 20:14:28.544122 | controller | - |_ Merger: OK 2026-06-09 20:14:28.544133 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:14:28.544144 | controller | - |_ Web: OK 2026-06-09 20:14:28.544155 | controller | 2026-06-09 20:14:28.544167 | controller | 2026-06-09T20:14:28-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-09 20:14:33.552368 | controller | 2026-06-09T20:14:33-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:14:33.574044 | controller | 2026-06-09T20:14:33-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:14:33.586328 | controller | 2026-06-09T20:14:33-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:14:34.158547 | controller | 2026-06-09T20:14:34-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:14:34.166846 | controller | 2026-06-09T20:14:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:14:34.166874 | controller | 2026-06-09T20:14:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:14:34.174354 | controller | 2026-06-09T20:14:34-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:14:34.183556 | controller | 2026-06-09T20:14:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:14:34.183582 | controller | 2026-06-09T20:14:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:14:34.562966 | controller | 2026-06-09T20:14:34-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:14:34.565082 | controller | 2026-06-09T20:14:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:14:34.962363 | controller | 2026-06-09T20:14:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:14:34.962428 | controller | 2026-06-09T20:14:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:14:35.163285 | controller | 2026-06-09T20:14:35-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:14:35.973830 | controller | 2026-06-09T20:14:35-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:14:35.981433 | controller | 2026-06-09T20:14:35-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:14:35.981487 | controller | 2026-06-09T20:14:35-04:00 INFO deployment rollout not ok, name: zuul-weeder 2026-06-09 20:14:36.161302 | controller | 2026-06-09T20:14:36-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:14:36.956084 | controller | 2026-06-09T20:14:36-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:14:36.956162 | controller | 2026-06-09T20:14:36-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:14:37.762091 | controller | 2026-06-09T20:14:37-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:14:37.764155 | controller | 2026-06-09T20:14:37-04:00 INFO 2026-06-09 20:14:37.764194 | controller | - Config: Waiting ... 2026-06-09 20:14:37.764205 | controller | - Gateway: OK 2026-06-09 20:14:37.764214 | controller | - GitServer: OK 2026-06-09 20:14:37.764223 | controller | - HoundSearch: OK 2026-06-09 20:14:37.764266 | controller | - LogJuicer: OK 2026-06-09 20:14:37.764275 | controller | - Logserver: OK 2026-06-09 20:14:37.764284 | controller | - MariaDB: OK 2026-06-09 20:14:37.764292 | controller | - NodePoolBuilder: OK 2026-06-09 20:14:37.764300 | controller | - NodePoolLauncher: OK 2026-06-09 20:14:37.764309 | controller | - Zookeeper: OK 2026-06-09 20:14:37.764317 | controller | - Zuul: Waiting ... 2026-06-09 20:14:37.764326 | controller | - |_ Config Secrets: OK 2026-06-09 20:14:37.764334 | controller | - |_ Executor: OK 2026-06-09 20:14:37.764342 | controller | - |_ Merger: OK 2026-06-09 20:14:37.764351 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:14:37.764360 | controller | - |_ Web: OK 2026-06-09 20:14:37.764368 | controller | 2026-06-09 20:14:37.764377 | controller | 2026-06-09T20:14:37-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-09 20:14:42.775146 | controller | 2026-06-09T20:14:42-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:14:42.806199 | controller | 2026-06-09T20:14:42-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:14:42.824576 | controller | 2026-06-09T20:14:42-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:14:43.382032 | controller | 2026-06-09T20:14:43-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:14:43.391483 | controller | 2026-06-09T20:14:43-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:14:43.391540 | controller | 2026-06-09T20:14:43-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:14:43.400721 | controller | 2026-06-09T20:14:43-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:14:43.417588 | controller | 2026-06-09T20:14:43-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:14:43.417717 | controller | 2026-06-09T20:14:43-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:14:43.810102 | controller | 2026-06-09T20:14:43-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:14:43.813461 | controller | 2026-06-09T20:14:43-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:14:44.195284 | controller | 2026-06-09T20:14:44-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:14:44.195361 | controller | 2026-06-09T20:14:44-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:14:44.392032 | controller | 2026-06-09T20:14:44-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:14:45.198401 | controller | 2026-06-09T20:14:45-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:14:45.207153 | controller | 2026-06-09T20:14:45-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:14:45.207189 | controller | 2026-06-09T20:14:45-04:00 INFO deployment rollout not ok, name: zuul-weeder 2026-06-09 20:14:45.386067 | controller | 2026-06-09T20:14:45-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:14:46.178712 | controller | 2026-06-09T20:14:46-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:14:46.178781 | controller | 2026-06-09T20:14:46-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:14:46.986959 | controller | 2026-06-09T20:14:46-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:14:46.990860 | controller | 2026-06-09T20:14:46-04:00 INFO 2026-06-09 20:14:46.990905 | controller | - Config: Waiting ... 2026-06-09 20:14:46.990916 | controller | - Gateway: OK 2026-06-09 20:14:46.990925 | controller | - GitServer: OK 2026-06-09 20:14:46.990933 | controller | - HoundSearch: OK 2026-06-09 20:14:46.990942 | controller | - LogJuicer: OK 2026-06-09 20:14:46.990950 | controller | - Logserver: OK 2026-06-09 20:14:46.990958 | controller | - MariaDB: OK 2026-06-09 20:14:46.990967 | controller | - NodePoolBuilder: OK 2026-06-09 20:14:46.990999 | controller | - NodePoolLauncher: OK 2026-06-09 20:14:46.991009 | controller | - Zookeeper: OK 2026-06-09 20:14:46.991017 | controller | - Zuul: Waiting ... 2026-06-09 20:14:46.991025 | controller | - |_ Config Secrets: OK 2026-06-09 20:14:46.991034 | controller | - |_ Executor: OK 2026-06-09 20:14:46.991042 | controller | - |_ Merger: OK 2026-06-09 20:14:46.991051 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:14:46.991059 | controller | - |_ Web: OK 2026-06-09 20:14:46.991068 | controller | 2026-06-09 20:14:46.991082 | controller | 2026-06-09T20:14:46-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-09 20:14:51.998787 | controller | 2026-06-09T20:14:51-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:14:52.025272 | controller | 2026-06-09T20:14:52-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:14:52.038150 | controller | 2026-06-09T20:14:52-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:14:52.607363 | controller | 2026-06-09T20:14:52-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:14:52.616527 | controller | 2026-06-09T20:14:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:14:52.616603 | controller | 2026-06-09T20:14:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:14:52.625076 | controller | 2026-06-09T20:14:52-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:14:52.635385 | controller | 2026-06-09T20:14:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:14:52.635437 | controller | 2026-06-09T20:14:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:14:53.008513 | controller | 2026-06-09T20:14:53-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:14:53.011139 | controller | 2026-06-09T20:14:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:14:53.414072 | controller | 2026-06-09T20:14:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:14:53.414166 | controller | 2026-06-09T20:14:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:14:53.614265 | controller | 2026-06-09T20:14:53-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:14:54.422839 | controller | 2026-06-09T20:14:54-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:14:54.429588 | controller | 2026-06-09T20:14:54-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:14:54.429623 | controller | 2026-06-09T20:14:54-04:00 INFO deployment rollout not ok, name: zuul-weeder 2026-06-09 20:14:54.611770 | controller | 2026-06-09T20:14:54-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:14:55.404079 | controller | 2026-06-09T20:14:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:14:55.404154 | controller | 2026-06-09T20:14:55-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:14:56.210377 | controller | 2026-06-09T20:14:56-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:14:56.212909 | controller | 2026-06-09T20:14:56-04:00 INFO 2026-06-09 20:14:56.212937 | controller | - Config: Waiting ... 2026-06-09 20:14:56.212947 | controller | - Gateway: OK 2026-06-09 20:14:56.212955 | controller | - GitServer: OK 2026-06-09 20:14:56.212963 | controller | - HoundSearch: OK 2026-06-09 20:14:56.212970 | controller | - LogJuicer: OK 2026-06-09 20:14:56.212978 | controller | - Logserver: OK 2026-06-09 20:14:56.212986 | controller | - MariaDB: OK 2026-06-09 20:14:56.212993 | controller | - NodePoolBuilder: OK 2026-06-09 20:14:56.213001 | controller | - NodePoolLauncher: OK 2026-06-09 20:14:56.213008 | controller | - Zookeeper: OK 2026-06-09 20:14:56.213016 | controller | - Zuul: Waiting ... 2026-06-09 20:14:56.213023 | controller | - |_ Config Secrets: OK 2026-06-09 20:14:56.213031 | controller | - |_ Executor: OK 2026-06-09 20:14:56.213039 | controller | - |_ Merger: OK 2026-06-09 20:14:56.213046 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:14:56.213054 | controller | - |_ Web: OK 2026-06-09 20:14:56.213062 | controller | 2026-06-09 20:14:56.213070 | controller | 2026-06-09T20:14:56-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-09 20:15:01.223399 | controller | 2026-06-09T20:15:01-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:15:01.258216 | controller | 2026-06-09T20:15:01-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:15:01.276762 | controller | 2026-06-09T20:15:01-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:15:01.828928 | controller | 2026-06-09T20:15:01-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:15:01.838434 | controller | 2026-06-09T20:15:01-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:15:01.838484 | controller | 2026-06-09T20:15:01-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:15:01.847561 | controller | 2026-06-09T20:15:01-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:15:01.857054 | controller | 2026-06-09T20:15:01-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:15:01.857130 | controller | 2026-06-09T20:15:01-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:15:02.234260 | controller | 2026-06-09T20:15:02-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:15:02.236543 | controller | 2026-06-09T20:15:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:15:02.635261 | controller | 2026-06-09T20:15:02-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:15:02.635330 | controller | 2026-06-09T20:15:02-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:15:02.836097 | controller | 2026-06-09T20:15:02-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:15:03.641943 | controller | 2026-06-09T20:15:03-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:15:03.650565 | controller | 2026-06-09T20:15:03-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:15:03.650626 | controller | 2026-06-09T20:15:03-04:00 INFO deployment rollout not ok, name: zuul-weeder 2026-06-09 20:15:03.834657 | controller | 2026-06-09T20:15:03-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:15:04.627523 | controller | 2026-06-09T20:15:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:15:04.627593 | controller | 2026-06-09T20:15:04-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:15:05.430660 | controller | 2026-06-09T20:15:05-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:15:05.437695 | controller | 2026-06-09T20:15:05-04:00 INFO 2026-06-09 20:15:05.437733 | controller | - Config: Waiting ... 2026-06-09 20:15:05.437739 | controller | - Gateway: OK 2026-06-09 20:15:05.437751 | controller | - GitServer: OK 2026-06-09 20:15:05.437756 | controller | - HoundSearch: OK 2026-06-09 20:15:05.437761 | controller | - LogJuicer: OK 2026-06-09 20:15:05.437766 | controller | - Logserver: OK 2026-06-09 20:15:05.437771 | controller | - MariaDB: OK 2026-06-09 20:15:05.437776 | controller | - NodePoolBuilder: OK 2026-06-09 20:15:05.437781 | controller | - NodePoolLauncher: OK 2026-06-09 20:15:05.437786 | controller | - Zookeeper: OK 2026-06-09 20:15:05.437791 | controller | - Zuul: Waiting ... 2026-06-09 20:15:05.437796 | controller | - |_ Config Secrets: OK 2026-06-09 20:15:05.437801 | controller | - |_ Executor: OK 2026-06-09 20:15:05.437806 | controller | - |_ Merger: OK 2026-06-09 20:15:05.437811 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:15:05.437817 | controller | - |_ Web: OK 2026-06-09 20:15:05.437825 | controller | 2026-06-09 20:15:05.437832 | controller | 2026-06-09T20:15:05-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-09 20:15:10.442352 | controller | 2026-06-09T20:15:10-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:15:10.467107 | controller | 2026-06-09T20:15:10-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:15:10.480288 | controller | 2026-06-09T20:15:10-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:15:11.054291 | controller | 2026-06-09T20:15:11-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:15:11.066715 | controller | 2026-06-09T20:15:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:15:11.066758 | controller | 2026-06-09T20:15:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:15:11.079589 | controller | 2026-06-09T20:15:11-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:15:11.092339 | controller | 2026-06-09T20:15:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:15:11.092408 | controller | 2026-06-09T20:15:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:15:11.453514 | controller | 2026-06-09T20:15:11-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:15:11.455469 | controller | 2026-06-09T20:15:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:15:11.853332 | controller | 2026-06-09T20:15:11-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:15:11.853400 | controller | 2026-06-09T20:15:11-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:15:12.052996 | controller | 2026-06-09T20:15:12-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:15:12.869077 | controller | 2026-06-09T20:15:12-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:15:12.878869 | controller | 2026-06-09T20:15:12-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:15:12.878918 | controller | 2026-06-09T20:15:12-04:00 INFO deployment rollout not ok, name: zuul-weeder 2026-06-09 20:15:13.056137 | controller | 2026-06-09T20:15:13-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:15:13.847313 | controller | 2026-06-09T20:15:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:15:13.847427 | controller | 2026-06-09T20:15:13-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:15:14.656788 | controller | 2026-06-09T20:15:14-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:15:14.661412 | controller | 2026-06-09T20:15:14-04:00 INFO 2026-06-09 20:15:14.661463 | controller | - Config: Waiting ... 2026-06-09 20:15:14.661475 | controller | - Gateway: OK 2026-06-09 20:15:14.661483 | controller | - GitServer: OK 2026-06-09 20:15:14.661491 | controller | - HoundSearch: OK 2026-06-09 20:15:14.661500 | controller | - LogJuicer: OK 2026-06-09 20:15:14.661507 | controller | - Logserver: OK 2026-06-09 20:15:14.661515 | controller | - MariaDB: OK 2026-06-09 20:15:14.661523 | controller | - NodePoolBuilder: OK 2026-06-09 20:15:14.661531 | controller | - NodePoolLauncher: OK 2026-06-09 20:15:14.661539 | controller | - Zookeeper: OK 2026-06-09 20:15:14.661547 | controller | - Zuul: Waiting ... 2026-06-09 20:15:14.661554 | controller | - |_ Config Secrets: OK 2026-06-09 20:15:14.661562 | controller | - |_ Executor: OK 2026-06-09 20:15:14.661570 | controller | - |_ Merger: OK 2026-06-09 20:15:14.661578 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:15:14.661587 | controller | - |_ Web: OK 2026-06-09 20:15:14.661595 | controller | 2026-06-09 20:15:14.661604 | controller | 2026-06-09T20:15:14-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-09 20:15:19.672770 | controller | 2026-06-09T20:15:19-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:15:19.713264 | controller | 2026-06-09T20:15:19-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:15:19.737296 | controller | 2026-06-09T20:15:19-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:15:20.282948 | controller | 2026-06-09T20:15:20-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:15:20.298792 | controller | 2026-06-09T20:15:20-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:15:20.298895 | controller | 2026-06-09T20:15:20-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:15:20.313146 | controller | 2026-06-09T20:15:20-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:15:20.329824 | controller | 2026-06-09T20:15:20-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:15:20.329972 | controller | 2026-06-09T20:15:20-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:15:20.686502 | controller | 2026-06-09T20:15:20-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:15:20.689002 | controller | 2026-06-09T20:15:20-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:15:21.089804 | controller | 2026-06-09T20:15:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:15:21.089931 | controller | 2026-06-09T20:15:21-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:15:21.288130 | controller | 2026-06-09T20:15:21-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:15:22.098514 | controller | 2026-06-09T20:15:22-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:15:22.108299 | controller | 2026-06-09T20:15:22-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:15:22.108408 | controller | 2026-06-09T20:15:22-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:15:22.283923 | controller | 2026-06-09T20:15:22-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:15:23.078694 | controller | 2026-06-09T20:15:23-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:15:23.078844 | controller | 2026-06-09T20:15:23-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:15:23.886643 | controller | 2026-06-09T20:15:23-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:15:24.075155 | controller | 2026-06-09T20:15:24-04:00 INFO Creating config map name: pymod-sf-operator-config-map 2026-06-09 20:15:24.075355 | controller | 2026-06-09T20:15:24-04:00 INFO Creating object, name: pymod-sf-operator-config-map 2026-06-09 20:15:24.477705 | controller | 2026-06-09T20:15:24-04:00 INFO Creating object, name: config-updater 2026-06-09 20:15:24.485685 | controller | 2026-06-09T20:15:24-04:00 INFO Creating object, name: config-updater-role 2026-06-09 20:15:24.493098 | controller | 2026-06-09T20:15:24-04:00 INFO Creating object, name: config-updater 2026-06-09 20:15:24.500951 | controller | 2026-06-09T20:15:24-04:00 INFO Creating the config-update service account secret 2026-06-09 20:15:24.501033 | controller | 2026-06-09T20:15:24-04:00 INFO Creating object, name: config-update-secrets 2026-06-09 20:15:24.507918 | controller | 2026-06-09T20:15:24-04:00 INFO Creating base secret job 2026-06-09 20:15:24.508549 | controller | 2026-06-09T20:15:24-04:00 INFO Creating object, name: config-base-secret-1-945-ns 2026-06-09 20:15:24.514835 | controller | 2026-06-09T20:15:24-04:00 INFO 2026-06-09 20:15:24.514872 | controller | - Config: Waiting ... 2026-06-09 20:15:24.514882 | controller | - Gateway: OK 2026-06-09 20:15:24.514891 | controller | - GitServer: OK 2026-06-09 20:15:24.514899 | controller | - HoundSearch: OK 2026-06-09 20:15:24.514907 | controller | - LogJuicer: OK 2026-06-09 20:15:24.514914 | controller | - Logserver: OK 2026-06-09 20:15:24.514922 | controller | - MariaDB: OK 2026-06-09 20:15:24.514929 | controller | - NodePoolBuilder: OK 2026-06-09 20:15:24.514937 | controller | - NodePoolLauncher: OK 2026-06-09 20:15:24.514945 | controller | - Zookeeper: OK 2026-06-09 20:15:24.514953 | controller | - Zuul: OK 2026-06-09 20:15:24.514962 | controller | 2026-06-09 20:15:24.515003 | controller | 2026-06-09T20:15:24-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-09 20:15:29.524063 | controller | 2026-06-09T20:15:29-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:15:29.551691 | controller | 2026-06-09T20:15:29-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:15:29.569762 | controller | 2026-06-09T20:15:29-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:15:30.133049 | controller | 2026-06-09T20:15:30-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:15:30.146783 | controller | 2026-06-09T20:15:30-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:15:30.146850 | controller | 2026-06-09T20:15:30-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:15:30.156952 | controller | 2026-06-09T20:15:30-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:15:30.170194 | controller | 2026-06-09T20:15:30-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:15:30.170340 | controller | 2026-06-09T20:15:30-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:15:30.539115 | controller | 2026-06-09T20:15:30-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:15:30.542680 | controller | 2026-06-09T20:15:30-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:15:30.947742 | controller | 2026-06-09T20:15:30-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:15:30.947863 | controller | 2026-06-09T20:15:30-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:15:31.141491 | controller | 2026-06-09T20:15:31-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:15:31.944659 | controller | 2026-06-09T20:15:31-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:15:31.951522 | controller | 2026-06-09T20:15:31-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:15:31.951626 | controller | 2026-06-09T20:15:31-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:15:32.133083 | controller | 2026-06-09T20:15:32-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:15:32.928735 | controller | 2026-06-09T20:15:32-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:15:32.928846 | controller | 2026-06-09T20:15:32-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:15:33.732654 | controller | 2026-06-09T20:15:33-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:15:34.135985 | controller | 2026-06-09T20:15:34-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-09 20:15:34.325592 | controller | 2026-06-09T20:15:34-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-09 20:15:40.067258 | controller | 2026-06-09T20:15:40-04:00 INFO Deleting object, name: zuul-web 2026-06-09 20:15:40.070918 | controller | 2026-06-09T20:15:40-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-09 20:15:40.073381 | controller | 2026-06-09T20:15:40-04:00 INFO 2026-06-09 20:15:40.073415 | controller | - Config: Waiting ... 2026-06-09 20:15:40.073425 | controller | - Gateway: OK 2026-06-09 20:15:40.073433 | controller | - GitServer: OK 2026-06-09 20:15:40.073441 | controller | - HoundSearch: OK 2026-06-09 20:15:40.073449 | controller | - LogJuicer: OK 2026-06-09 20:15:40.073457 | controller | - Logserver: OK 2026-06-09 20:15:40.073465 | controller | - MariaDB: OK 2026-06-09 20:15:40.073473 | controller | - NodePoolBuilder: OK 2026-06-09 20:15:40.073480 | controller | - NodePoolLauncher: OK 2026-06-09 20:15:40.073488 | controller | - Zookeeper: OK 2026-06-09 20:15:40.073496 | controller | - Zuul: OK 2026-06-09 20:15:40.073505 | controller | 2026-06-09 20:15:40.073513 | controller | 2026-06-09T20:15:40-04:00 INFO [attempt #13] Waiting 5s for the next reconcile call ... 2026-06-09 20:15:45.084217 | controller | 2026-06-09T20:15:45-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:15:45.111569 | controller | 2026-06-09T20:15:45-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:15:45.133589 | controller | 2026-06-09T20:15:45-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:15:45.688260 | controller | 2026-06-09T20:15:45-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:15:45.696225 | controller | 2026-06-09T20:15:45-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:15:45.696305 | controller | 2026-06-09T20:15:45-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:15:45.702818 | controller | 2026-06-09T20:15:45-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:15:45.713935 | controller | 2026-06-09T20:15:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:15:45.713990 | controller | 2026-06-09T20:15:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:15:46.094033 | controller | 2026-06-09T20:15:46-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:15:46.096956 | controller | 2026-06-09T20:15:46-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:15:46.495965 | controller | 2026-06-09T20:15:46-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:15:46.496054 | controller | 2026-06-09T20:15:46-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:15:46.699862 | controller | 2026-06-09T20:15:46-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:15:47.500796 | controller | 2026-06-09T20:15:47-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:15:47.508685 | controller | 2026-06-09T20:15:47-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:15:47.508741 | controller | 2026-06-09T20:15:47-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:15:47.691628 | controller | 2026-06-09T20:15:47-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:15:48.486002 | controller | 2026-06-09T20:15:48-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:15:49.291760 | controller | 2026-06-09T20:15:49-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:15:49.294045 | controller | 2026-06-09T20:15:49-04:00 INFO 2026-06-09 20:15:49.294069 | controller | - Config: Waiting ... 2026-06-09 20:15:49.294076 | controller | - Gateway: OK 2026-06-09 20:15:49.294081 | controller | - GitServer: OK 2026-06-09 20:15:49.294086 | controller | - HoundSearch: OK 2026-06-09 20:15:49.294092 | controller | - LogJuicer: OK 2026-06-09 20:15:49.294096 | controller | - Logserver: OK 2026-06-09 20:15:49.294101 | controller | - MariaDB: OK 2026-06-09 20:15:49.294106 | controller | - NodePoolBuilder: OK 2026-06-09 20:15:49.294111 | controller | - NodePoolLauncher: OK 2026-06-09 20:15:49.294116 | controller | - Zookeeper: OK 2026-06-09 20:15:49.294121 | controller | - Zuul: Waiting ... 2026-06-09 20:15:49.294126 | controller | - |_ Config Secrets: OK 2026-06-09 20:15:49.294132 | controller | - |_ Executor: OK 2026-06-09 20:15:49.294138 | controller | - |_ Merger: OK 2026-06-09 20:15:49.294143 | controller | - |_ Scheduler: OK 2026-06-09 20:15:49.294148 | controller | - |_ Web: Waiting ... 2026-06-09 20:15:49.294155 | controller | 2026-06-09 20:15:49.294161 | controller | 2026-06-09T20:15:49-04:00 INFO [attempt #14] Waiting 5s for the next reconcile call ... 2026-06-09 20:15:54.304545 | controller | 2026-06-09T20:15:54-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:15:54.332617 | controller | 2026-06-09T20:15:54-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:15:54.354407 | controller | 2026-06-09T20:15:54-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:15:54.908443 | controller | 2026-06-09T20:15:54-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:15:54.917933 | controller | 2026-06-09T20:15:54-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:15:54.917967 | controller | 2026-06-09T20:15:54-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:15:54.924983 | controller | 2026-06-09T20:15:54-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:15:54.937834 | controller | 2026-06-09T20:15:54-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:15:54.937894 | controller | 2026-06-09T20:15:54-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:15:55.314445 | controller | 2026-06-09T20:15:55-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:15:55.317175 | controller | 2026-06-09T20:15:55-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:15:55.717887 | controller | 2026-06-09T20:15:55-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:15:55.717967 | controller | 2026-06-09T20:15:55-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:15:55.918990 | controller | 2026-06-09T20:15:55-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:15:56.718101 | controller | 2026-06-09T20:15:56-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:15:56.726149 | controller | 2026-06-09T20:15:56-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:15:56.726261 | controller | 2026-06-09T20:15:56-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:15:56.914829 | controller | 2026-06-09T20:15:56-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:15:57.709634 | controller | 2026-06-09T20:15:57-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:15:57.709698 | controller | 2026-06-09T20:15:57-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:15:58.513746 | controller | 2026-06-09T20:15:58-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:15:58.516832 | controller | 2026-06-09T20:15:58-04:00 INFO 2026-06-09 20:15:58.516874 | controller | - Config: Waiting ... 2026-06-09 20:15:58.516886 | controller | - Gateway: OK 2026-06-09 20:15:58.516895 | controller | - GitServer: OK 2026-06-09 20:15:58.516903 | controller | - HoundSearch: OK 2026-06-09 20:15:58.516911 | controller | - LogJuicer: OK 2026-06-09 20:15:58.516918 | controller | - Logserver: OK 2026-06-09 20:15:58.516926 | controller | - MariaDB: OK 2026-06-09 20:15:58.516934 | controller | - NodePoolBuilder: OK 2026-06-09 20:15:58.516941 | controller | - NodePoolLauncher: OK 2026-06-09 20:15:58.516949 | controller | - Zookeeper: OK 2026-06-09 20:15:58.516957 | controller | - Zuul: Waiting ... 2026-06-09 20:15:58.516964 | controller | - |_ Config Secrets: OK 2026-06-09 20:15:58.516972 | controller | - |_ Executor: OK 2026-06-09 20:15:58.516980 | controller | - |_ Merger: OK 2026-06-09 20:15:58.516987 | controller | - |_ Scheduler: OK 2026-06-09 20:15:58.516995 | controller | - |_ Web: Waiting ... 2026-06-09 20:15:58.517003 | controller | 2026-06-09 20:15:58.517011 | controller | 2026-06-09T20:15:58-04:00 INFO [attempt #15] Waiting 5s for the next reconcile call ... 2026-06-09 20:16:03.526587 | controller | 2026-06-09T20:16:03-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:16:03.549265 | controller | 2026-06-09T20:16:03-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:16:03.568097 | controller | 2026-06-09T20:16:03-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:16:04.135733 | controller | 2026-06-09T20:16:04-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:16:04.146516 | controller | 2026-06-09T20:16:04-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:16:04.146555 | controller | 2026-06-09T20:16:04-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:16:04.156666 | controller | 2026-06-09T20:16:04-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:16:04.169258 | controller | 2026-06-09T20:16:04-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:16:04.169297 | controller | 2026-06-09T20:16:04-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:16:04.541474 | controller | 2026-06-09T20:16:04-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:16:04.543992 | controller | 2026-06-09T20:16:04-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:16:04.942375 | controller | 2026-06-09T20:16:04-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:16:04.942459 | controller | 2026-06-09T20:16:04-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:16:05.143073 | controller | 2026-06-09T20:16:05-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:16:05.952293 | controller | 2026-06-09T20:16:05-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:16:05.962635 | controller | 2026-06-09T20:16:05-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:16:05.962693 | controller | 2026-06-09T20:16:05-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:16:06.137693 | controller | 2026-06-09T20:16:06-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:16:06.932703 | controller | 2026-06-09T20:16:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:16:06.932791 | controller | 2026-06-09T20:16:06-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:16:07.737452 | controller | 2026-06-09T20:16:07-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:16:07.740943 | controller | 2026-06-09T20:16:07-04:00 INFO 2026-06-09 20:16:07.740978 | controller | - Config: Waiting ... 2026-06-09 20:16:07.740989 | controller | - Gateway: OK 2026-06-09 20:16:07.740998 | controller | - GitServer: OK 2026-06-09 20:16:07.741007 | controller | - HoundSearch: OK 2026-06-09 20:16:07.741015 | controller | - LogJuicer: OK 2026-06-09 20:16:07.741023 | controller | - Logserver: OK 2026-06-09 20:16:07.741031 | controller | - MariaDB: OK 2026-06-09 20:16:07.741039 | controller | - NodePoolBuilder: OK 2026-06-09 20:16:07.741047 | controller | - NodePoolLauncher: OK 2026-06-09 20:16:07.741054 | controller | - Zookeeper: OK 2026-06-09 20:16:07.741062 | controller | - Zuul: Waiting ... 2026-06-09 20:16:07.741071 | controller | - |_ Config Secrets: OK 2026-06-09 20:16:07.741079 | controller | - |_ Executor: OK 2026-06-09 20:16:07.741087 | controller | - |_ Merger: OK 2026-06-09 20:16:07.741095 | controller | - |_ Scheduler: OK 2026-06-09 20:16:07.741128 | controller | - |_ Web: Waiting ... 2026-06-09 20:16:07.741137 | controller | 2026-06-09 20:16:07.741146 | controller | 2026-06-09T20:16:07-04:00 INFO [attempt #16] Waiting 5s for the next reconcile call ... 2026-06-09 20:16:12.746987 | controller | 2026-06-09T20:16:12-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:16:12.779435 | controller | 2026-06-09T20:16:12-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:16:12.795784 | controller | 2026-06-09T20:16:12-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:16:13.350564 | controller | 2026-06-09T20:16:13-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:16:13.360324 | controller | 2026-06-09T20:16:13-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:16:13.360375 | controller | 2026-06-09T20:16:13-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:16:13.367614 | controller | 2026-06-09T20:16:13-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:16:13.376227 | controller | 2026-06-09T20:16:13-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:16:13.376289 | controller | 2026-06-09T20:16:13-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:16:13.759928 | controller | 2026-06-09T20:16:13-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:16:13.763621 | controller | 2026-06-09T20:16:13-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:16:14.162116 | controller | 2026-06-09T20:16:14-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:16:14.162189 | controller | 2026-06-09T20:16:14-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:16:14.359783 | controller | 2026-06-09T20:16:14-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:16:15.172196 | controller | 2026-06-09T20:16:15-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:16:15.181132 | controller | 2026-06-09T20:16:15-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:16:15.181274 | controller | 2026-06-09T20:16:15-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:16:15.357219 | controller | 2026-06-09T20:16:15-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:16:16.150741 | controller | 2026-06-09T20:16:16-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:16:16.150845 | controller | 2026-06-09T20:16:16-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:16:16.957716 | controller | 2026-06-09T20:16:16-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:16:16.961021 | controller | 2026-06-09T20:16:16-04:00 INFO 2026-06-09 20:16:16.961053 | controller | - Config: Waiting ... 2026-06-09 20:16:16.961063 | controller | - Gateway: OK 2026-06-09 20:16:16.961072 | controller | - GitServer: OK 2026-06-09 20:16:16.961081 | controller | - HoundSearch: OK 2026-06-09 20:16:16.961089 | controller | - LogJuicer: OK 2026-06-09 20:16:16.961098 | controller | - Logserver: OK 2026-06-09 20:16:16.961106 | controller | - MariaDB: OK 2026-06-09 20:16:16.961114 | controller | - NodePoolBuilder: OK 2026-06-09 20:16:16.961123 | controller | - NodePoolLauncher: OK 2026-06-09 20:16:16.961131 | controller | - Zookeeper: OK 2026-06-09 20:16:16.961140 | controller | - Zuul: Waiting ... 2026-06-09 20:16:16.961148 | controller | - |_ Config Secrets: OK 2026-06-09 20:16:16.961157 | controller | - |_ Executor: OK 2026-06-09 20:16:16.961165 | controller | - |_ Merger: OK 2026-06-09 20:16:16.961173 | controller | - |_ Scheduler: OK 2026-06-09 20:16:16.961182 | controller | - |_ Web: Waiting ... 2026-06-09 20:16:16.961190 | controller | 2026-06-09 20:16:16.961199 | controller | 2026-06-09T20:16:16-04:00 INFO [attempt #17] Waiting 5s for the next reconcile call ... 2026-06-09 20:16:21.968995 | controller | 2026-06-09T20:16:21-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:16:21.990809 | controller | 2026-06-09T20:16:21-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:16:22.003675 | controller | 2026-06-09T20:16:22-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:16:22.581054 | controller | 2026-06-09T20:16:22-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:16:22.592576 | controller | 2026-06-09T20:16:22-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:16:22.592649 | controller | 2026-06-09T20:16:22-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:16:22.603295 | controller | 2026-06-09T20:16:22-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:16:22.616196 | controller | 2026-06-09T20:16:22-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:16:22.616327 | controller | 2026-06-09T20:16:22-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:16:22.978667 | controller | 2026-06-09T20:16:22-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:16:22.980520 | controller | 2026-06-09T20:16:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:16:23.385435 | controller | 2026-06-09T20:16:23-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:16:23.385512 | controller | 2026-06-09T20:16:23-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:16:23.584021 | controller | 2026-06-09T20:16:23-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:16:24.391322 | controller | 2026-06-09T20:16:24-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:16:24.399897 | controller | 2026-06-09T20:16:24-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:16:24.400003 | controller | 2026-06-09T20:16:24-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:16:24.580103 | controller | 2026-06-09T20:16:24-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:16:25.374215 | controller | 2026-06-09T20:16:25-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:16:25.374342 | controller | 2026-06-09T20:16:25-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:16:26.180039 | controller | 2026-06-09T20:16:26-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:16:26.183016 | controller | 2026-06-09T20:16:26-04:00 INFO 2026-06-09 20:16:26.183058 | controller | - Config: Waiting ... 2026-06-09 20:16:26.183070 | controller | - Gateway: OK 2026-06-09 20:16:26.183082 | controller | - GitServer: OK 2026-06-09 20:16:26.183094 | controller | - HoundSearch: OK 2026-06-09 20:16:26.183106 | controller | - LogJuicer: OK 2026-06-09 20:16:26.183117 | controller | - Logserver: OK 2026-06-09 20:16:26.183128 | controller | - MariaDB: OK 2026-06-09 20:16:26.183139 | controller | - NodePoolBuilder: OK 2026-06-09 20:16:26.183150 | controller | - NodePoolLauncher: OK 2026-06-09 20:16:26.183161 | controller | - Zookeeper: OK 2026-06-09 20:16:26.183170 | controller | - Zuul: Waiting ... 2026-06-09 20:16:26.183178 | controller | - |_ Config Secrets: OK 2026-06-09 20:16:26.183188 | controller | - |_ Executor: OK 2026-06-09 20:16:26.183200 | controller | - |_ Merger: OK 2026-06-09 20:16:26.183211 | controller | - |_ Scheduler: OK 2026-06-09 20:16:26.183222 | controller | - |_ Web: Waiting ... 2026-06-09 20:16:26.183291 | controller | 2026-06-09 20:16:26.183305 | controller | 2026-06-09T20:16:26-04:00 INFO [attempt #18] Waiting 5s for the next reconcile call ... 2026-06-09 20:16:31.193306 | controller | 2026-06-09T20:16:31-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:16:31.224756 | controller | 2026-06-09T20:16:31-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:16:31.243167 | controller | 2026-06-09T20:16:31-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:16:31.799969 | controller | 2026-06-09T20:16:31-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:16:31.809602 | controller | 2026-06-09T20:16:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:16:31.809634 | controller | 2026-06-09T20:16:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:16:31.817579 | controller | 2026-06-09T20:16:31-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:16:31.827050 | controller | 2026-06-09T20:16:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:16:31.827094 | controller | 2026-06-09T20:16:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:16:32.206680 | controller | 2026-06-09T20:16:32-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:16:32.209188 | controller | 2026-06-09T20:16:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:16:32.607636 | controller | 2026-06-09T20:16:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:16:32.607713 | controller | 2026-06-09T20:16:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:16:32.805473 | controller | 2026-06-09T20:16:32-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:16:33.618669 | controller | 2026-06-09T20:16:33-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:16:33.627805 | controller | 2026-06-09T20:16:33-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:16:33.627892 | controller | 2026-06-09T20:16:33-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:16:33.805278 | controller | 2026-06-09T20:16:33-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:16:34.597205 | controller | 2026-06-09T20:16:34-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:16:34.597299 | controller | 2026-06-09T20:16:34-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:16:35.406047 | controller | 2026-06-09T20:16:35-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:16:35.409663 | controller | 2026-06-09T20:16:35-04:00 INFO 2026-06-09 20:16:35.409750 | controller | - Config: Waiting ... 2026-06-09 20:16:35.409765 | controller | - Gateway: OK 2026-06-09 20:16:35.409777 | controller | - GitServer: OK 2026-06-09 20:16:35.409788 | controller | - HoundSearch: OK 2026-06-09 20:16:35.409799 | controller | - LogJuicer: OK 2026-06-09 20:16:35.409811 | controller | - Logserver: OK 2026-06-09 20:16:35.409822 | controller | - MariaDB: OK 2026-06-09 20:16:35.409834 | controller | - NodePoolBuilder: OK 2026-06-09 20:16:35.409846 | controller | - NodePoolLauncher: OK 2026-06-09 20:16:35.409857 | controller | - Zookeeper: OK 2026-06-09 20:16:35.409868 | controller | - Zuul: Waiting ... 2026-06-09 20:16:35.409879 | controller | - |_ Config Secrets: OK 2026-06-09 20:16:35.409890 | controller | - |_ Executor: OK 2026-06-09 20:16:35.409901 | controller | - |_ Merger: OK 2026-06-09 20:16:35.409912 | controller | - |_ Scheduler: OK 2026-06-09 20:16:35.409923 | controller | - |_ Web: Waiting ... 2026-06-09 20:16:35.409935 | controller | 2026-06-09 20:16:35.409997 | controller | 2026-06-09T20:16:35-04:00 INFO [attempt #19] Waiting 5s for the next reconcile call ... 2026-06-09 20:16:40.418046 | controller | 2026-06-09T20:16:40-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:16:40.439647 | controller | 2026-06-09T20:16:40-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:16:40.453330 | controller | 2026-06-09T20:16:40-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:16:41.028273 | controller | 2026-06-09T20:16:41-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:16:41.038632 | controller | 2026-06-09T20:16:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:16:41.038729 | controller | 2026-06-09T20:16:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:16:41.049596 | controller | 2026-06-09T20:16:41-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:16:41.061109 | controller | 2026-06-09T20:16:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:16:41.061184 | controller | 2026-06-09T20:16:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:16:41.430542 | controller | 2026-06-09T20:16:41-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:16:41.433895 | controller | 2026-06-09T20:16:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:16:41.829914 | controller | 2026-06-09T20:16:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:16:41.830028 | controller | 2026-06-09T20:16:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:16:42.032511 | controller | 2026-06-09T20:16:42-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:16:42.832869 | controller | 2026-06-09T20:16:42-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:16:42.840631 | controller | 2026-06-09T20:16:42-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:16:42.840671 | controller | 2026-06-09T20:16:42-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:16:43.027817 | controller | 2026-06-09T20:16:43-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:16:43.822901 | controller | 2026-06-09T20:16:43-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:16:43.822970 | controller | 2026-06-09T20:16:43-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:16:44.630219 | controller | 2026-06-09T20:16:44-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:16:44.633634 | controller | 2026-06-09T20:16:44-04:00 INFO 2026-06-09 20:16:44.633709 | controller | - Config: Waiting ... 2026-06-09 20:16:44.633719 | controller | - Gateway: OK 2026-06-09 20:16:44.633728 | controller | - GitServer: OK 2026-06-09 20:16:44.633736 | controller | - HoundSearch: OK 2026-06-09 20:16:44.633744 | controller | - LogJuicer: OK 2026-06-09 20:16:44.633751 | controller | - Logserver: OK 2026-06-09 20:16:44.633759 | controller | - MariaDB: OK 2026-06-09 20:16:44.633778 | controller | - NodePoolBuilder: OK 2026-06-09 20:16:44.633786 | controller | - NodePoolLauncher: OK 2026-06-09 20:16:44.633794 | controller | - Zookeeper: OK 2026-06-09 20:16:44.633802 | controller | - Zuul: Waiting ... 2026-06-09 20:16:44.633809 | controller | - |_ Config Secrets: OK 2026-06-09 20:16:44.633817 | controller | - |_ Executor: OK 2026-06-09 20:16:44.633825 | controller | - |_ Merger: OK 2026-06-09 20:16:44.633833 | controller | - |_ Scheduler: OK 2026-06-09 20:16:44.633841 | controller | - |_ Web: Waiting ... 2026-06-09 20:16:44.633849 | controller | 2026-06-09 20:16:44.633894 | controller | 2026-06-09T20:16:44-04:00 INFO [attempt #20] Waiting 5s for the next reconcile call ... 2026-06-09 20:16:49.641171 | controller | 2026-06-09T20:16:49-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:16:49.659490 | controller | 2026-06-09T20:16:49-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:16:49.672798 | controller | 2026-06-09T20:16:49-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:16:50.246652 | controller | 2026-06-09T20:16:50-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:16:50.255620 | controller | 2026-06-09T20:16:50-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:16:50.255666 | controller | 2026-06-09T20:16:50-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:16:50.264480 | controller | 2026-06-09T20:16:50-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:16:50.277061 | controller | 2026-06-09T20:16:50-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:16:50.277105 | controller | 2026-06-09T20:16:50-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:16:50.652251 | controller | 2026-06-09T20:16:50-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:16:50.654382 | controller | 2026-06-09T20:16:50-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:16:51.049648 | controller | 2026-06-09T20:16:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:16:51.049719 | controller | 2026-06-09T20:16:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:16:51.252794 | controller | 2026-06-09T20:16:51-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:16:52.059182 | controller | 2026-06-09T20:16:52-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:16:52.066435 | controller | 2026-06-09T20:16:52-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:16:52.066508 | controller | 2026-06-09T20:16:52-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:16:52.249921 | controller | 2026-06-09T20:16:52-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:16:53.047466 | controller | 2026-06-09T20:16:53-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:16:53.047566 | controller | 2026-06-09T20:16:53-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:16:53.850928 | controller | 2026-06-09T20:16:53-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:16:54.255739 | controller | 2026-06-09T20:16:54-04:00 INFO 2026-06-09 20:16:54.255819 | controller | - Config: OK 2026-06-09 20:16:54.255833 | controller | - Gateway: OK 2026-06-09 20:16:54.255844 | controller | - GitServer: OK 2026-06-09 20:16:54.255855 | controller | - HoundSearch: OK 2026-06-09 20:16:54.255866 | controller | - LogJuicer: OK 2026-06-09 20:16:54.255877 | controller | - Logserver: OK 2026-06-09 20:16:54.255887 | controller | - MariaDB: OK 2026-06-09 20:16:54.255898 | controller | - NodePoolBuilder: OK 2026-06-09 20:16:54.255909 | controller | - NodePoolLauncher: OK 2026-06-09 20:16:54.255918 | controller | - Zookeeper: OK 2026-06-09 20:16:54.255926 | controller | - Zuul: OK 2026-06-09 20:16:54.255934 | controller | 2026-06-09 20:16:54.255980 | controller | 2026-06-09T20:16:54-04:00 INFO Updating controller configmap ... 2026-06-09 20:16:54.643678 | controller | 2026-06-09T20:16:54-04:00 INFO Standalone reconcile done. 2026-06-10 00:16:54.746773 | controller | ok: Runtime: 0:09:53.532198 2026-06-10 00:16:54.765358 | 2026-06-10 00:16:54.765562 | PLAY RECAP 2026-06-10 00:16:54.765691 | controller | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 00:16:54.765737 | 2026-06-10 00:16:54.918429 | RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-deploy.yaml@master] 2026-06-10 00:16:54.921462 | RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-tests.yaml@master] 2026-06-10 00:16:55.723936 | 2026-06-10 00:16:55.724165 | PLAY [controller] 2026-06-10 00:16:55.762966 | 2026-06-10 00:16:55.763169 | TASK [Install test dependencies] 2026-06-10 00:16:56.839695 | controller | ok: Nothing to do 2026-06-10 00:16:56.855890 | 2026-06-10 00:16:56.856077 | TASK [setup-variables : Include global variables] 2026-06-10 00:16:56.893162 | controller | skipping: Conditional result was False 2026-06-10 00:16:56.903692 | 2026-06-10 00:16:56.903886 | TASK [setup-variables : Include zuul variables if zuul.executor is defined] 2026-06-10 00:16:56.963324 | controller | ok 2026-06-10 00:16:56.984492 | 2026-06-10 00:16:56.984706 | TASK [setup-nodepool-ns : Set up nodepool namespace] 2026-06-09 20:16:57.800885 | controller | 2026-06-09T20:16:57-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:16:57.800970 | controller | 2026-06-09 20:16:57.812819 | controller | 2026-06-09T20:16:57-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:16:57.812882 | controller | 2026-06-09 20:16:57.813845 | controller | 2026-06-09T20:16:57-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:16:57.813871 | controller | 2026-06-09 20:16:57.817402 | controller | 2026-06-09T20:16:57-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:16:57.817448 | controller | 2026-06-09 20:16:57.821741 | controller | 2026-06-09T20:16:57-04:00 INFO Creating "nodepool" in nodepool 2026-06-09 20:16:57.825265 | controller | 2026-06-09T20:16:57-04:00 INFO "nodepool" created 2026-06-09 20:16:57.827582 | controller | 2026-06-09T20:16:57-04:00 INFO Creating "nodepool-sa" in nodepool 2026-06-09 20:16:57.829984 | controller | 2026-06-09T20:16:57-04:00 INFO "nodepool-sa" created 2026-06-09 20:16:57.833401 | controller | 2026-06-09T20:16:57-04:00 INFO Creating "nodepool-role" in nodepool 2026-06-09 20:16:57.836530 | controller | 2026-06-09T20:16:57-04:00 INFO "nodepool-role" created 2026-06-09 20:16:57.837876 | controller | 2026-06-09T20:16:57-04:00 INFO Creating "nodepool-rb" in nodepool 2026-06-09 20:16:57.841760 | controller | 2026-06-09T20:16:57-04:00 INFO "nodepool-rb" created 2026-06-09 20:16:57.843483 | controller | 2026-06-09T20:16:57-04:00 INFO Creating "nodepool-token" in nodepool 2026-06-09 20:16:57.845880 | controller | 2026-06-09T20:16:57-04:00 INFO "nodepool-token" created 2026-06-09 20:16:58.851562 | controller | 2026-06-09T20:16:58-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:16:58.851641 | controller | 2026-06-09 20:16:58.856028 | controller | 2026-06-09T20:16:58-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:16:58.856064 | controller | 2026-06-09 20:16:58.861384 | controller | 2026-06-09T20:16:58-04:00 INFO Creating "nodepool-providers-secrets" in sf 2026-06-09 20:16:58.864424 | controller | 2026-06-09T20:16:58-04:00 INFO "nodepool-providers-secrets" created 2026-06-10 00:16:59.028651 | controller | ok: Runtime: 0:00:01.410068 2026-06-10 00:16:59.048225 | 2026-06-10 00:16:59.048483 | TASK [utils/get-gerrit-admin-user-api-key : Get Gerrit admin user API key] 2026-06-10 00:16:59.582025 | controller | ok: Runtime: 0:00:00.069349 2026-06-10 00:16:59.589166 | 2026-06-10 00:16:59.589326 | TASK [utils/get-gerrit-admin-user-api-key : Set fact for _gerrit_admin_api_key] 2026-06-10 00:16:59.623545 | controller | ok 2026-06-10 00:16:59.640988 | 2026-06-10 00:16:59.641170 | LOOP [utils/config-repo-reset : Config repo, reset] 2026-06-10 00:17:00.014816 | controller | Fetching origin 2026-06-10 00:17:00.015180 | controller | Fetching gerrit 2026-06-10 00:17:00.015242 | controller | From https://gerrit.sfop.me/a/config 2026-06-10 00:17:00.015291 | controller | * [new branch] master -> gerrit/master 2026-06-10 00:17:00.015392 | controller | ok: Item: git fetch --all Runtime: 0:00:00.117115 2026-06-10 00:17:00.271175 | controller | Already on 'master' 2026-06-10 00:17:00.271317 | controller | Your branch is up to date with 'origin/master'. 2026-06-10 00:17:00.271382 | controller | ok: Item: git checkout master Runtime: 0:00:00.005191 2026-06-10 00:17:00.394610 | controller | HEAD is now at 4f0de5c Automatic update 2026-06-10 00:17:00.394792 | controller | ok: Item: git reset --hard origin/master -- Runtime: 0:00:00.007967 2026-06-10 00:17:00.615927 | controller | ok: Item: git clean -f -d Runtime: 0:00:00.008581 2026-06-10 00:17:00.645108 | 2026-06-10 00:17:00.645332 | TASK [utils/setup-log-forwarding-stack : Deploy Loki via included role] 2026-06-10 00:17:00.671647 | controller | ok 2026-06-10 00:17:00.685173 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/deploy-loki.yaml 2026-06-10 00:17:00.694417 | 2026-06-10 00:17:00.694552 | TASK [utils/setup-log-forwarding-stack : Deploy Loki] 2026-06-09 20:17:01.209043 | controller | configmap/loki-config created 2026-06-09 20:17:01.216061 | controller | deployment.apps/test-loki created 2026-06-09 20:17:01.228049 | controller | service/loki-http created 2026-06-10 00:17:01.731614 | controller | ok: Runtime: 0:00:00.281934 2026-06-10 00:17:01.741918 | 2026-06-10 00:17:01.742145 | TASK [utils/setup-log-forwarding-stack : Wait for Loki deployment to be ready] 2026-06-10 00:17:12.467217 | controller | ok: Runtime: 0:00:00.066449 2026-06-10 00:17:12.476239 | 2026-06-10 00:17:12.476424 | TASK [utils/setup-log-forwarding-stack : Deploy Fluent Bit via included role] 2026-06-10 00:17:12.510252 | controller | ok 2026-06-10 00:17:12.521651 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/deploy-fluentbit.yaml 2026-06-10 00:17:12.530694 | 2026-06-10 00:17:12.530784 | TASK [utils/setup-log-forwarding-stack : Deploy Fluent Bit] 2026-06-09 20:17:12.993966 | controller | configmap/test-fluentbit-config created 2026-06-09 20:17:13.002109 | controller | pod/test-fluentbit created 2026-06-09 20:17:13.016055 | controller | service/fluent-bit-forward-input created 2026-06-10 00:17:13.069532 | controller | ok: Runtime: 0:00:00.266651 2026-06-10 00:17:13.079310 | 2026-06-10 00:17:13.079446 | TASK [utils/setup-log-forwarding-stack : Wait for Fluent Bit pod to be ready] 2026-06-10 00:17:23.803674 | controller | ok: Runtime: 0:00:00.086593 2026-06-10 00:17:23.813426 | 2026-06-10 00:17:23.813629 | TASK [utils/setup-log-forwarding-stack : Install LogCLI] 2026-06-10 00:17:23.840182 | controller | ok 2026-06-10 00:17:23.855512 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/install-logCLI.yaml 2026-06-10 00:17:23.867489 | 2026-06-10 00:17:23.867605 | TASK [utils/setup-log-forwarding-stack : check if logCLI is present] 2026-06-10 00:17:24.312236 | controller | ok 2026-06-10 00:17:24.318702 | 2026-06-10 00:17:24.318799 | TASK [utils/setup-log-forwarding-stack : download and unzip logCLI] 2026-06-10 00:17:26.416989 | controller | changed 2026-06-10 00:17:26.426091 | 2026-06-10 00:17:26.426217 | TASK [utils/setup-log-forwarding-stack : ensure ~/bin exists] 2026-06-10 00:17:26.882221 | controller | changed 2026-06-10 00:17:26.888211 | 2026-06-10 00:17:26.888291 | TASK [utils/setup-log-forwarding-stack : move logCLI to ~/bin/] 2026-06-10 00:17:27.425076 | controller | ok: Runtime: 0:00:00.005351 2026-06-10 00:17:27.442089 | 2026-06-10 00:17:27.442290 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:17:27.732049 | controller | ok 2026-06-10 00:17:27.742768 | 2026-06-10 00:17:27.742912 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:17:27.990931 | controller | ok 2026-06-10 00:17:27.996557 | 2026-06-10 00:17:27.996671 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:17:28.560951 | controller | 2026-06-09T20:17:28-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:17:28.561014 | controller | 2026-06-09 20:17:28.571481 | controller | 2026-06-09T20:17:28-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:17:28.571524 | controller | 2026-06-09 20:17:28.586352 | controller | 2026-06-09T20:17:28-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:17:28.607762 | controller | 2026-06-09T20:17:28-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:17:28.619663 | controller | 2026-06-09T20:17:28-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:17:29.194801 | controller | 2026-06-09T20:17:29-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:17:29.200834 | controller | 2026-06-09T20:17:29-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:17:29.321092 | controller | 2026-06-09T20:17:29-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:17:29.327020 | controller | 2026-06-09T20:17:29-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:17:29.327077 | controller | 2026-06-09T20:17:29-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:17:29.334528 | controller | 2026-06-09T20:17:29-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:17:29.344217 | controller | 2026-06-09T20:17:29-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:17:29.344319 | controller | 2026-06-09T20:17:29-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:17:29.596397 | controller | 2026-06-09T20:17:29-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:17:29.599675 | controller | 2026-06-09T20:17:29-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:17:30.003960 | controller | 2026-06-09T20:17:30-04:00 INFO Creating object, name: zuul-capacity 2026-06-09 20:17:30.015585 | controller | 2026-06-09T20:17:30-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-09 20:17:30.015671 | controller | 2026-06-09T20:17:30-04:00 INFO Updating object, name: nodepool-launcher 2026-06-09 20:17:30.201032 | controller | 2026-06-09T20:17:30-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-09 20:17:30.201088 | controller | 2026-06-09T20:17:30-04:00 INFO Updating object, name: nodepool-builder 2026-06-09 20:17:31.010032 | controller | 2026-06-09T20:17:31-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:17:31.019410 | controller | 2026-06-09T20:17:31-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:17:31.019460 | controller | 2026-06-09T20:17:31-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:17:31.390918 | controller | 2026-06-09T20:17:31-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:17:32.188069 | controller | 2026-06-09T20:17:32-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:17:32.188137 | controller | 2026-06-09T20:17:32-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:17:32.990623 | controller | 2026-06-09T20:17:32-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:17:33.399744 | controller | 2026-06-09T20:17:33-04:00 INFO 2026-06-09 20:17:33.399786 | controller | - Config: OK 2026-06-09 20:17:33.399793 | controller | - Gateway: OK 2026-06-09 20:17:33.399798 | controller | - GitServer: OK 2026-06-09 20:17:33.399803 | controller | - HoundSearch: OK 2026-06-09 20:17:33.399808 | controller | - LogJuicer: OK 2026-06-09 20:17:33.399813 | controller | - Logserver: OK 2026-06-09 20:17:33.399818 | controller | - MariaDB: OK 2026-06-09 20:17:33.399823 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:17:33.399828 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:17:33.399833 | controller | - Zookeeper: OK 2026-06-09 20:17:33.399838 | controller | - Zuul: OK 2026-06-09 20:17:33.399843 | controller | 2026-06-09 20:17:33.399850 | controller | 2026-06-09T20:17:33-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:17:38.404565 | controller | 2026-06-09T20:17:38-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:17:38.429652 | controller | 2026-06-09T20:17:38-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:17:38.447177 | controller | 2026-06-09T20:17:38-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:17:39.013143 | controller | 2026-06-09T20:17:39-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:17:39.023423 | controller | 2026-06-09T20:17:39-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:17:39.023492 | controller | 2026-06-09T20:17:39-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:17:39.033370 | controller | 2026-06-09T20:17:39-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:17:39.048478 | controller | 2026-06-09T20:17:39-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:17:39.048513 | controller | 2026-06-09T20:17:39-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:17:39.417048 | controller | 2026-06-09T20:17:39-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:17:39.419389 | controller | 2026-06-09T20:17:39-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:17:39.819875 | controller | 2026-06-09T20:17:39-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:17:39.819939 | controller | 2026-06-09T20:17:39-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:17:40.017541 | controller | 2026-06-09T20:17:40-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:17:40.017626 | controller | 2026-06-09T20:17:40-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:17:40.822849 | controller | 2026-06-09T20:17:40-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:17:40.830066 | controller | 2026-06-09T20:17:40-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:17:40.830096 | controller | 2026-06-09T20:17:40-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:17:41.016098 | controller | 2026-06-09T20:17:41-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:17:41.808850 | controller | 2026-06-09T20:17:41-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:17:41.808925 | controller | 2026-06-09T20:17:41-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:17:42.613540 | controller | 2026-06-09T20:17:42-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:17:43.019694 | controller | 2026-06-09T20:17:43-04:00 INFO 2026-06-09 20:17:43.019745 | controller | - Config: OK 2026-06-09 20:17:43.019752 | controller | - Gateway: OK 2026-06-09 20:17:43.019757 | controller | - GitServer: OK 2026-06-09 20:17:43.019762 | controller | - HoundSearch: OK 2026-06-09 20:17:43.019767 | controller | - LogJuicer: OK 2026-06-09 20:17:43.019772 | controller | - Logserver: OK 2026-06-09 20:17:43.019777 | controller | - MariaDB: OK 2026-06-09 20:17:43.019782 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:17:43.019787 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:17:43.019793 | controller | - Zookeeper: OK 2026-06-09 20:17:43.019798 | controller | - Zuul: OK 2026-06-09 20:17:43.019803 | controller | 2026-06-09 20:17:43.019808 | controller | 2026-06-09T20:17:43-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:17:48.029793 | controller | 2026-06-09T20:17:48-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:17:48.064983 | controller | 2026-06-09T20:17:48-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:17:48.084097 | controller | 2026-06-09T20:17:48-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:17:48.635634 | controller | 2026-06-09T20:17:48-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:17:48.647165 | controller | 2026-06-09T20:17:48-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:17:48.647210 | controller | 2026-06-09T20:17:48-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:17:48.654112 | controller | 2026-06-09T20:17:48-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:17:48.665983 | controller | 2026-06-09T20:17:48-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:17:48.666046 | controller | 2026-06-09T20:17:48-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:17:49.044001 | controller | 2026-06-09T20:17:49-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:17:49.048421 | controller | 2026-06-09T20:17:49-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:17:49.448396 | controller | 2026-06-09T20:17:49-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:17:49.448478 | controller | 2026-06-09T20:17:49-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:17:49.651805 | controller | 2026-06-09T20:17:49-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:17:49.651891 | controller | 2026-06-09T20:17:49-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:17:50.443127 | controller | 2026-06-09T20:17:50-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:17:50.448819 | controller | 2026-06-09T20:17:50-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:17:50.448854 | controller | 2026-06-09T20:17:50-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:17:50.638217 | controller | 2026-06-09T20:17:50-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:17:51.434313 | controller | 2026-06-09T20:17:51-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:17:51.434394 | controller | 2026-06-09T20:17:51-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:17:52.235542 | controller | 2026-06-09T20:17:52-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:17:52.638123 | controller | 2026-06-09T20:17:52-04:00 INFO 2026-06-09 20:17:52.638179 | controller | - Config: OK 2026-06-09 20:17:52.638188 | controller | - Gateway: OK 2026-06-09 20:17:52.638195 | controller | - GitServer: OK 2026-06-09 20:17:52.638202 | controller | - HoundSearch: OK 2026-06-09 20:17:52.638209 | controller | - LogJuicer: OK 2026-06-09 20:17:52.638215 | controller | - Logserver: OK 2026-06-09 20:17:52.638222 | controller | - MariaDB: OK 2026-06-09 20:17:52.638252 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:17:52.638260 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:17:52.638267 | controller | - Zookeeper: OK 2026-06-09 20:17:52.638274 | controller | - Zuul: OK 2026-06-09 20:17:52.638282 | controller | 2026-06-09 20:17:52.638289 | controller | 2026-06-09T20:17:52-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:17:57.645046 | controller | 2026-06-09T20:17:57-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:17:57.677954 | controller | 2026-06-09T20:17:57-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:17:57.701380 | controller | 2026-06-09T20:17:57-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:17:58.256141 | controller | 2026-06-09T20:17:58-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:17:58.269328 | controller | 2026-06-09T20:17:58-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:17:58.269395 | controller | 2026-06-09T20:17:58-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:17:58.279986 | controller | 2026-06-09T20:17:58-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:17:58.291283 | controller | 2026-06-09T20:17:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:17:58.291325 | controller | 2026-06-09T20:17:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:17:58.656322 | controller | 2026-06-09T20:17:58-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:17:58.658343 | controller | 2026-06-09T20:17:58-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:17:59.062021 | controller | 2026-06-09T20:17:59-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:17:59.062101 | controller | 2026-06-09T20:17:59-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:17:59.257825 | controller | 2026-06-09T20:17:59-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:17:59.257887 | controller | 2026-06-09T20:17:59-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:18:00.070830 | controller | 2026-06-09T20:18:00-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:18:00.081353 | controller | 2026-06-09T20:18:00-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:18:00.081431 | controller | 2026-06-09T20:18:00-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:18:00.255223 | controller | 2026-06-09T20:18:00-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:18:01.048664 | controller | 2026-06-09T20:18:01-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:18:01.048739 | controller | 2026-06-09T20:18:01-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:18:01.859398 | controller | 2026-06-09T20:18:01-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:18:02.259831 | controller | 2026-06-09T20:18:02-04:00 INFO 2026-06-09 20:18:02.259914 | controller | - Config: OK 2026-06-09 20:18:02.259925 | controller | - Gateway: OK 2026-06-09 20:18:02.259934 | controller | - GitServer: OK 2026-06-09 20:18:02.259943 | controller | - HoundSearch: OK 2026-06-09 20:18:02.259958 | controller | - LogJuicer: OK 2026-06-09 20:18:02.259966 | controller | - Logserver: OK 2026-06-09 20:18:02.259973 | controller | - MariaDB: OK 2026-06-09 20:18:02.259981 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:18:02.259990 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:18:02.259998 | controller | - Zookeeper: OK 2026-06-09 20:18:02.260006 | controller | - Zuul: OK 2026-06-09 20:18:02.260014 | controller | 2026-06-09 20:18:02.260024 | controller | 2026-06-09T20:18:02-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-09 20:18:07.270052 | controller | 2026-06-09T20:18:07-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:18:07.296614 | controller | 2026-06-09T20:18:07-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:18:07.311280 | controller | 2026-06-09T20:18:07-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:18:07.878362 | controller | 2026-06-09T20:18:07-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:18:07.895130 | controller | 2026-06-09T20:18:07-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:18:07.895227 | controller | 2026-06-09T20:18:07-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:18:07.906676 | controller | 2026-06-09T20:18:07-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:18:07.921506 | controller | 2026-06-09T20:18:07-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:18:07.921550 | controller | 2026-06-09T20:18:07-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:18:08.285475 | controller | 2026-06-09T20:18:08-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:18:08.288699 | controller | 2026-06-09T20:18:08-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:18:08.688531 | controller | 2026-06-09T20:18:08-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:18:08.688580 | controller | 2026-06-09T20:18:08-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:18:08.881097 | controller | 2026-06-09T20:18:08-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:18:08.881179 | controller | 2026-06-09T20:18:08-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:18:09.694494 | controller | 2026-06-09T20:18:09-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:18:09.703768 | controller | 2026-06-09T20:18:09-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:18:09.703781 | controller | 2026-06-09T20:18:09-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:18:09.883045 | controller | 2026-06-09T20:18:09-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:18:10.676320 | controller | 2026-06-09T20:18:10-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:18:10.676375 | controller | 2026-06-09T20:18:10-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:18:11.478144 | controller | 2026-06-09T20:18:11-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:18:11.877936 | controller | 2026-06-09T20:18:11-04:00 INFO 2026-06-09 20:18:11.877987 | controller | - Config: OK 2026-06-09 20:18:11.877992 | controller | - Gateway: OK 2026-06-09 20:18:11.877996 | controller | - GitServer: OK 2026-06-09 20:18:11.878000 | controller | - HoundSearch: OK 2026-06-09 20:18:11.878003 | controller | - LogJuicer: OK 2026-06-09 20:18:11.878007 | controller | - Logserver: OK 2026-06-09 20:18:11.878012 | controller | - MariaDB: OK 2026-06-09 20:18:11.878016 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:18:11.878021 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:18:11.878024 | controller | - Zookeeper: OK 2026-06-09 20:18:11.878028 | controller | - Zuul: OK 2026-06-09 20:18:11.878033 | controller | 2026-06-09 20:18:11.878038 | controller | 2026-06-09T20:18:11-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-09 20:18:16.886417 | controller | 2026-06-09T20:18:16-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:18:16.901924 | controller | 2026-06-09T20:18:16-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:18:16.911042 | controller | 2026-06-09T20:18:16-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:18:17.492779 | controller | 2026-06-09T20:18:17-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:18:17.500891 | controller | 2026-06-09T20:18:17-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:18:17.500934 | controller | 2026-06-09T20:18:17-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:18:17.510086 | controller | 2026-06-09T20:18:17-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:18:17.521017 | controller | 2026-06-09T20:18:17-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:18:17.521066 | controller | 2026-06-09T20:18:17-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:18:17.900353 | controller | 2026-06-09T20:18:17-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:18:17.903138 | controller | 2026-06-09T20:18:17-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:18:18.306170 | controller | 2026-06-09T20:18:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:18:18.306289 | controller | 2026-06-09T20:18:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:18:18.500849 | controller | 2026-06-09T20:18:18-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:18:19.311828 | controller | 2026-06-09T20:18:19-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:18:19.321954 | controller | 2026-06-09T20:18:19-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:18:19.322089 | controller | 2026-06-09T20:18:19-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:18:19.499002 | controller | 2026-06-09T20:18:19-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:18:20.291537 | controller | 2026-06-09T20:18:20-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:18:20.291600 | controller | 2026-06-09T20:18:20-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:18:21.098744 | controller | 2026-06-09T20:18:21-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:18:21.500314 | controller | 2026-06-09T20:18:21-04:00 INFO 2026-06-09 20:18:21.500378 | controller | - Config: OK 2026-06-09 20:18:21.500389 | controller | - Gateway: OK 2026-06-09 20:18:21.500399 | controller | - GitServer: OK 2026-06-09 20:18:21.500407 | controller | - HoundSearch: OK 2026-06-09 20:18:21.500416 | controller | - LogJuicer: OK 2026-06-09 20:18:21.500424 | controller | - Logserver: OK 2026-06-09 20:18:21.500433 | controller | - MariaDB: OK 2026-06-09 20:18:21.500441 | controller | - NodePoolBuilder: OK 2026-06-09 20:18:21.500450 | controller | - NodePoolLauncher: OK 2026-06-09 20:18:21.500458 | controller | - Zookeeper: OK 2026-06-09 20:18:21.500467 | controller | - Zuul: OK 2026-06-09 20:18:21.500476 | controller | 2026-06-09 20:18:21.500485 | controller | 2026-06-09T20:18:21-04:00 INFO Updating controller configmap ... 2026-06-09 20:18:21.889088 | controller | 2026-06-09T20:18:21-04:00 INFO Standalone reconcile done. 2026-06-10 00:18:22.098779 | controller | ok: Runtime: 0:00:53.643802 2026-06-10 00:18:22.111385 | 2026-06-10 00:18:22.111534 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:18:22.656907 | controller | ok: Runtime: 0:00:00.005635 2026-06-10 00:18:22.676426 | 2026-06-10 00:18:22.676777 | TASK [golang-integration-tests : Run integration tests] 2026-06-09 20:18:23.024173 | controller | go test -timeout 0 -v ./tests/... -args --ginkgo.v --ginkgo.no-color 2026-06-09 20:18:23.045522 | controller | go: downloading github.com/onsi/ginkgo/v2 v2.17.2 2026-06-09 20:18:23.045773 | controller | go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 2026-06-09 20:18:23.048314 | controller | go: downloading github.com/onsi/gomega v1.33.1 2026-06-09 20:18:32.244800 | controller | === RUN TestSFOP 2026-06-09 20:18:32.245191 | controller | Running Suite: Software Factory CI - /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests 2026-06-09 20:18:32.245222 | controller | ======================================================================================================================== 2026-06-09 20:18:32.245275 | controller | Random Seed: 1781050712 2026-06-09 20:18:32.245290 | controller | 2026-06-09 20:18:32.245300 | controller | Will run 16 of 16 specs 2026-06-09 20:18:32.245310 | controller | ------------------------------ 2026-06-09 20:18:32.245351 | controller | [BeforeSuite] 2026-06-09 20:18:32.245361 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/main_test.go:55 2026-06-09 20:18:32.246690 | controller | 2026-06-09T20:18:32-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:18:32.246718 | controller | 2026-06-09 20:18:32.258442 | controller | 2026-06-09T20:18:32-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:18:32.258497 | controller | 2026-06-09 20:18:32.263713 | controller | [BeforeSuite] PASSED [0.018 seconds] 2026-06-09 20:18:32.263763 | controller | ------------------------------ 2026-06-09 20:18:32.263819 | controller | Deploy Tests Reconcile default 2026-06-09 20:18:32.263840 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:11 2026-06-09 20:18:32.267968 | controller | 2026-06-09T20:18:32-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:18:32.289906 | controller | 2026-06-09T20:18:32-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:18:32.306568 | controller | 2026-06-09T20:18:32-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:18:32.874574 | controller | 2026-06-09T20:18:32-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:18:32.874662 | controller | 2026-06-09T20:18:32-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:18:32.878952 | controller | 2026-06-09T20:18:32-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:18:32.884792 | controller | 2026-06-09T20:18:32-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:18:33.063606 | controller | 2026-06-09T20:18:33-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:18:33.070777 | controller | 2026-06-09T20:18:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:18:33.070817 | controller | 2026-06-09T20:18:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:18:33.078435 | controller | 2026-06-09T20:18:33-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:18:33.086944 | controller | 2026-06-09T20:18:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:18:33.086974 | controller | 2026-06-09T20:18:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:18:33.282419 | controller | 2026-06-09T20:18:33-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:18:33.285093 | controller | 2026-06-09T20:18:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:18:33.685773 | controller | 2026-06-09T20:18:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:18:33.685847 | controller | 2026-06-09T20:18:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:18:33.885009 | controller | 2026-06-09T20:18:33-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:18:34.691178 | controller | 2026-06-09T20:18:34-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:18:34.698478 | controller | 2026-06-09T20:18:34-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:18:34.698533 | controller | 2026-06-09T20:18:34-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:18:35.075525 | controller | 2026-06-09T20:18:35-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:18:35.868301 | controller | 2026-06-09T20:18:35-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:18:35.868395 | controller | 2026-06-09T20:18:35-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:18:36.671418 | controller | 2026-06-09T20:18:36-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:18:37.077195 | controller | 2026-06-09T20:18:37-04:00 INFO 2026-06-09 20:18:37.077293 | controller | - Config: OK 2026-06-09 20:18:37.077305 | controller | - Gateway: OK 2026-06-09 20:18:37.077314 | controller | - GitServer: OK 2026-06-09 20:18:37.077322 | controller | - HoundSearch: OK 2026-06-09 20:18:37.077330 | controller | - LogJuicer: OK 2026-06-09 20:18:37.077338 | controller | - Logserver: OK 2026-06-09 20:18:37.077345 | controller | - MariaDB: OK 2026-06-09 20:18:37.077353 | controller | - NodePoolBuilder: OK 2026-06-09 20:18:37.077363 | controller | - NodePoolLauncher: OK 2026-06-09 20:18:37.077371 | controller | - Zookeeper: OK 2026-06-09 20:18:37.077379 | controller | - Zuul: OK 2026-06-09 20:18:37.077388 | controller | 2026-06-09 20:18:37.077396 | controller | • [4.813 seconds] 2026-06-09 20:18:37.077404 | controller | ------------------------------ 2026-06-09 20:18:37.077412 | controller | Deploy Tests Match zuul secrets 2026-06-09 20:18:37.077421 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:14 2026-06-09 20:18:37.079198 | controller | 2026-06-09T20:18:37-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /etc/zuul/zuul.conf] 2026-06-09 20:18:37.263459 | controller | • [0.186 seconds] 2026-06-09 20:18:37.263521 | controller | ------------------------------ 2026-06-09 20:18:37.263534 | controller | Deploy Tests Connect to services 2026-06-09 20:18:37.263544 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:17 2026-06-09 20:18:37.449723 | controller | • [0.186 seconds] 2026-06-09 20:18:37.449772 | controller | ------------------------------ 2026-06-09 20:18:37.449781 | controller | Project Private Keys Rotation ensures no config errors 2026-06-09 20:18:37.449786 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:33 2026-06-09 20:18:37.449915 | controller | STEP: Ensuring tenant have no config errors @ 06/09/26 20:18:37.449 2026-06-09 20:18:37.449959 | controller | 2026-06-09T20:18:37-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/internal/config-errors] 2026-06-09 20:18:37.585421 | controller | 2026-06-09T20:18:37-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/demo-tenant/config-errors] 2026-06-09 20:18:37.676036 | controller | • [0.226 seconds] 2026-06-09 20:18:37.676090 | controller | ------------------------------ 2026-06-09 20:18:37.676101 | controller | Project Private Keys Rotation inject new secret in demo-project 2026-06-09 20:18:37.676106 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:37 2026-06-09 20:18:37.676266 | controller | STEP: Ensure a test ssh key is available @ 06/09/26 20:18:37.676 2026-06-09 20:18:37.681932 | controller | STEP: Loading test secret from fixture (fake SSH key) @ 06/09/26 20:18:37.681 2026-06-09 20:18:37.682014 | controller | STEP: Writing test secret into zuul-scheduler pod and encrypting with zuul-client @ 06/09/26 20:18:37.681 2026-06-09 20:18:37.682074 | controller | 2026-06-09T20:18:37-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat > /tmp/secret-to-encrypt.txt] 2026-06-09 20:18:37.781462 | controller | 2026-06-09T20:18:37-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat /tmp/secret-to-encrypt.txt | zuul-client encrypt --tenant demo-tenant --project demo-project] 2026-06-09 20:18:38.464979 | controller | writing RSA key 2026-06-09 20:18:38.570401 | controller | STEP: Replacing placeholders in secrets.yaml @ 06/09/26 20:18:38.57 2026-06-09 20:18:38.670007 | controller | STEP: Writing .zuul.d/secrets.yaml in repo @ 06/09/26 20:18:38.669 2026-06-09 20:18:38.670086 | controller | STEP: Committing and pushing locally as admin (HTTPS) @ 06/09/26 20:18:38.669 2026-06-09 20:18:38.776333 | controller | • [1.100 seconds] 2026-06-09 20:18:38.776400 | controller | ------------------------------ 2026-06-09 20:18:38.776420 | controller | Project Private Keys Rotation runs rotate-projects-private-keys 2026-06-09 20:18:38.776441 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:84 2026-06-09 20:18:38.776452 | controller | STEP: Fetching project private keys state from ZooKeeper (to validate change after) @ 06/09/26 20:18:38.776 2026-06-09 20:18:38.776520 | controller | 2026-06-09T20:18:38-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-09 20:18:38.776530 | controller | import configparser 2026-06-09 20:18:38.776538 | controller | import hashlib 2026-06-09 20:18:38.776547 | controller | from zuul.zk import ZooKeeperClient 2026-06-09 20:18:38.776555 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-09 20:18:38.776564 | controller | from zuul.lib import encryption 2026-06-09 20:18:38.776572 | controller | 2026-06-09 20:18:38.776580 | controller | config = configparser.ConfigParser() 2026-06-09 20:18:38.776593 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-09 20:18:38.776601 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-09 20:18:38.776609 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-09 20:18:38.776617 | controller | zk_client.connect() 2026-06-09 20:18:38.776626 | controller | lines = [] 2026-06-09 20:18:38.776634 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-09 20:18:38.776642 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-09 20:18:38.776650 | controller | created = obj["keys"][0]["created"] 2026-06-09 20:18:38.776658 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-09 20:18:38.776666 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-09 20:18:38.776674 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-09 20:18:38.776682 | controller | if "demo-project" in path: 2026-06-09 20:18:38.776690 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-09 20:18:38.776698 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-09 20:18:38.776706 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-09 20:18:38.776714 | controller | f.write(pem) 2026-06-09 20:18:38.776722 | controller | print("\n".join(sorted(lines))) 2026-06-09 20:18:38.776731 | controller | PYEOF 2026-06-09 20:18:38.776739 | controller | ] 2026-06-09 20:18:39.907428 | controller | STEP: Project keys before rotation: /keystorage/gerrit/config/config/secrets 1781050431 ad153732ec063ce1 2026-06-09 20:18:39.907485 | controller | /keystorage/gerrit/demo-project/demo-project/secrets 1781050429 013ae1da781a6d97 2026-06-09 20:18:39.907492 | controller | /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 1781050428 34863ce5b14f83f7 2026-06-09 20:18:39.907497 | controller | /keystorage/git-server/system-config/system-config/secrets 1781050428 6d7ba49fef79e231 2026-06-09 20:18:39.907503 | controller | /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 1781050430 c124e4702a5df929 2026-06-09 20:18:39.907508 | controller | @ 06/09/26 20:18:39.907 2026-06-09 20:18:39.907518 | controller | STEP: Preparing SSH key for rotation from zuul-ssh-key secret @ 06/09/26 20:18:39.907 2026-06-09 20:18:39.910267 | controller | STEP: Running rotate-projects-private-keys @ 06/09/26 20:18:39.91 2026-06-09 20:18:39.913640 | controller | 2026-06-09T20:18:39-04:00 INFO Creating object, name: zuul-kazoo 2026-06-09 20:18:40.925506 | controller | 2026-06-09T20:18:40-04:00 INFO Deleting object, name: config-base-secret-1-945-ns 2026-06-09 20:18:40.928928 | controller | 2026-06-09T20:18:40-04:00 INFO KubeAPIWarningLogger child pods are preserved by default when jobs are deleted; set propagationPolicy=Background to remove them or set propagationPolicy=Orphan to suppress this warning 2026-06-09 20:18:40.929011 | controller | 2026-06-09T20:18:40-04:00 INFO Deleting object, name: zs-internal-tenant-reconfigure 2026-06-09 20:18:40.934077 | controller | 2026-06-09T20:18:40-04:00 INFO Creating config map name: zk-clients-need-refresh-config-map 2026-06-09 20:18:40.934115 | controller | 2026-06-09T20:18:40-04:00 INFO Creating object, name: zk-clients-need-refresh-config-map 2026-06-09 20:18:40.938379 | controller | 2026-06-09T20:18:40-04:00 INFO Creating config map name: zuul-needs-full-reconfigure-config-map 2026-06-09 20:18:40.938437 | controller | 2026-06-09T20:18:40-04:00 INFO Creating object, name: zuul-needs-full-reconfigure-config-map 2026-06-09 20:18:40.941154 | controller | 2026-06-09T20:18:40-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /tmp/rotate-projects-private-keys.py && chmod 755 /tmp/*.py] 2026-06-09 20:18:41.076537 | controller | 2026-06-09T20:18:41-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /var/lib/zuul/.ssh_push_key && chmod 0600 /var/lib/zuul/.ssh_push_key] 2026-06-09 20:18:41.173907 | controller | 2026-06-09T20:18:41-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /var/lib/zuul/main.yaml] 2026-06-09 20:18:41.267517 | controller | 2026-06-09T20:18:41-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /var/lib/zuul/main.yaml] 2026-06-09 20:18:41.358309 | controller | 2026-06-09T20:18:41-04:00 INFO Running pod execution pod: zuul-kazoo, command: [env PYTHONUNBUFFERED=1 /tmp/rotate-projects-private-keys.py --age 9223372036854775807 --author zuul --email zuul@sfop.me --logserver-key LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlKS1FJQkFBS0NBZ0VBeVNrTjhKQ3V6aWtZS3VndExZcktoVC9FSGg0bDFDMlZZVGw0OWs5VW5xc1hROEc0CmhialJqcW96UnpLVVUyU21odTdaYnVKaTVCeVRsR2JjejdVTm8zbkF1bm9NdlIrL09jT08rTjB1TE9EeVFTUGoKbFFVdlZRZzQ2d1QxOTlNSDlOUXR5OVBqL0hQMWJaTmszMTZoT2hoV2xzZnhRL3Q4czJmNDQrcVpxejlXRGRoeApzbDVOM1FxTmpFTGlGUmRySGVqSVdNSWFBdmdTQ3YyS2VSdHNkeVBCZHZRK2I2UmNyYlBYQWVwUUsrS2dqUFFmClQ0TnhDSDA1dm0xN3o5ZlNidStwZlpUU1Z0cVhzMVErYSs4VCtMM04ydUpseUpVOHdXZEZVWkN2Y0h1WUFjeW8KcEh3czRGNUQwaVVlcE9zRDF2ckZab1Y2WHF4YXdTUW9YbHNxbUtxL0R1WWM4TlR6ekRMTXBmcWZqSGs4ZkZIbApRVm12N2IwUnZMZVdVOUxRZ0VvMjRGem5QVTZjVTZRSnRoNkZiS295b2RvTFd5c3M2SUxsU1FrOGNMcEhTd3pmCkRrUkEyampWdUpCVlBNMmt5SXRqbmNyUUtVTll1NWFObUJuUFZyL0dPU3cwRmZtb1BXWnVtVTZpaDlpMXZTUGEKS0RiMWViclhodlBwc1hrRURkY280L1NjTHhxQ09wNkE1aXZ5OVdLTkkyazhkZ2RYQmtwUTh3NFd1T2hwaUJRbgppa3k0clYrM2w5MDY3cERhNlhTektTZUY3UGk1eFpaL3NsR25KYVg2Y0Y2RGgrZjZpNFpRYy8zbUc3TWpyRzFOCnR5Yng0czVGVEQ0RGlUUHNKWHczSksxcnh5bUlqdEVRUk5oaEZPMHZ4RTZ0S1lwSUM4akZwK0RuVERFQ0F3RUEKQVFLQ0FnQUV5WnNaRTF4eHNDVXcyanMvQnZ0WWJnTFZJNnNMVUhWcWJiTjlDempNbU81U0VNRjdQMEdQaDJuMgpneFBvSXpCYWJFTlZBZ0xRQW9VWnBsUVJobkhYWnUycEVvQU1uVDJRR3FFanQ2SGlkQTBYODBwMmtMdkdabzV5Ck8zaytKM0hjU0JONTh5Wk5LeFpYZ0FkSjBLYitLcmVlMlkzYXJCalU3VC9OYnRjdTdmQTJucVRnRmFIOVJZUEcKWjQ0WmFlZ0UrM1lQTkxscFFpMnR3TWlEcHQwdlBiVGNSNlpYd2wzcFRXc3IwcitMTjliUjIvcUZ5aTlOd0NzNApac01vV0c3YjhJekVYaDhlYlkvb052UlFYYzJJSVQ1TnZBOG5sYk92TjYyODZmaGU4VFlXTHVDQm42bnl3aE9UCnBBb0ZhMGw1VzVrSW5sc2g5ekNtd1oxTFY1aWRKRFJuV2diSVg0aWVtemR5Q2V3OFhmeWdBUC9QWk1QVHBFbFEKUG1mOEMweDZvRktEaDQ3cE83OXplVk9KT3NXaGJlTTRvcUszRktzeENmZFZZY3lhbE5YOGxkM3c3NXdVbHd3VApVbW1EWTI2QTRDZFVqYXZScWxxaG5qWmVVL3MzQ2o2RWxZRTBKckkvbXYwbFVYc1ZlK0swRGNEM3ZSUi9RdjB4CmRvK1Uzb2tOTTZXMzBvVVBGdnI4WGVlNnEyQUU1eU5pK2xVaFhwWEJuY3dJU3BJVHpON3g1em83UndqaVlQaEMKc2pjRVNjSU52alNFTGFWbC9Yc1g1SHdYWjNNb1g1ZEhFYVVKQ04vYUtsaFQ3amxvUDNoRU9iYTJZV09FNDBCVwpIQXl3Mkc3dnF1bXYzVjRDZE0xRGVXMmdUL2Z2UjBNZmVyZk84Sit2TC9FTnRqNGg4UUtDQVFFQStKcEcvcFltCkVMdVpnc1UwdXh4RGRmeE9FdWg1OW9tY3pSUXNjc0JMN2tGckVrek1ZUUErY1ArNTJYekYxSmtFVkhPMTNjSXAKc3RQdVg5YUtWc0IvcWg4WFl2VGtFTzk2RkhER3NPUUduOWxHTzFzZVpMRjFPTnNNS0JHWjgvOWViYjVac1BKUgozOW9ud1ptWlczdkFjUkJCREtabmtCVkpQQ0pldXJWQXpiMDB0MEtZOWZ4a2xNVUdkR2xIUW9aY0V2d29rTnp1ClJqU3o4QVdpaXNRc1hsZjZlOFo1czVuVk10bHVYT01CU1RrZ1praWJ1QS9Qb05BNDN4RHJLcTlVcFBLRmpiMU4KZXZiam5WQysrQytPc3NKTGhBdUc5cFZmTkdKekt6QlpqK0x5ZkRrRHV5S24vemZpQVFGczVDaXloK3dIYnZ6YworR24zRmoxdTcyZDVPUUtDQVFFQXp5VmpHWVhTNFVXQ3I3RTdtWDRSUzZJQXd3OUN6TVYrMVgyenhFa2laT2RsCmdwSHNsTXhSV3ZuVW9IMnFYRDh0YzJNcmh1QlNCeGhST1BMZU1Dc0xzL0pNWTU5RnVZZ1VhckpGWUJyMElFV24KRFQxdnJ1NlpiSGZkQU1RejdMQS9wQ0J3UFVyRWZJdUZsUFBPUFRDTTVkOHZuUlkxNmdHOUhWaDVJMmpQQmlLVAo2V0dFaGhkTXl4STA0VDZNVHA2blVXZ0N1bnZrTktqL1ZmWWQvRVFWOXZnRzhMTUw0SEx2QTF5RG5MUHJyT3pxClNueGhMUThFOHZXWHlFeXl5eUMycVZ4LzIzZDFQV1Q2VDc2Z05kN1pWM0xKQTEyQUQrRmZQa1B6OStaSWh1WGEKbWE2bEQ2OXhxOGJYOG5ldXMzak9NaWVYVUhyajZHSlJVb1NySU9CQ3VRS0NBUUVBaFVneFI4eTRHOEI0VE9QdApvV05GeDd1OUgrWURmV25KNXlrcHExdnRXWTg1R0ZhVVBHMGxEUEFOYzh5eFE0bkdDSUVBd0NPWXQ0ZG9CWG5BCk1VS0RMM0g3MTlIemRDRHdFb1ZrQVBJL212T2g4N0NtV3RYSlF2SXRVazRkZS9NdGJTamdiVDE5SjhhT0NmMlAKZXJ5ZUwvL0c0RUZXeEtpazJ0aVNjUWM3OXhHNWNZY0MrdDZXRGl5VGlWWG9Ib0pxV3NOVi9GUjJhcTFXeXhnNgpUS3RnSGpTTTFrOVNjRGZKVmJtOHNaQUM0SDhBSDlaTnBpRFQ4Q095V2VWNEx5cFE5eDRPL0M3Qm0ydENITlZ0CmpTSHlIMlNMbC8rV0hCMTVMVnlhNGt6SnQ1cTA1bVNxMjJsNUtCdU56M3cwSlI2c21MS3lTWTdINldibVVjWWQKODd4R0FRS0NBUUVBeHZ3b0xUMVRiTmprQ0RmZGc3NEppVGp2T0hGeGZ0bDgxRUp6TEd3QW9TTnpiTEVXclgxego3VXV1aUJrUFRKdlg0azd5WWhjY1ZQcDBFNXVoK20xdlNZWEpBYTNmUWJGSnc1REtlZDZLZ25IR1hqNlZDVnJrCmdNREh5Tk93K1gzNU5JV0FyREFsZldwZGdNK1BYMUpERnhLRlg0WVZkbHdET2F4aWViUlZvcFFXdUFyWVRpTVYKdnNFWGtRUVByZnBUeWl4NjZuN1dvZ3R4ZERiL2FQUlNrVEtESURjdWdvYUZ1Um5teCtKQjA1U21IN1ZKZUFwRQplOHVWNlhCY0ZDcXhyUkZSTlJhVXhjdzVjOTh2Y1hCSUlldiswcisvcEFHZ0IyVXBlQXA3d3pJTUFyTTBsbE8zCmd3WjVMTTd4UURBZEN5cmkvUWwvNHh5RytCaS9kczVXNlFLQ0FRQkNQS3JtWjM3bWlzcVlrbzVoZTl0YXdzUncKdGVhaHRSR3F1OGVaZjFvdDVFdTAvaU12eHJQSzJkVTFraHpNeUNaUXVlcVBoM2h2UzRCelNpYmpnWTBzRGcyZQo0NEwwV0tNamJjWGU5QVZTTGZvVGQ2bkQ1eVFpODhEeDM5ZHpuZDNObUlZS3MvU3o5ZXFyU1djMFV1TlhrZ0JjCi9hWUZlNkJUSGpQQmJoZG9ZeitXQ215ckVqemZ5eG1pdGg2SStROUh4R2NGMkl4ZkV5US91RDRjaGFPNUlOQTkKbGl2TkU0YStXKy9MQjErSFZSNTNQYUdURjVxYUdhbDM3M3Q4Vm1XMzk1NkM5Mjd6bDl5TjkyRmVibUFXRDcrNAp6d0M1RVEzUEJpTWd2Zm5qTzJRa2J2dDhsQnJIQUE4STVyb2d1YUVITnNFdkVJdGlLKzlYb2F1VDNhbEMKLS0tLS1FTkQgUlNBIFBSSVZBVEUgS0VZLS0tLS0K] 2026-06-09 20:18:41.853812 | controller | [+] Collecting keys from ZooKeeper 2026-06-09 20:18:41.871712 | controller | [+] Deleting /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh because Non secrets private key 2026-06-09 20:18:41.873932 | controller | [+] Deleting /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh because Non secrets private key 2026-06-09 20:18:41.875461 | controller | [+] Deleting /keystorage/gerrit/demo-project/demo-project/ssh because Non secrets private key 2026-06-09 20:18:41.876949 | controller | [+] Deleting /keystorage/gerrit/config/config/ssh because Non secrets private key 2026-06-09 20:18:41.878425 | controller | [+] Deleting /keystorage/git-server/system-config/system-config/ssh because Non secrets private key 2026-06-09 20:18:41.880026 | controller | [+] Deleting /keystorage2/tenant/internal/ssh because Non secrets private key 2026-06-09 20:18:41.881473 | controller | [+] Deleting /keystorage2/tenant/demo-tenant/ssh because Non secrets private key 2026-06-09 20:18:41.883056 | controller | [+] Deleting /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets because zuul-jobs do not have secrets 2026-06-09 20:18:41.889366 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/demo-tenant-config to /tmp/current-repo 2026-06-09 20:18:41.892897 | controller | Cloning into '/tmp/current-repo'... 2026-06-09 20:18:41.953678 | controller | Warning: Permanently added '[gerrit-sshd.sf]:29418' (ED25519) to the list of known hosts. 2026-06-09 20:18:42.114730 | controller | Total 9 (delta 0), reused 9 (delta 0) 2026-06-09 20:18:42.165793 | controller | [+] Deleting /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets because project had no secret 2026-06-09 20:18:42.174925 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/demo-project to /tmp/current-repo 2026-06-09 20:18:42.180685 | controller | Cloning into '/tmp/current-repo'... 2026-06-09 20:18:42.352103 | controller | Total 10 (delta 0), reused 10 (delta 0) 2026-06-09 20:18:43.655354 | controller | [+] Re-Encrypting secret(s) in /tmp/current-repo/.zuul.d/secrets.yaml 2026-06-09 20:18:43.668707 | controller | [master 4ab7d04] Automatic secret re-encryption 2026-06-09 20:18:43.668760 | controller | 1 file changed, 10 insertions(+), 10 deletions(-) 2026-06-09 20:18:43.841949 | controller | remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-09 20:18:43.842026 | controller | To ssh://gerrit-sshd.sf:29418/demo-project 2026-06-09 20:18:43.842034 | controller | 24b588e..4ab7d04 master -> master 2026-06-09 20:18:43.843108 | controller | [+] Updating key for /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-09 20:18:43.850801 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/config to /tmp/current-repo 2026-06-09 20:18:43.853343 | controller | Cloning into '/tmp/current-repo'... 2026-06-09 20:18:43.971767 | controller | Total 5 (delta 0), reused 5 (delta 0) 2026-06-09 20:18:44.021800 | controller | [+] Deleting /keystorage/gerrit/config/config/secrets because project had no secret 2026-06-09 20:18:44.024671 | controller | [+] Deleting /keystorage/git-server/system-config/system-config/secrets because internal repo will be handled in reconcile 2026-06-09 20:18:44.129408 | controller | STEP: Reconciling @ 06/09/26 20:18:44.129 2026-06-09 20:18:44.147133 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-09 20:18:44.147190 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-09 20:18:44.235632 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: nodepool-builder 2026-06-09 20:18:44.242164 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-76d8b68c95-9bccz launcher 2026-06-09 20:18:44.242200 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: nodepool-launcher-76d8b68c95-9bccz, command: [kill -9 1] 2026-06-09 20:18:44.337155 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: launcher 2026-06-09 20:18:44.339481 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-52vwg mariadb-client 2026-06-09 20:18:44.339509 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: zuul-db-provision-52vwg, command: [kill -9 1] 2026-06-09 20:18:44.357592 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: mariadb-client 2026-06-09 20:18:44.359683 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-09 20:18:44.359705 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-09 20:18:44.417548 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: zuul-executor 2026-06-09 20:18:44.421150 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: zuul-kazoo zuul-kazoo 2026-06-09 20:18:44.421180 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: zuul-kazoo, command: [kill -9 1] 2026-06-09 20:18:44.502086 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-09 20:18:44.506484 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-09 20:18:44.506545 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-09 20:18:44.585263 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: zuul-merger 2026-06-09 20:18:44.588821 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-09 20:18:44.588839 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-09 20:18:44.689075 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-09 20:18:44.692765 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-785b7c7c88-c9fgw zuul-web 2026-06-09 20:18:44.692846 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: zuul-web-785b7c7c88-c9fgw, command: [kill -9 1] 2026-06-09 20:18:44.775205 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: zuul-web 2026-06-09 20:18:44.779029 | controller | 2026-06-09T20:18:44-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-f4z5l zuul-weeder 2026-06-09 20:18:44.779094 | controller | 2026-06-09T20:18:44-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-f4z5l, command: [kill -9 1] 2026-06-09 20:18:44.859327 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: zuul-weeder 2026-06-09 20:18:44.862343 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: zk-clients-need-refresh-config-map 2026-06-09 20:18:44.864952 | controller | 2026-06-09T20:18:44-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:18:44.883450 | controller | 2026-06-09T20:18:44-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:18:44.896488 | controller | 2026-06-09T20:18:44-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:18:45.271700 | controller | 2026-06-09T20:18:45-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:18:45.271766 | controller | 2026-06-09T20:18:45-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:18:45.273476 | controller | 2026-06-09T20:18:45-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:18:45.277262 | controller | 2026-06-09T20:18:45-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:18:45.358989 | controller | 2026-06-09T20:18:45-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:18:45.366703 | controller | 2026-06-09T20:18:45-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:18:45.366733 | controller | 2026-06-09T20:18:45-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:18:45.373852 | controller | 2026-06-09T20:18:45-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:18:45.384840 | controller | 2026-06-09T20:18:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:18:45.384870 | controller | 2026-06-09T20:18:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:18:45.679910 | controller | 2026-06-09T20:18:45-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:18:45.682768 | controller | 2026-06-09T20:18:45-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:18:46.089633 | controller | 2026-06-09T20:18:46-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:18:46.089696 | controller | 2026-06-09T20:18:46-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:18:46.285166 | controller | 2026-06-09T20:18:46-04:00 INFO Creating object, name: nodepool-builder 2026-06-09 20:18:47.091111 | controller | 2026-06-09T20:18:47-04:00 INFO Creating object, name: zuul-scheduler 2026-06-09 20:18:47.471388 | controller | 2026-06-09T20:18:47-04:00 INFO Creating object, name: zuul-executor 2026-06-09 20:18:48.272996 | controller | 2026-06-09T20:18:48-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:18:49.068346 | controller | 2026-06-09T20:18:49-04:00 INFO Creating object, name: zuul-merger 2026-06-09 20:18:49.073555 | controller | 2026-06-09T20:18:49-04:00 INFO 2026-06-09 20:18:49.073601 | controller | - Config: Waiting ... 2026-06-09 20:18:49.073613 | controller | - Gateway: OK 2026-06-09 20:18:49.073625 | controller | - GitServer: OK 2026-06-09 20:18:49.073669 | controller | - HoundSearch: OK 2026-06-09 20:18:49.073710 | controller | - LogJuicer: OK 2026-06-09 20:18:49.073725 | controller | - Logserver: OK 2026-06-09 20:18:49.073759 | controller | - MariaDB: OK 2026-06-09 20:18:49.073794 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:18:49.073830 | controller | - NodePoolLauncher: OK 2026-06-09 20:18:49.073878 | controller | - Zookeeper: OK 2026-06-09 20:18:49.073891 | controller | - Zuul: Waiting ... 2026-06-09 20:18:49.073928 | controller | - |_ Config Secrets: OK 2026-06-09 20:18:49.073961 | controller | - |_ Executor: Waiting ... 2026-06-09 20:18:49.073997 | controller | - |_ Merger: Waiting ... 2026-06-09 20:18:49.074030 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:18:49.074067 | controller | - |_ Web: Waiting ... 2026-06-09 20:18:49.074081 | controller | 2026-06-09 20:18:50.080152 | controller | 2026-06-09T20:18:50-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:18:50.107938 | controller | 2026-06-09T20:18:50-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:18:50.131166 | controller | 2026-06-09T20:18:50-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:18:50.687131 | controller | 2026-06-09T20:18:50-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:18:50.687194 | controller | 2026-06-09T20:18:50-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:18:50.690124 | controller | 2026-06-09T20:18:50-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:18:50.700388 | controller | 2026-06-09T20:18:50-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:18:50.700404 | controller | 2026-06-09T20:18:50-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:18:50.710815 | controller | 2026-06-09T20:18:50-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:18:50.722994 | controller | 2026-06-09T20:18:50-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:18:50.723015 | controller | 2026-06-09T20:18:50-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:18:51.087118 | controller | 2026-06-09T20:18:51-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:18:51.090119 | controller | 2026-06-09T20:18:51-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:18:51.504172 | controller | 2026-06-09T20:18:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:18:51.504251 | controller | 2026-06-09T20:18:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:18:52.083984 | controller | 2026-06-09T20:18:52-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:18:52.084033 | controller | 2026-06-09T20:18:52-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:18:52.881396 | controller | 2026-06-09T20:18:52-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:18:52.881480 | controller | 2026-06-09T20:18:52-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:18:57.893815 | controller | 2026-06-09T20:18:57-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:18:57.893909 | controller | 2026-06-09T20:18:57-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:02.905561 | controller | 2026-06-09T20:19:02-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:02.905645 | controller | 2026-06-09T20:19:02-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:07.920138 | controller | 2026-06-09T20:19:07-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:07.920217 | controller | 2026-06-09T20:19:07-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:12.933373 | controller | 2026-06-09T20:19:12-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:12.933459 | controller | 2026-06-09T20:19:12-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:17.946287 | controller | 2026-06-09T20:19:17-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:17.946366 | controller | 2026-06-09T20:19:17-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:22.951261 | controller | 2026-06-09T20:19:22-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:19:22.979841 | controller | 2026-06-09T20:19:22-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:19:23.003563 | controller | 2026-06-09T20:19:23-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:19:23.003643 | controller | 2026-06-09T20:19:23-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:19:23.364046 | controller | 2026-06-09T20:19:23-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:19:23.365910 | controller | 2026-06-09T20:19:23-04:00 INFO 2026-06-09 20:19:23.365940 | controller | - Config: Waiting ... 2026-06-09 20:19:23.365949 | controller | - Gateway: OK 2026-06-09 20:19:23.365956 | controller | - GitServer: OK 2026-06-09 20:19:23.365963 | controller | - HoundSearch: OK 2026-06-09 20:19:23.365970 | controller | - LogJuicer: OK 2026-06-09 20:19:23.365977 | controller | - Logserver: OK 2026-06-09 20:19:23.365983 | controller | - MariaDB: OK 2026-06-09 20:19:23.365990 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:19:23.365998 | controller | - NodePoolLauncher: OK 2026-06-09 20:19:23.366005 | controller | - Zookeeper: OK 2026-06-09 20:19:23.366012 | controller | - Zuul: Waiting ... 2026-06-09 20:19:23.366018 | controller | - |_ Config Secrets: OK 2026-06-09 20:19:23.366025 | controller | - |_ Executor: OK 2026-06-09 20:19:23.366032 | controller | - |_ Merger: OK 2026-06-09 20:19:23.366039 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:19:23.366046 | controller | - |_ Web: Waiting ... 2026-06-09 20:19:23.366053 | controller | 2026-06-09 20:19:24.370682 | controller | 2026-06-09T20:19:24-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:19:24.395351 | controller | 2026-06-09T20:19:24-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:19:24.410185 | controller | 2026-06-09T20:19:24-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:19:24.981362 | controller | 2026-06-09T20:19:24-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:19:24.981447 | controller | 2026-06-09T20:19:24-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:19:24.984555 | controller | 2026-06-09T20:19:24-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:19:24.994368 | controller | 2026-06-09T20:19:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:19:24.994428 | controller | 2026-06-09T20:19:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:19:25.005547 | controller | 2026-06-09T20:19:25-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:19:25.017401 | controller | 2026-06-09T20:19:25-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:19:25.017481 | controller | 2026-06-09T20:19:25-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:19:25.385801 | controller | 2026-06-09T20:19:25-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:19:25.388637 | controller | 2026-06-09T20:19:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:19:25.786763 | controller | 2026-06-09T20:19:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:19:25.786856 | controller | 2026-06-09T20:19:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:19:26.358830 | controller | 2026-06-09T20:19:26-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:19:26.358885 | controller | 2026-06-09T20:19:26-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:19:27.161222 | controller | 2026-06-09T20:19:27-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:27.161307 | controller | 2026-06-09T20:19:27-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:32.171149 | controller | 2026-06-09T20:19:32-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:32.171204 | controller | 2026-06-09T20:19:32-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:37.182254 | controller | 2026-06-09T20:19:37-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:37.182335 | controller | 2026-06-09T20:19:37-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:42.190853 | controller | 2026-06-09T20:19:42-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:42.190941 | controller | 2026-06-09T20:19:42-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:47.198709 | controller | 2026-06-09T20:19:47-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:47.198772 | controller | 2026-06-09T20:19:47-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:19:52.210719 | controller | 2026-06-09T20:19:52-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:52.219314 | controller | 2026-06-09T20:19:52-04:00 INFO Creating object, name: zuul-weeder 2026-06-09 20:19:52.278482 | controller | 2026-06-09T20:19:52-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:19:52.301876 | controller | 2026-06-09T20:19:52-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:19:52.301974 | controller | 2026-06-09T20:19:52-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:19:52.639519 | controller | 2026-06-09T20:19:52-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:19:52.642224 | controller | 2026-06-09T20:19:52-04:00 INFO 2026-06-09 20:19:52.642302 | controller | - Config: Waiting ... 2026-06-09 20:19:52.642316 | controller | - Gateway: OK 2026-06-09 20:19:52.642329 | controller | - GitServer: OK 2026-06-09 20:19:52.642337 | controller | - HoundSearch: OK 2026-06-09 20:19:52.642345 | controller | - LogJuicer: OK 2026-06-09 20:19:52.642353 | controller | - Logserver: OK 2026-06-09 20:19:52.642361 | controller | - MariaDB: OK 2026-06-09 20:19:52.642370 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:19:52.642378 | controller | - NodePoolLauncher: OK 2026-06-09 20:19:52.642386 | controller | - Zookeeper: OK 2026-06-09 20:19:52.642394 | controller | - Zuul: Waiting ... 2026-06-09 20:19:52.642402 | controller | - |_ Config Secrets: OK 2026-06-09 20:19:52.642410 | controller | - |_ Executor: OK 2026-06-09 20:19:52.642418 | controller | - |_ Merger: OK 2026-06-09 20:19:52.642427 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:19:52.642435 | controller | - |_ Web: OK 2026-06-09 20:19:52.642443 | controller | 2026-06-09 20:19:53.648574 | controller | 2026-06-09T20:19:53-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:19:53.677851 | controller | 2026-06-09T20:19:53-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:19:53.699978 | controller | 2026-06-09T20:19:53-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:19:54.251678 | controller | 2026-06-09T20:19:54-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:19:54.251764 | controller | 2026-06-09T20:19:54-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:19:54.254398 | controller | 2026-06-09T20:19:54-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:19:54.263449 | controller | 2026-06-09T20:19:54-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:19:54.263479 | controller | 2026-06-09T20:19:54-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:19:54.270712 | controller | 2026-06-09T20:19:54-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:19:54.281909 | controller | 2026-06-09T20:19:54-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:19:54.281973 | controller | 2026-06-09T20:19:54-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:19:54.663904 | controller | 2026-06-09T20:19:54-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:19:54.667191 | controller | 2026-06-09T20:19:54-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:19:55.065940 | controller | 2026-06-09T20:19:55-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:19:55.066000 | controller | 2026-06-09T20:19:55-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:19:55.639004 | controller | 2026-06-09T20:19:55-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:19:56.436626 | controller | 2026-06-09T20:19:56-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:19:56.446488 | controller | 2026-06-09T20:19:56-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:19:56.446561 | controller | 2026-06-09T20:19:56-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:19:57.240744 | controller | 2026-06-09T20:19:57-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:19:58.034501 | controller | 2026-06-09T20:19:58-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:19:58.034587 | controller | 2026-06-09T20:19:58-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:19:58.840004 | controller | 2026-06-09T20:19:58-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:19:59.242776 | controller | 2026-06-09T20:19:59-04:00 INFO Creating base secret job 2026-06-09 20:19:59.243697 | controller | 2026-06-09T20:19:59-04:00 INFO Creating object, name: config-base-secret-1-945-ns 2026-06-09 20:19:59.249054 | controller | 2026-06-09T20:19:59-04:00 INFO 2026-06-09 20:19:59.249103 | controller | - Config: Waiting ... 2026-06-09 20:19:59.249165 | controller | - Gateway: OK 2026-06-09 20:19:59.249269 | controller | - GitServer: OK 2026-06-09 20:19:59.249352 | controller | - HoundSearch: OK 2026-06-09 20:19:59.249410 | controller | - LogJuicer: OK 2026-06-09 20:19:59.249488 | controller | - Logserver: OK 2026-06-09 20:19:59.249549 | controller | - MariaDB: OK 2026-06-09 20:19:59.249624 | controller | - NodePoolBuilder: OK 2026-06-09 20:19:59.249687 | controller | - NodePoolLauncher: OK 2026-06-09 20:19:59.249743 | controller | - Zookeeper: OK 2026-06-09 20:19:59.249835 | controller | - Zuul: OK 2026-06-09 20:19:59.249843 | controller | 2026-06-09 20:20:00.257082 | controller | 2026-06-09T20:20:00-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:00.289285 | controller | 2026-06-09T20:20:00-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:00.305192 | controller | 2026-06-09T20:20:00-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:00.861312 | controller | 2026-06-09T20:20:00-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:00.861450 | controller | 2026-06-09T20:20:00-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:00.865711 | controller | 2026-06-09T20:20:00-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:00.878907 | controller | 2026-06-09T20:20:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:00.879013 | controller | 2026-06-09T20:20:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:00.888664 | controller | 2026-06-09T20:20:00-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:00.904162 | controller | 2026-06-09T20:20:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:00.904257 | controller | 2026-06-09T20:20:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:01.268543 | controller | 2026-06-09T20:20:01-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:01.270881 | controller | 2026-06-09T20:20:01-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:01.674860 | controller | 2026-06-09T20:20:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:01.674972 | controller | 2026-06-09T20:20:01-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:02.241464 | controller | 2026-06-09T20:20:02-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:03.033170 | controller | 2026-06-09T20:20:03-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:03.039806 | controller | 2026-06-09T20:20:03-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:03.039854 | controller | 2026-06-09T20:20:03-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:03.840178 | controller | 2026-06-09T20:20:03-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:04.634291 | controller | 2026-06-09T20:20:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:20:04.634371 | controller | 2026-06-09T20:20:04-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:20:05.445371 | controller | 2026-06-09T20:20:05-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:20:05.844095 | controller | 2026-06-09T20:20:05-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-09 20:20:06.030872 | controller | 2026-06-09T20:20:06-04:00 INFO Deleting object, name: zuul-needs-full-reconfigure-config-map 2026-06-09 20:20:06.231599 | controller | 2026-06-09T20:20:06-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler full-reconfigure] 2026-06-09 20:20:12.082560 | controller | 2026-06-09T20:20:12-04:00 INFO Deleting object, name: zuul-web 2026-06-09 20:20:12.087736 | controller | 2026-06-09T20:20:12-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-09 20:20:12.091504 | controller | 2026-06-09T20:20:12-04:00 INFO 2026-06-09 20:20:12.091560 | controller | - Config: Waiting ... 2026-06-09 20:20:12.091568 | controller | - Gateway: OK 2026-06-09 20:20:12.091573 | controller | - GitServer: OK 2026-06-09 20:20:12.091578 | controller | - HoundSearch: OK 2026-06-09 20:20:12.091583 | controller | - LogJuicer: OK 2026-06-09 20:20:12.091588 | controller | - Logserver: OK 2026-06-09 20:20:12.091593 | controller | - MariaDB: OK 2026-06-09 20:20:12.091598 | controller | - NodePoolBuilder: OK 2026-06-09 20:20:12.091602 | controller | - NodePoolLauncher: OK 2026-06-09 20:20:12.091607 | controller | - Zookeeper: OK 2026-06-09 20:20:12.091619 | controller | - Zuul: OK 2026-06-09 20:20:12.091627 | controller | 2026-06-09 20:20:13.096938 | controller | 2026-06-09T20:20:13-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:13.116447 | controller | 2026-06-09T20:20:13-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:13.128483 | controller | 2026-06-09T20:20:13-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:13.701700 | controller | 2026-06-09T20:20:13-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:13.701784 | controller | 2026-06-09T20:20:13-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:13.704216 | controller | 2026-06-09T20:20:13-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:13.714610 | controller | 2026-06-09T20:20:13-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:13.714668 | controller | 2026-06-09T20:20:13-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:13.723439 | controller | 2026-06-09T20:20:13-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:13.735102 | controller | 2026-06-09T20:20:13-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:13.735192 | controller | 2026-06-09T20:20:13-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:14.105452 | controller | 2026-06-09T20:20:14-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:14.107707 | controller | 2026-06-09T20:20:14-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:14.512057 | controller | 2026-06-09T20:20:14-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:14.512127 | controller | 2026-06-09T20:20:14-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:14.709507 | controller | 2026-06-09T20:20:14-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:15.515545 | controller | 2026-06-09T20:20:15-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:15.521947 | controller | 2026-06-09T20:20:15-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:15.521984 | controller | 2026-06-09T20:20:15-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:15.708826 | controller | 2026-06-09T20:20:15-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:16.501506 | controller | 2026-06-09T20:20:16-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:20:17.309508 | controller | 2026-06-09T20:20:17-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:20:17.312803 | controller | 2026-06-09T20:20:17-04:00 INFO 2026-06-09 20:20:17.312835 | controller | - Config: Waiting ... 2026-06-09 20:20:17.312845 | controller | - Gateway: OK 2026-06-09 20:20:17.312853 | controller | - GitServer: OK 2026-06-09 20:20:17.312861 | controller | - HoundSearch: OK 2026-06-09 20:20:17.312869 | controller | - LogJuicer: OK 2026-06-09 20:20:17.312877 | controller | - Logserver: OK 2026-06-09 20:20:17.312884 | controller | - MariaDB: OK 2026-06-09 20:20:17.312892 | controller | - NodePoolBuilder: OK 2026-06-09 20:20:17.312900 | controller | - NodePoolLauncher: OK 2026-06-09 20:20:17.312908 | controller | - Zookeeper: OK 2026-06-09 20:20:17.312915 | controller | - Zuul: Waiting ... 2026-06-09 20:20:17.312923 | controller | - |_ Config Secrets: OK 2026-06-09 20:20:17.312931 | controller | - |_ Executor: OK 2026-06-09 20:20:17.312939 | controller | - |_ Merger: OK 2026-06-09 20:20:17.312946 | controller | - |_ Scheduler: OK 2026-06-09 20:20:17.312954 | controller | - |_ Web: Waiting ... 2026-06-09 20:20:17.312962 | controller | 2026-06-09 20:20:18.319119 | controller | 2026-06-09T20:20:18-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:18.338821 | controller | 2026-06-09T20:20:18-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:18.353466 | controller | 2026-06-09T20:20:18-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:18.925815 | controller | 2026-06-09T20:20:18-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:18.925889 | controller | 2026-06-09T20:20:18-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:18.928750 | controller | 2026-06-09T20:20:18-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:18.937096 | controller | 2026-06-09T20:20:18-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:18.937152 | controller | 2026-06-09T20:20:18-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:18.945641 | controller | 2026-06-09T20:20:18-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:18.957223 | controller | 2026-06-09T20:20:18-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:18.957339 | controller | 2026-06-09T20:20:18-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:19.330627 | controller | 2026-06-09T20:20:19-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:19.333884 | controller | 2026-06-09T20:20:19-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:19.736330 | controller | 2026-06-09T20:20:19-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:19.736410 | controller | 2026-06-09T20:20:19-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:20.303932 | controller | 2026-06-09T20:20:20-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:21.102019 | controller | 2026-06-09T20:20:21-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:21.106762 | controller | 2026-06-09T20:20:21-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:21.106779 | controller | 2026-06-09T20:20:21-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:21.906632 | controller | 2026-06-09T20:20:21-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:22.702391 | controller | 2026-06-09T20:20:22-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:20:22.702472 | controller | 2026-06-09T20:20:22-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:20:23.507971 | controller | 2026-06-09T20:20:23-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:20:23.511607 | controller | 2026-06-09T20:20:23-04:00 INFO 2026-06-09 20:20:23.511659 | controller | - Config: Waiting ... 2026-06-09 20:20:23.511671 | controller | - Gateway: OK 2026-06-09 20:20:23.511679 | controller | - GitServer: OK 2026-06-09 20:20:23.511687 | controller | - HoundSearch: OK 2026-06-09 20:20:23.511695 | controller | - LogJuicer: OK 2026-06-09 20:20:23.511703 | controller | - Logserver: OK 2026-06-09 20:20:23.511710 | controller | - MariaDB: OK 2026-06-09 20:20:23.511718 | controller | - NodePoolBuilder: OK 2026-06-09 20:20:23.511726 | controller | - NodePoolLauncher: OK 2026-06-09 20:20:23.511733 | controller | - Zookeeper: OK 2026-06-09 20:20:23.511741 | controller | - Zuul: Waiting ... 2026-06-09 20:20:23.511817 | controller | - |_ Config Secrets: OK 2026-06-09 20:20:23.511829 | controller | - |_ Executor: OK 2026-06-09 20:20:23.511837 | controller | - |_ Merger: OK 2026-06-09 20:20:23.511845 | controller | - |_ Scheduler: OK 2026-06-09 20:20:23.511852 | controller | - |_ Web: Waiting ... 2026-06-09 20:20:23.511861 | controller | 2026-06-09 20:20:24.518943 | controller | 2026-06-09T20:20:24-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:24.547060 | controller | 2026-06-09T20:20:24-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:24.591742 | controller | 2026-06-09T20:20:24-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:25.121310 | controller | 2026-06-09T20:20:25-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:25.121378 | controller | 2026-06-09T20:20:25-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:25.123650 | controller | 2026-06-09T20:20:25-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:25.132354 | controller | 2026-06-09T20:20:25-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:25.132392 | controller | 2026-06-09T20:20:25-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:25.139833 | controller | 2026-06-09T20:20:25-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:25.150512 | controller | 2026-06-09T20:20:25-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:25.150556 | controller | 2026-06-09T20:20:25-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:25.534839 | controller | 2026-06-09T20:20:25-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:25.538107 | controller | 2026-06-09T20:20:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:25.937193 | controller | 2026-06-09T20:20:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:25.937299 | controller | 2026-06-09T20:20:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:26.502204 | controller | 2026-06-09T20:20:26-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:27.306320 | controller | 2026-06-09T20:20:27-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:27.314707 | controller | 2026-06-09T20:20:27-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:27.314770 | controller | 2026-06-09T20:20:27-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:28.111445 | controller | 2026-06-09T20:20:28-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:28.902834 | controller | 2026-06-09T20:20:28-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:20:28.902908 | controller | 2026-06-09T20:20:28-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:20:29.708663 | controller | 2026-06-09T20:20:29-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:20:29.711689 | controller | 2026-06-09T20:20:29-04:00 INFO 2026-06-09 20:20:29.711738 | controller | - Config: Waiting ... 2026-06-09 20:20:29.711752 | controller | - Gateway: OK 2026-06-09 20:20:29.711765 | controller | - GitServer: OK 2026-06-09 20:20:29.711777 | controller | - HoundSearch: OK 2026-06-09 20:20:29.711789 | controller | - LogJuicer: OK 2026-06-09 20:20:29.711801 | controller | - Logserver: OK 2026-06-09 20:20:29.711813 | controller | - MariaDB: OK 2026-06-09 20:20:29.711826 | controller | - NodePoolBuilder: OK 2026-06-09 20:20:29.711838 | controller | - NodePoolLauncher: OK 2026-06-09 20:20:29.711850 | controller | - Zookeeper: OK 2026-06-09 20:20:29.711863 | controller | - Zuul: Waiting ... 2026-06-09 20:20:29.711876 | controller | - |_ Config Secrets: OK 2026-06-09 20:20:29.711897 | controller | - |_ Executor: OK 2026-06-09 20:20:29.711910 | controller | - |_ Merger: OK 2026-06-09 20:20:29.711941 | controller | - |_ Scheduler: OK 2026-06-09 20:20:29.711954 | controller | - |_ Web: Waiting ... 2026-06-09 20:20:29.711966 | controller | 2026-06-09 20:20:30.718334 | controller | 2026-06-09T20:20:30-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:30.751817 | controller | 2026-06-09T20:20:30-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:30.772424 | controller | 2026-06-09T20:20:30-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:31.322615 | controller | 2026-06-09T20:20:31-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:31.322682 | controller | 2026-06-09T20:20:31-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:31.324432 | controller | 2026-06-09T20:20:31-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:31.335840 | controller | 2026-06-09T20:20:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:31.335865 | controller | 2026-06-09T20:20:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:31.345612 | controller | 2026-06-09T20:20:31-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:31.359266 | controller | 2026-06-09T20:20:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:31.359300 | controller | 2026-06-09T20:20:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:31.727800 | controller | 2026-06-09T20:20:31-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:31.731049 | controller | 2026-06-09T20:20:31-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:32.133817 | controller | 2026-06-09T20:20:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:32.133889 | controller | 2026-06-09T20:20:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:32.706197 | controller | 2026-06-09T20:20:32-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:33.505043 | controller | 2026-06-09T20:20:33-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:33.513381 | controller | 2026-06-09T20:20:33-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:33.513418 | controller | 2026-06-09T20:20:33-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:34.304748 | controller | 2026-06-09T20:20:34-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:35.103126 | controller | 2026-06-09T20:20:35-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:20:35.103216 | controller | 2026-06-09T20:20:35-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:20:35.911765 | controller | 2026-06-09T20:20:35-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:20:35.914350 | controller | 2026-06-09T20:20:35-04:00 INFO 2026-06-09 20:20:35.914403 | controller | - Config: Waiting ... 2026-06-09 20:20:35.914420 | controller | - Gateway: OK 2026-06-09 20:20:35.914430 | controller | - GitServer: OK 2026-06-09 20:20:35.914439 | controller | - HoundSearch: OK 2026-06-09 20:20:35.914448 | controller | - LogJuicer: OK 2026-06-09 20:20:35.914456 | controller | - Logserver: OK 2026-06-09 20:20:35.914465 | controller | - MariaDB: OK 2026-06-09 20:20:35.914473 | controller | - NodePoolBuilder: OK 2026-06-09 20:20:35.914481 | controller | - NodePoolLauncher: OK 2026-06-09 20:20:35.914489 | controller | - Zookeeper: OK 2026-06-09 20:20:35.914498 | controller | - Zuul: Waiting ... 2026-06-09 20:20:35.914506 | controller | - |_ Config Secrets: OK 2026-06-09 20:20:35.914515 | controller | - |_ Executor: OK 2026-06-09 20:20:35.914523 | controller | - |_ Merger: OK 2026-06-09 20:20:35.914531 | controller | - |_ Scheduler: OK 2026-06-09 20:20:35.914540 | controller | - |_ Web: Waiting ... 2026-06-09 20:20:35.914549 | controller | 2026-06-09 20:20:36.918434 | controller | 2026-06-09T20:20:36-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:36.939033 | controller | 2026-06-09T20:20:36-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:36.949714 | controller | 2026-06-09T20:20:36-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:37.525160 | controller | 2026-06-09T20:20:37-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:37.525326 | controller | 2026-06-09T20:20:37-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:37.527837 | controller | 2026-06-09T20:20:37-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:37.537743 | controller | 2026-06-09T20:20:37-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:37.537811 | controller | 2026-06-09T20:20:37-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:37.547809 | controller | 2026-06-09T20:20:37-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:37.562384 | controller | 2026-06-09T20:20:37-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:37.562428 | controller | 2026-06-09T20:20:37-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:37.932802 | controller | 2026-06-09T20:20:37-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:37.936022 | controller | 2026-06-09T20:20:37-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:38.337751 | controller | 2026-06-09T20:20:38-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:38.337827 | controller | 2026-06-09T20:20:38-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:38.908292 | controller | 2026-06-09T20:20:38-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:39.705714 | controller | 2026-06-09T20:20:39-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:39.714644 | controller | 2026-06-09T20:20:39-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:39.714706 | controller | 2026-06-09T20:20:39-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:40.508511 | controller | 2026-06-09T20:20:40-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:41.302886 | controller | 2026-06-09T20:20:41-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:20:41.302960 | controller | 2026-06-09T20:20:41-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:20:42.108496 | controller | 2026-06-09T20:20:42-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:20:42.111506 | controller | 2026-06-09T20:20:42-04:00 INFO 2026-06-09 20:20:42.111542 | controller | - Config: Waiting ... 2026-06-09 20:20:42.111552 | controller | - Gateway: OK 2026-06-09 20:20:42.111560 | controller | - GitServer: OK 2026-06-09 20:20:42.111568 | controller | - HoundSearch: OK 2026-06-09 20:20:42.111587 | controller | - LogJuicer: OK 2026-06-09 20:20:42.111595 | controller | - Logserver: OK 2026-06-09 20:20:42.111603 | controller | - MariaDB: OK 2026-06-09 20:20:42.111611 | controller | - NodePoolBuilder: OK 2026-06-09 20:20:42.111618 | controller | - NodePoolLauncher: OK 2026-06-09 20:20:42.111626 | controller | - Zookeeper: OK 2026-06-09 20:20:42.111634 | controller | - Zuul: Waiting ... 2026-06-09 20:20:42.111641 | controller | - |_ Config Secrets: OK 2026-06-09 20:20:42.111649 | controller | - |_ Executor: OK 2026-06-09 20:20:42.111657 | controller | - |_ Merger: OK 2026-06-09 20:20:42.111664 | controller | - |_ Scheduler: OK 2026-06-09 20:20:42.111672 | controller | - |_ Web: Waiting ... 2026-06-09 20:20:42.111683 | controller | 2026-06-09 20:20:43.117315 | controller | 2026-06-09T20:20:43-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:43.146104 | controller | 2026-06-09T20:20:43-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:43.164185 | controller | 2026-06-09T20:20:43-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:43.722647 | controller | 2026-06-09T20:20:43-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:43.722705 | controller | 2026-06-09T20:20:43-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:43.724929 | controller | 2026-06-09T20:20:43-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:43.733032 | controller | 2026-06-09T20:20:43-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:43.733056 | controller | 2026-06-09T20:20:43-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:43.741332 | controller | 2026-06-09T20:20:43-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:43.750136 | controller | 2026-06-09T20:20:43-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:43.750159 | controller | 2026-06-09T20:20:43-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:44.127941 | controller | 2026-06-09T20:20:44-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:44.130341 | controller | 2026-06-09T20:20:44-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:44.534022 | controller | 2026-06-09T20:20:44-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:44.534083 | controller | 2026-06-09T20:20:44-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:45.105603 | controller | 2026-06-09T20:20:45-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:45.905116 | controller | 2026-06-09T20:20:45-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:45.913282 | controller | 2026-06-09T20:20:45-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:45.913322 | controller | 2026-06-09T20:20:45-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:46.705882 | controller | 2026-06-09T20:20:46-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:47.501152 | controller | 2026-06-09T20:20:47-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:20:47.501256 | controller | 2026-06-09T20:20:47-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:20:48.310197 | controller | 2026-06-09T20:20:48-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:20:48.313194 | controller | 2026-06-09T20:20:48-04:00 INFO 2026-06-09 20:20:48.313259 | controller | - Config: Waiting ... 2026-06-09 20:20:48.313273 | controller | - Gateway: OK 2026-06-09 20:20:48.313281 | controller | - GitServer: OK 2026-06-09 20:20:48.313290 | controller | - HoundSearch: OK 2026-06-09 20:20:48.313298 | controller | - LogJuicer: OK 2026-06-09 20:20:48.313305 | controller | - Logserver: OK 2026-06-09 20:20:48.313313 | controller | - MariaDB: OK 2026-06-09 20:20:48.313321 | controller | - NodePoolBuilder: OK 2026-06-09 20:20:48.313329 | controller | - NodePoolLauncher: OK 2026-06-09 20:20:48.313336 | controller | - Zookeeper: OK 2026-06-09 20:20:48.313344 | controller | - Zuul: Waiting ... 2026-06-09 20:20:48.313352 | controller | - |_ Config Secrets: OK 2026-06-09 20:20:48.313360 | controller | - |_ Executor: OK 2026-06-09 20:20:48.313367 | controller | - |_ Merger: OK 2026-06-09 20:20:48.313375 | controller | - |_ Scheduler: OK 2026-06-09 20:20:48.313382 | controller | - |_ Web: Waiting ... 2026-06-09 20:20:48.313390 | controller | 2026-06-09 20:20:49.319992 | controller | 2026-06-09T20:20:49-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:49.348180 | controller | 2026-06-09T20:20:49-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:49.361979 | controller | 2026-06-09T20:20:49-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:49.925557 | controller | 2026-06-09T20:20:49-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:49.925636 | controller | 2026-06-09T20:20:49-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:49.928546 | controller | 2026-06-09T20:20:49-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:49.939737 | controller | 2026-06-09T20:20:49-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:49.939790 | controller | 2026-06-09T20:20:49-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:49.949737 | controller | 2026-06-09T20:20:49-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:49.968050 | controller | 2026-06-09T20:20:49-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:49.968086 | controller | 2026-06-09T20:20:49-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:50.332644 | controller | 2026-06-09T20:20:50-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:50.335642 | controller | 2026-06-09T20:20:50-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:50.741342 | controller | 2026-06-09T20:20:50-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:50.741419 | controller | 2026-06-09T20:20:50-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:51.306760 | controller | 2026-06-09T20:20:51-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:52.105147 | controller | 2026-06-09T20:20:52-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:52.113811 | controller | 2026-06-09T20:20:52-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:52.113886 | controller | 2026-06-09T20:20:52-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:52.909332 | controller | 2026-06-09T20:20:52-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:53.701600 | controller | 2026-06-09T20:20:53-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:20:53.701670 | controller | 2026-06-09T20:20:53-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:20:54.508259 | controller | 2026-06-09T20:20:54-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:20:54.511561 | controller | 2026-06-09T20:20:54-04:00 INFO 2026-06-09 20:20:54.511600 | controller | - Config: Waiting ... 2026-06-09 20:20:54.511607 | controller | - Gateway: OK 2026-06-09 20:20:54.511613 | controller | - GitServer: OK 2026-06-09 20:20:54.511619 | controller | - HoundSearch: OK 2026-06-09 20:20:54.511626 | controller | - LogJuicer: OK 2026-06-09 20:20:54.511633 | controller | - Logserver: OK 2026-06-09 20:20:54.511640 | controller | - MariaDB: OK 2026-06-09 20:20:54.511647 | controller | - NodePoolBuilder: OK 2026-06-09 20:20:54.511654 | controller | - NodePoolLauncher: OK 2026-06-09 20:20:54.511661 | controller | - Zookeeper: OK 2026-06-09 20:20:54.511667 | controller | - Zuul: Waiting ... 2026-06-09 20:20:54.511677 | controller | - |_ Config Secrets: OK 2026-06-09 20:20:54.511682 | controller | - |_ Executor: OK 2026-06-09 20:20:54.511687 | controller | - |_ Merger: OK 2026-06-09 20:20:54.511691 | controller | - |_ Scheduler: OK 2026-06-09 20:20:54.511696 | controller | - |_ Web: Waiting ... 2026-06-09 20:20:54.511701 | controller | 2026-06-09 20:20:55.519401 | controller | 2026-06-09T20:20:55-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:20:55.537429 | controller | 2026-06-09T20:20:55-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:20:55.548395 | controller | 2026-06-09T20:20:55-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:20:56.121571 | controller | 2026-06-09T20:20:56-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:20:56.121659 | controller | 2026-06-09T20:20:56-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:20:56.123954 | controller | 2026-06-09T20:20:56-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:20:56.131664 | controller | 2026-06-09T20:20:56-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:20:56.131716 | controller | 2026-06-09T20:20:56-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:20:56.139745 | controller | 2026-06-09T20:20:56-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:20:56.148624 | controller | 2026-06-09T20:20:56-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:20:56.148654 | controller | 2026-06-09T20:20:56-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:20:56.528510 | controller | 2026-06-09T20:20:56-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:20:56.530933 | controller | 2026-06-09T20:20:56-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:20:56.936321 | controller | 2026-06-09T20:20:56-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:20:56.936432 | controller | 2026-06-09T20:20:56-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:20:57.503923 | controller | 2026-06-09T20:20:57-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:20:58.306587 | controller | 2026-06-09T20:20:58-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:20:58.317715 | controller | 2026-06-09T20:20:58-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:20:58.317781 | controller | 2026-06-09T20:20:58-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:20:59.111616 | controller | 2026-06-09T20:20:59-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:20:59.901638 | controller | 2026-06-09T20:20:59-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:20:59.901747 | controller | 2026-06-09T20:20:59-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:21:00.707952 | controller | 2026-06-09T20:21:00-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:21:00.711489 | controller | 2026-06-09T20:21:00-04:00 INFO 2026-06-09 20:21:00.711532 | controller | - Config: Waiting ... 2026-06-09 20:21:00.711543 | controller | - Gateway: OK 2026-06-09 20:21:00.711552 | controller | - GitServer: OK 2026-06-09 20:21:00.711560 | controller | - HoundSearch: OK 2026-06-09 20:21:00.711568 | controller | - LogJuicer: OK 2026-06-09 20:21:00.711576 | controller | - Logserver: OK 2026-06-09 20:21:00.711594 | controller | - MariaDB: OK 2026-06-09 20:21:00.711602 | controller | - NodePoolBuilder: OK 2026-06-09 20:21:00.711609 | controller | - NodePoolLauncher: OK 2026-06-09 20:21:00.711617 | controller | - Zookeeper: OK 2026-06-09 20:21:00.711625 | controller | - Zuul: Waiting ... 2026-06-09 20:21:00.711633 | controller | - |_ Config Secrets: OK 2026-06-09 20:21:00.711641 | controller | - |_ Executor: OK 2026-06-09 20:21:00.711649 | controller | - |_ Merger: OK 2026-06-09 20:21:00.711656 | controller | - |_ Scheduler: OK 2026-06-09 20:21:00.711664 | controller | - |_ Web: Waiting ... 2026-06-09 20:21:00.711673 | controller | 2026-06-09 20:21:01.718490 | controller | 2026-06-09T20:21:01-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:21:01.750508 | controller | 2026-06-09T20:21:01-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:21:01.771180 | controller | 2026-06-09T20:21:01-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:21:02.325407 | controller | 2026-06-09T20:21:02-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:21:02.325487 | controller | 2026-06-09T20:21:02-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:21:02.328647 | controller | 2026-06-09T20:21:02-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:21:02.340643 | controller | 2026-06-09T20:21:02-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:21:02.340679 | controller | 2026-06-09T20:21:02-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:21:02.352764 | controller | 2026-06-09T20:21:02-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:21:02.368678 | controller | 2026-06-09T20:21:02-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:21:02.368745 | controller | 2026-06-09T20:21:02-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:21:02.738951 | controller | 2026-06-09T20:21:02-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:21:02.743283 | controller | 2026-06-09T20:21:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:21:03.132577 | controller | 2026-06-09T20:21:03-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:21:03.132641 | controller | 2026-06-09T20:21:03-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:21:03.712362 | controller | 2026-06-09T20:21:03-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:21:04.506098 | controller | 2026-06-09T20:21:04-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:21:04.514552 | controller | 2026-06-09T20:21:04-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:21:04.514630 | controller | 2026-06-09T20:21:04-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:21:05.309697 | controller | 2026-06-09T20:21:05-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:21:06.103165 | controller | 2026-06-09T20:21:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:21:06.103266 | controller | 2026-06-09T20:21:06-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:21:06.909970 | controller | 2026-06-09T20:21:06-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:21:06.913470 | controller | 2026-06-09T20:21:06-04:00 INFO 2026-06-09 20:21:06.913550 | controller | - Config: Waiting ... 2026-06-09 20:21:06.913562 | controller | - Gateway: OK 2026-06-09 20:21:06.913574 | controller | - GitServer: OK 2026-06-09 20:21:06.913585 | controller | - HoundSearch: OK 2026-06-09 20:21:06.913597 | controller | - LogJuicer: OK 2026-06-09 20:21:06.913609 | controller | - Logserver: OK 2026-06-09 20:21:06.913620 | controller | - MariaDB: OK 2026-06-09 20:21:06.913633 | controller | - NodePoolBuilder: OK 2026-06-09 20:21:06.913645 | controller | - NodePoolLauncher: OK 2026-06-09 20:21:06.913656 | controller | - Zookeeper: OK 2026-06-09 20:21:06.913668 | controller | - Zuul: Waiting ... 2026-06-09 20:21:06.913680 | controller | - |_ Config Secrets: OK 2026-06-09 20:21:06.913692 | controller | - |_ Executor: OK 2026-06-09 20:21:06.913703 | controller | - |_ Merger: OK 2026-06-09 20:21:06.913714 | controller | - |_ Scheduler: OK 2026-06-09 20:21:06.913725 | controller | - |_ Web: Waiting ... 2026-06-09 20:21:06.913736 | controller | 2026-06-09 20:21:07.918277 | controller | 2026-06-09T20:21:07-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:21:07.939861 | controller | 2026-06-09T20:21:07-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:21:07.959093 | controller | 2026-06-09T20:21:07-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:21:08.523161 | controller | 2026-06-09T20:21:08-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:21:08.523280 | controller | 2026-06-09T20:21:08-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:21:08.526590 | controller | 2026-06-09T20:21:08-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:21:08.537058 | controller | 2026-06-09T20:21:08-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:21:08.537108 | controller | 2026-06-09T20:21:08-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:21:08.545173 | controller | 2026-06-09T20:21:08-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:21:08.554843 | controller | 2026-06-09T20:21:08-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:21:08.554881 | controller | 2026-06-09T20:21:08-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:21:08.929144 | controller | 2026-06-09T20:21:08-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:21:08.931563 | controller | 2026-06-09T20:21:08-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:21:09.330943 | controller | 2026-06-09T20:21:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:21:09.331014 | controller | 2026-06-09T20:21:09-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:21:09.905879 | controller | 2026-06-09T20:21:09-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:21:10.703179 | controller | 2026-06-09T20:21:10-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:21:10.714413 | controller | 2026-06-09T20:21:10-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:21:10.714477 | controller | 2026-06-09T20:21:10-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:21:11.511367 | controller | 2026-06-09T20:21:11-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:21:12.301538 | controller | 2026-06-09T20:21:12-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:21:12.301687 | controller | 2026-06-09T20:21:12-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:21:13.109269 | controller | 2026-06-09T20:21:13-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:21:13.112844 | controller | 2026-06-09T20:21:13-04:00 INFO 2026-06-09 20:21:13.112878 | controller | - Config: Waiting ... 2026-06-09 20:21:13.112889 | controller | - Gateway: OK 2026-06-09 20:21:13.112897 | controller | - GitServer: OK 2026-06-09 20:21:13.112905 | controller | - HoundSearch: OK 2026-06-09 20:21:13.112913 | controller | - LogJuicer: OK 2026-06-09 20:21:13.112921 | controller | - Logserver: OK 2026-06-09 20:21:13.112928 | controller | - MariaDB: OK 2026-06-09 20:21:13.112936 | controller | - NodePoolBuilder: OK 2026-06-09 20:21:13.112943 | controller | - NodePoolLauncher: OK 2026-06-09 20:21:13.112951 | controller | - Zookeeper: OK 2026-06-09 20:21:13.112959 | controller | - Zuul: Waiting ... 2026-06-09 20:21:13.112966 | controller | - |_ Config Secrets: OK 2026-06-09 20:21:13.112974 | controller | - |_ Executor: OK 2026-06-09 20:21:13.112982 | controller | - |_ Merger: OK 2026-06-09 20:21:13.112989 | controller | - |_ Scheduler: OK 2026-06-09 20:21:13.112997 | controller | - |_ Web: Waiting ... 2026-06-09 20:21:13.113005 | controller | 2026-06-09 20:21:14.118779 | controller | 2026-06-09T20:21:14-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:21:14.163457 | controller | 2026-06-09T20:21:14-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:21:14.179674 | controller | 2026-06-09T20:21:14-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:21:14.724122 | controller | 2026-06-09T20:21:14-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:21:14.724198 | controller | 2026-06-09T20:21:14-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:21:14.726061 | controller | 2026-06-09T20:21:14-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:21:14.737425 | controller | 2026-06-09T20:21:14-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:21:14.737485 | controller | 2026-06-09T20:21:14-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:21:14.746532 | controller | 2026-06-09T20:21:14-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:21:14.757089 | controller | 2026-06-09T20:21:14-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:21:14.757135 | controller | 2026-06-09T20:21:14-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:21:15.130600 | controller | 2026-06-09T20:21:15-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:21:15.132955 | controller | 2026-06-09T20:21:15-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:21:15.534715 | controller | 2026-06-09T20:21:15-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:21:15.534785 | controller | 2026-06-09T20:21:15-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:21:16.109145 | controller | 2026-06-09T20:21:16-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:21:16.905780 | controller | 2026-06-09T20:21:16-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:21:16.914304 | controller | 2026-06-09T20:21:16-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:21:16.914345 | controller | 2026-06-09T20:21:16-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:21:17.707852 | controller | 2026-06-09T20:21:17-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:21:18.503065 | controller | 2026-06-09T20:21:18-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:21:18.503146 | controller | 2026-06-09T20:21:18-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:21:19.310481 | controller | 2026-06-09T20:21:19-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:21:19.712362 | controller | 2026-06-09T20:21:19-04:00 INFO 2026-06-09 20:21:19.712432 | controller | - Config: OK 2026-06-09 20:21:19.712472 | controller | - Gateway: OK 2026-06-09 20:21:19.712479 | controller | - GitServer: OK 2026-06-09 20:21:19.712484 | controller | - HoundSearch: OK 2026-06-09 20:21:19.712489 | controller | - LogJuicer: OK 2026-06-09 20:21:19.712494 | controller | - Logserver: OK 2026-06-09 20:21:19.712499 | controller | - MariaDB: OK 2026-06-09 20:21:19.712504 | controller | - NodePoolBuilder: OK 2026-06-09 20:21:19.712510 | controller | - NodePoolLauncher: OK 2026-06-09 20:21:19.712517 | controller | - Zookeeper: OK 2026-06-09 20:21:19.712534 | controller | - Zuul: OK 2026-06-09 20:21:19.712544 | controller | 2026-06-09 20:21:19.712558 | controller | • [160.936 seconds] 2026-06-09 20:21:19.712566 | controller | ------------------------------ 2026-06-09 20:21:19.712625 | controller | Project Private Keys Rotation zuul-web config-errors remain empty after rotation 2026-06-09 20:21:19.712633 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:103 2026-06-09 20:21:19.712771 | controller | STEP: Ensuring tenant have no config errors @ 06/09/26 20:21:19.712 2026-06-09 20:21:19.712807 | controller | 2026-06-09T20:21:19-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/internal/config-errors] 2026-06-09 20:21:19.813892 | controller | 2026-06-09T20:21:19-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/demo-tenant/config-errors] 2026-06-09 20:21:19.936530 | controller | • [0.224 seconds] 2026-06-09 20:21:19.936580 | controller | ------------------------------ 2026-06-09 20:21:19.936617 | controller | Project Private Keys Rotation project private keys changed when rotation was applied 2026-06-09 20:21:19.936626 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:107 2026-06-09 20:21:19.936760 | controller | 2026-06-09T20:21:19-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-09 20:21:19.936778 | controller | import configparser 2026-06-09 20:21:19.936783 | controller | import hashlib 2026-06-09 20:21:19.936800 | controller | from zuul.zk import ZooKeeperClient 2026-06-09 20:21:19.936836 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-09 20:21:19.936852 | controller | from zuul.lib import encryption 2026-06-09 20:21:19.936857 | controller | 2026-06-09 20:21:19.936862 | controller | config = configparser.ConfigParser() 2026-06-09 20:21:19.936867 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-09 20:21:19.936873 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-09 20:21:19.936883 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-09 20:21:19.936888 | controller | zk_client.connect() 2026-06-09 20:21:19.936893 | controller | lines = [] 2026-06-09 20:21:19.936898 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-09 20:21:19.936904 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-09 20:21:19.936909 | controller | created = obj["keys"][0]["created"] 2026-06-09 20:21:19.936914 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-09 20:21:19.936919 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-09 20:21:19.936924 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-09 20:21:19.936929 | controller | if "demo-project" in path: 2026-06-09 20:21:19.936961 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-09 20:21:19.936969 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-09 20:21:19.936975 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-09 20:21:19.936980 | controller | f.write(pem) 2026-06-09 20:21:19.936985 | controller | print("\n".join(sorted(lines))) 2026-06-09 20:21:19.936990 | controller | PYEOF 2026-06-09 20:21:19.936995 | controller | ] 2026-06-09 20:21:21.277216 | controller | STEP: Project keys after rotation: /keystorage/gerrit/config/config/secrets 1781050740 b731017c3d4435ae 2026-06-09 20:21:21.277304 | controller | /keystorage/gerrit/demo-project/demo-project/secrets 1781050722 18520e65a33fa213 2026-06-09 20:21:21.277311 | controller | /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 1781050737 bd908a6bc0a770a7 2026-06-09 20:21:21.277315 | controller | /keystorage/git-server/system-config/system-config/secrets 1781050741 fd663e97de08a969 2026-06-09 20:21:21.277319 | controller | /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 1781050738 a7f73a27ec1dd2f6 2026-06-09 20:21:21.277353 | controller | @ 06/09/26 20:21:21.277 2026-06-09 20:21:21.277411 | controller | STEP: Pulling latest demo-project and reading .zuul.d/secrets.yaml @ 06/09/26 20:21:21.277 2026-06-09 20:21:21.366432 | controller | STEP: Writing secrets.yaml into zuul-scheduler pod for decryption @ 06/09/26 20:21:21.366 2026-06-09 20:21:21.366469 | controller | 2026-06-09T20:21:21-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat > /tmp/secrets.yaml] 2026-06-09 20:21:21.466907 | controller | STEP: Loading project keys state (writes demo-project key to /tmp/demo-project.key) @ 06/09/26 20:21:21.466 2026-06-09 20:21:21.466968 | controller | 2026-06-09T20:21:21-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-09 20:21:21.466973 | controller | import configparser 2026-06-09 20:21:21.466978 | controller | import hashlib 2026-06-09 20:21:21.466982 | controller | from zuul.zk import ZooKeeperClient 2026-06-09 20:21:21.466986 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-09 20:21:21.466990 | controller | from zuul.lib import encryption 2026-06-09 20:21:21.466994 | controller | 2026-06-09 20:21:21.466997 | controller | config = configparser.ConfigParser() 2026-06-09 20:21:21.467001 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-09 20:21:21.467004 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-09 20:21:21.467008 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-09 20:21:21.467012 | controller | zk_client.connect() 2026-06-09 20:21:21.467015 | controller | lines = [] 2026-06-09 20:21:21.467019 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-09 20:21:21.467022 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-09 20:21:21.467026 | controller | created = obj["keys"][0]["created"] 2026-06-09 20:21:21.467030 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-09 20:21:21.467033 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-09 20:21:21.467037 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-09 20:21:21.467040 | controller | if "demo-project" in path: 2026-06-09 20:21:21.467044 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-09 20:21:21.467053 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-09 20:21:21.467057 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-09 20:21:21.467060 | controller | f.write(pem) 2026-06-09 20:21:21.467064 | controller | print("\n".join(sorted(lines))) 2026-06-09 20:21:21.467067 | controller | PYEOF 2026-06-09 20:21:21.467071 | controller | ] 2026-06-09 20:21:22.590757 | controller | STEP: Decrypting with Zuul library @ 06/09/26 20:21:22.59 2026-06-09 20:21:22.590837 | controller | 2026-06-09T20:21:22-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-09 20:21:22.590848 | controller | from zuul.lib import encryption 2026-06-09 20:21:22.590858 | controller | import zuul.configloader 2026-06-09 20:21:22.590867 | controller | import zuul.model 2026-06-09 20:21:22.590875 | controller | 2026-06-09 20:21:22.590883 | controller | with open("/tmp/demo-project.key", "rb") as f: 2026-06-09 20:21:22.590892 | controller | private_secrets_key, _ = encryption.deserialize_rsa_keypair(f.read()) 2026-06-09 20:21:22.590900 | controller | with open("/tmp/secrets.yaml") as f: 2026-06-09 20:21:22.590908 | controller | yaml_content = f.read() 2026-06-09 20:21:22.590916 | controller | 2026-06-09 20:21:22.590924 | controller | parser = zuul.configloader.SecretParser(None) 2026-06-09 20:21:22.590932 | controller | sc = zuul.model.SourceContext(None, "project", None, "master", "path") 2026-06-09 20:21:22.590940 | controller | data = zuul.configloader.safe_load_yaml(yaml_content, sc) 2026-06-09 20:21:22.590948 | controller | for element in data: 2026-06-09 20:21:22.590956 | controller | if "secret" not in element: 2026-06-09 20:21:22.590964 | controller | continue 2026-06-09 20:21:22.590972 | controller | secret = parser.fromYaml(element["secret"]) 2026-06-09 20:21:22.590980 | controller | decrypted = secret.decrypt(private_secrets_key).secret_data 2026-06-09 20:21:22.590988 | controller | if isinstance(decrypted, dict) and "password" in decrypted: 2026-06-09 20:21:22.590996 | controller | print(decrypted["password"], end="") 2026-06-09 20:21:22.591004 | controller | elif isinstance(decrypted, str): 2026-06-09 20:21:22.591011 | controller | print(decrypted, end="") 2026-06-09 20:21:22.591019 | controller | else: 2026-06-09 20:21:22.591027 | controller | print(decrypted, end="") 2026-06-09 20:21:22.591035 | controller | break 2026-06-09 20:21:22.591043 | controller | PYEOF 2026-06-09 20:21:22.591051 | controller | ] 2026-06-09 20:21:23.781679 | controller | STEP: Decrypted secret: -----BEGIN OPENSSH PRIVATE KEY----- 2026-06-09 20:21:23.781740 | controller | b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW 2026-06-09 20:21:23.781746 | controller | QyNTUxOQAAACDCGkE2KXxnuIrkm8yr3DZ57JfWqkvy+g/5HYbYKsSVtgAAAKgPCt6rDwre 2026-06-09 20:21:23.781750 | controller | qwAAAAtzc2gtZWQyNTUxOQAAACDCGkE2KXxnuIrkm8yr3DZ57JfWqkvy+g/5HYbYKsSVtg 2026-06-09 20:21:23.781754 | controller | AAAECA8zXmTxTkOG79Oc8nnKe7Q68fIe0zjQYtaW98CuXw8cIaQTYpfGe4iuSbzKvcNnns 2026-06-09 20:21:23.781757 | controller | l9aqS/L6D/kdhtgqxJW2AAAAInp1dWwtd29ya2VyQG5wMDAwMDA2OTIwNi5ub3ZhbG9jYW 2026-06-09 20:21:23.781761 | controller | wBAgM= 2026-06-09 20:21:23.781765 | controller | -----END OPENSSH PRIVATE KEY----- @ 06/09/26 20:21:23.781 2026-06-09 20:21:23.781774 | controller | • [3.845 seconds] 2026-06-09 20:21:23.781778 | controller | ------------------------------ 2026-06-09 20:21:23.781782 | controller | Secret Rotations zuul-auth works 2026-06-09 20:21:23.781786 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:60 2026-06-09 20:21:23.786200 | controller | STEP: Deleting secrets not rotated by the CLI @ 06/09/26 20:21:23.786 2026-06-09 20:21:23.794867 | controller | STEP: Running secret rotation CLI @ 06/09/26 20:21:23.794 2026-06-09 20:21:23.803811 | controller | 2026-06-09T20:21:23-04:00 INFO Warning: At least one executor running, this may cause issues when rotating Zookeeper secrets. 2026-06-09 20:21:23.803875 | controller | 2026-06-09T20:21:23-04:00 INFO Rotating Keystore password... 2026-06-09 20:21:23.810359 | controller | 2026-06-09T20:21:23-04:00 INFO Creating secret, name: zuul-keystore-password-new 2026-06-09 20:21:23.810406 | controller | 2026-06-09T20:21:23-04:00 INFO Creating object, name: zuul-keystore-password-new 2026-06-09 20:21:24.470278 | controller | 2026-06-09T20:21:24-04:00 INFO Running pod execution pod: zuul-kazoo, command: [cat /var/lib/zuul/keys-backup.json] 2026-06-09 20:21:24.575363 | controller | 2026-06-09T20:21:24-04:00 INFO Updating object, name: zuul-keystore-password 2026-06-09 20:21:24.578617 | controller | 2026-06-09T20:21:24-04:00 INFO Deleting object, name: zuul-keystore-password-new 2026-06-09 20:21:24.582448 | controller | 2026-06-09T20:21:24-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-09 20:21:24.584311 | controller | 2026-06-09T20:21:24-04:00 INFO Killing every kazoo client... 2026-06-09 20:21:24.601309 | controller | 2026-06-09T20:21:24-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-09 20:21:24.601382 | controller | 2026-06-09T20:21:24-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-09 20:21:24.761415 | controller | 2026-06-09T20:21:24-04:00 INFO Deleting object, name: nodepool-builder 2026-06-09 20:21:24.764682 | controller | 2026-06-09T20:21:24-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-76d8b68c95-9bccz launcher 2026-06-09 20:21:24.764742 | controller | 2026-06-09T20:21:24-04:00 INFO Running pod execution pod: nodepool-launcher-76d8b68c95-9bccz, command: [kill -9 1] 2026-06-09 20:21:24.858579 | controller | 2026-06-09T20:21:24-04:00 INFO Deleting object, name: launcher 2026-06-09 20:21:24.860330 | controller | 2026-06-09T20:21:24-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-52vwg mariadb-client 2026-06-09 20:21:24.860355 | controller | 2026-06-09T20:21:24-04:00 INFO Running pod execution pod: zuul-db-provision-52vwg, command: [kill -9 1] 2026-06-09 20:21:24.874646 | controller | 2026-06-09T20:21:24-04:00 INFO Deleting object, name: mariadb-client 2026-06-09 20:21:24.876438 | controller | 2026-06-09T20:21:24-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-09 20:21:24.876461 | controller | 2026-06-09T20:21:24-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-09 20:22:26.483494 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-executor 2026-06-09 20:22:26.486404 | controller | 2026-06-09T20:22:26-04:00 INFO Warning: Killing ZooKeeper client: zuul-kazoo zuul-kazoo 2026-06-09 20:22:26.486471 | controller | 2026-06-09T20:22:26-04:00 INFO Running pod execution pod: zuul-kazoo, command: [kill -9 1] 2026-06-09 20:22:26.494555 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-09 20:22:26.497137 | controller | 2026-06-09T20:22:26-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-09 20:22:26.497264 | controller | 2026-06-09T20:22:26-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-09 20:22:26.615406 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-merger 2026-06-09 20:22:26.619895 | controller | 2026-06-09T20:22:26-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-09 20:22:26.619942 | controller | 2026-06-09T20:22:26-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-09 20:22:26.730372 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-09 20:22:26.735463 | controller | 2026-06-09T20:22:26-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-785b7c7c88-9cq6j zuul-web 2026-06-09 20:22:26.735509 | controller | 2026-06-09T20:22:26-04:00 INFO Running pod execution pod: zuul-web-785b7c7c88-9cq6j, command: [kill -9 1] 2026-06-09 20:22:26.847137 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-web 2026-06-09 20:22:26.851055 | controller | 2026-06-09T20:22:26-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-dsgjk zuul-weeder 2026-06-09 20:22:26.851083 | controller | 2026-06-09T20:22:26-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-dsgjk, command: [kill -9 1] 2026-06-09 20:22:26.931798 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-weeder 2026-06-09 20:22:26.935465 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zookeeper 2026-06-09 20:22:26.939356 | controller | 2026-06-09T20:22:26-04:00 INFO Rotating Zuul Client Authenticator secret... 2026-06-09 20:22:26.943616 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-auth-secret 2026-06-09 20:22:26.951451 | controller | 2026-06-09T20:22:26-04:00 INFO Creating secret, name: zuul-auth-secret 2026-06-09 20:22:26.951545 | controller | 2026-06-09T20:22:26-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-09 20:22:26.955531 | controller | 2026-06-09T20:22:26-04:00 INFO Rotating Zuul DB Connection secret... 2026-06-09 20:22:26.957890 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-db-connection 2026-06-09 20:22:26.960876 | controller | 2026-06-09T20:22:26-04:00 INFO Rotating Zookeeper certificates... 2026-06-09 20:22:26.964854 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-09 20:22:26.967757 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-09 20:22:26.971091 | controller | 2026-06-09T20:22:26-04:00 INFO Force Restart impacted services... 2026-06-09 20:22:26.971155 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 20:22:26.977097 | controller | 2026-06-09T20:22:26-04:00 INFO Deleting object, name: zuul-web-785b7c7c88-9cq6j 2026-06-09 20:22:26.981906 | controller | STEP: Reconciling @ 06/09/26 20:22:26.981 2026-06-09 20:22:27.171297 | controller | 2026-06-09T20:22:27-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-76d8b68c95-9bccz launcher 2026-06-09 20:22:27.171367 | controller | 2026-06-09T20:22:27-04:00 INFO Running pod execution pod: nodepool-launcher-76d8b68c95-9bccz, command: [kill -9 1] 2026-06-09 20:22:27.260247 | controller | 2026-06-09T20:22:27-04:00 INFO Deleting object, name: launcher 2026-06-09 20:22:27.262975 | controller | 2026-06-09T20:22:27-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-52vwg mariadb-client 2026-06-09 20:22:27.263023 | controller | 2026-06-09T20:22:27-04:00 INFO Running pod execution pod: zuul-db-provision-52vwg, command: [kill -9 1] 2026-06-09 20:22:27.281990 | controller | 2026-06-09T20:22:27-04:00 INFO Deleting object, name: mariadb-client 2026-06-09 20:22:27.284710 | controller | 2026-06-09T20:22:27-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-09 20:22:27.284740 | controller | 2026-06-09T20:22:27-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-09 20:22:28.118199 | controller | 2026-06-09T20:22:28-04:00 INFO Deleting object, name: zuul-merger 2026-06-09 20:22:28.122350 | controller | 2026-06-09T20:22:28-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-09 20:22:28.122423 | controller | 2026-06-09T20:22:28-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-09 20:22:56.933080 | controller | 2026-06-09T20:22:56-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-09 20:22:56.936103 | controller | 2026-06-09T20:22:56-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-785b7c7c88-9cq6j zuul-web 2026-06-09 20:22:56.936162 | controller | 2026-06-09T20:22:56-04:00 INFO Running pod execution pod: zuul-web-785b7c7c88-9cq6j, command: [kill -9 1] 2026-06-09 20:22:56.948737 | controller | 2026-06-09T20:22:56-04:00 INFO Deleting object, name: zuul-web 2026-06-09 20:22:56.951472 | controller | 2026-06-09T20:22:56-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-dsgjk zuul-weeder 2026-06-09 20:22:56.951525 | controller | 2026-06-09T20:22:56-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-dsgjk, command: [kill -9 1] 2026-06-09 20:22:56.962305 | controller | 2026-06-09T20:22:56-04:00 INFO Deleting object, name: zuul-weeder 2026-06-09 20:22:56.964501 | controller | 2026-06-09T20:22:56-04:00 INFO Deleting object, name: zookeeper 2026-06-09 20:22:56.970763 | controller | 2026-06-09T20:22:56-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:22:57.000356 | controller | 2026-06-09T20:22:57-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:22:57.018902 | controller | 2026-06-09T20:22:57-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:22:57.023196 | controller | 2026-06-09T20:22:57-04:00 INFO Creating object, name: zuul-db-connection 2026-06-09 20:22:57.029415 | controller | 2026-06-09T20:22:57-04:00 INFO Re-provisioning db password 2026-06-09 20:22:57.029472 | controller | 2026-06-09T20:22:57-04:00 INFO Deleting object, name: zuul-db-provision 2026-06-09 20:22:57.033363 | controller | 2026-06-09T20:22:57-04:00 INFO Creating object, name: zuul-db-provision 2026-06-09 20:22:57.583404 | controller | 2026-06-09T20:22:57-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:22:57.583490 | controller | 2026-06-09T20:22:57-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:22:57.586774 | controller | 2026-06-09T20:22:57-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:22:57.595937 | controller | 2026-06-09T20:22:57-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:22:57.762573 | controller | 2026-06-09T20:22:57-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:22:57.769985 | controller | 2026-06-09T20:22:57-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:22:57.770025 | controller | 2026-06-09T20:22:57-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:22:57.778918 | controller | 2026-06-09T20:22:57-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:22:57.790196 | controller | 2026-06-09T20:22:57-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:22:57.790227 | controller | 2026-06-09T20:22:57-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:22:57.791520 | controller | 2026-06-09T20:22:57-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-09 20:22:57.973632 | controller | 2026-06-09T20:22:57-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-09 20:22:59.607297 | controller | 2026-06-09T20:22:59-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-09 20:22:59.855523 | controller | 2026-06-09T20:22:59-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-09 20:22:59.868922 | controller | 2026-06-09T20:22:59-04:00 INFO Creating object, name: zookeeper 2026-06-09 20:22:59.873803 | controller | 2026-06-09T20:22:59-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-09 20:22:59.873902 | controller | 2026-06-09T20:22:59-04:00 INFO 2026-06-09 20:22:59.873910 | controller | - Config: Waiting ... 2026-06-09 20:22:59.873951 | controller | - Gateway: OK 2026-06-09 20:22:59.873957 | controller | - GitServer: OK 2026-06-09 20:22:59.873960 | controller | - HoundSearch: OK 2026-06-09 20:22:59.873965 | controller | - LogJuicer: OK 2026-06-09 20:22:59.873969 | controller | - Logserver: OK 2026-06-09 20:22:59.873972 | controller | - MariaDB: Waiting ... 2026-06-09 20:22:59.873976 | controller | - Zookeeper: Waiting ... 2026-06-09 20:22:59.873979 | controller | 2026-06-09 20:23:00.880456 | controller | 2026-06-09T20:23:00-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:23:00.909917 | controller | 2026-06-09T20:23:00-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:23:00.928275 | controller | 2026-06-09T20:23:00-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:23:00.934845 | controller | 2026-06-09T20:23:00-04:00 INFO Creating object, name: zuul-db-provision 2026-06-09 20:23:01.483614 | controller | 2026-06-09T20:23:01-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:23:01.483690 | controller | 2026-06-09T20:23:01-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:23:01.486756 | controller | 2026-06-09T20:23:01-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:23:01.496559 | controller | 2026-06-09T20:23:01-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:23:01.496610 | controller | 2026-06-09T20:23:01-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:23:01.506332 | controller | 2026-06-09T20:23:01-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:23:01.518746 | controller | 2026-06-09T20:23:01-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:23:01.518795 | controller | 2026-06-09T20:23:01-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:23:01.886479 | controller | 2026-06-09T20:23:01-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:23:01.886537 | controller | 2026-06-09T20:23:01-04:00 INFO statefulset rollout not ok, name: zookeeper 2026-06-09 20:23:06.894742 | controller | 2026-06-09T20:23:06-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:23:06.894820 | controller | 2026-06-09T20:23:06-04:00 INFO statefulset rollout not ok, name: zookeeper 2026-06-09 20:23:11.902631 | controller | 2026-06-09T20:23:11-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:23:11.902704 | controller | 2026-06-09T20:23:11-04:00 INFO statefulset rollout not ok, name: zookeeper 2026-06-09 20:23:16.912380 | controller | 2026-06-09T20:23:16-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:23:16.912450 | controller | 2026-06-09T20:23:16-04:00 INFO statefulset rollout not ok, name: zookeeper 2026-06-09 20:23:21.920014 | controller | 2026-06-09T20:23:21-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:23:21.923842 | controller | 2026-06-09T20:23:21-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 20:23:21.923911 | controller | 2026-06-09T20:23:21-04:00 INFO 2026-06-09 20:23:21.923923 | controller | - Config: Waiting ... 2026-06-09 20:23:21.923933 | controller | - Gateway: OK 2026-06-09 20:23:21.923942 | controller | - GitServer: OK 2026-06-09 20:23:21.923951 | controller | - HoundSearch: OK 2026-06-09 20:23:21.923960 | controller | - LogJuicer: OK 2026-06-09 20:23:21.923968 | controller | - Logserver: OK 2026-06-09 20:23:21.923977 | controller | - MariaDB: Waiting ... 2026-06-09 20:23:21.923985 | controller | - Zookeeper: OK 2026-06-09 20:23:21.923995 | controller | 2026-06-09 20:23:22.930401 | controller | 2026-06-09T20:23:22-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:23:22.962693 | controller | 2026-06-09T20:23:22-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:23:22.981644 | controller | 2026-06-09T20:23:22-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:23:23.537389 | controller | 2026-06-09T20:23:23-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:23:23.537476 | controller | 2026-06-09T20:23:23-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:23:23.541064 | controller | 2026-06-09T20:23:23-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:23:23.552011 | controller | 2026-06-09T20:23:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:23:23.552057 | controller | 2026-06-09T20:23:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:23:23.561610 | controller | 2026-06-09T20:23:23-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:23:23.574666 | controller | 2026-06-09T20:23:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:23:23.574737 | controller | 2026-06-09T20:23:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:23:23.941260 | controller | 2026-06-09T20:23:23-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:23:23.944495 | controller | 2026-06-09T20:23:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:23:24.351378 | controller | 2026-06-09T20:23:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:23:24.351466 | controller | 2026-06-09T20:23:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:23:24.540823 | controller | 2026-06-09T20:23:24-04:00 INFO Creating object, name: nodepool-builder 2026-06-09 20:23:25.328102 | controller | 2026-06-09T20:23:25-04:00 INFO Updating secret, name: zuul-config 2026-06-09 20:23:25.328175 | controller | 2026-06-09T20:23:25-04:00 INFO Updating object, name: zuul-config 2026-06-09 20:23:25.551962 | controller | 2026-06-09T20:23:25-04:00 INFO Creating object, name: zuul-scheduler 2026-06-09 20:23:25.573599 | controller | 2026-06-09T20:23:25-04:00 INFO Creating object, name: zuul-executor 2026-06-09 20:23:26.333058 | controller | 2026-06-09T20:23:26-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:23:27.132880 | controller | 2026-06-09T20:23:27-04:00 INFO Creating object, name: zuul-merger 2026-06-09 20:23:27.137768 | controller | 2026-06-09T20:23:27-04:00 INFO 2026-06-09 20:23:27.137792 | controller | - Config: Waiting ... 2026-06-09 20:23:27.137799 | controller | - Gateway: OK 2026-06-09 20:23:27.137805 | controller | - GitServer: OK 2026-06-09 20:23:27.137810 | controller | - HoundSearch: OK 2026-06-09 20:23:27.137816 | controller | - LogJuicer: OK 2026-06-09 20:23:27.137828 | controller | - Logserver: OK 2026-06-09 20:23:27.137833 | controller | - MariaDB: OK 2026-06-09 20:23:27.137839 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:23:27.137844 | controller | - NodePoolLauncher: OK 2026-06-09 20:23:27.137849 | controller | - Zookeeper: OK 2026-06-09 20:23:27.137855 | controller | - Zuul: Waiting ... 2026-06-09 20:23:27.137869 | controller | - |_ Config Secrets: OK 2026-06-09 20:23:27.137874 | controller | - |_ Executor: Waiting ... 2026-06-09 20:23:27.137880 | controller | - |_ Merger: Waiting ... 2026-06-09 20:23:27.137885 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:23:27.137890 | controller | - |_ Web: Waiting ... 2026-06-09 20:23:27.137904 | controller | 2026-06-09 20:23:28.143205 | controller | 2026-06-09T20:23:28-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:23:28.166185 | controller | 2026-06-09T20:23:28-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:23:28.186287 | controller | 2026-06-09T20:23:28-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:23:28.745723 | controller | 2026-06-09T20:23:28-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:23:28.745798 | controller | 2026-06-09T20:23:28-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:23:28.747337 | controller | 2026-06-09T20:23:28-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:23:28.753036 | controller | 2026-06-09T20:23:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:23:28.753060 | controller | 2026-06-09T20:23:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:23:28.758027 | controller | 2026-06-09T20:23:28-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:23:28.768304 | controller | 2026-06-09T20:23:28-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:23:28.768352 | controller | 2026-06-09T20:23:28-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:23:29.151888 | controller | 2026-06-09T20:23:29-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:23:29.154103 | controller | 2026-06-09T20:23:29-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:23:29.565042 | controller | 2026-06-09T20:23:29-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:23:29.565112 | controller | 2026-06-09T20:23:29-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:23:30.136216 | controller | 2026-06-09T20:23:30-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:23:30.136322 | controller | 2026-06-09T20:23:30-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:23:30.937400 | controller | 2026-06-09T20:23:30-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:23:30.937477 | controller | 2026-06-09T20:23:30-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:23:35.946761 | controller | 2026-06-09T20:23:35-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:23:35.946845 | controller | 2026-06-09T20:23:35-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:23:40.956844 | controller | 2026-06-09T20:23:40-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:23:40.956919 | controller | 2026-06-09T20:23:40-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:23:45.967924 | controller | 2026-06-09T20:23:45-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:23:45.968003 | controller | 2026-06-09T20:23:45-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:23:50.979433 | controller | 2026-06-09T20:23:50-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:23:50.979500 | controller | 2026-06-09T20:23:50-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:23:55.990482 | controller | 2026-06-09T20:23:55-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:23:55.990549 | controller | 2026-06-09T20:23:55-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:24:00.997154 | controller | 2026-06-09T20:24:00-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:24:01.019873 | controller | 2026-06-09T20:24:01-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:24:01.037800 | controller | 2026-06-09T20:24:01-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:24:01.037876 | controller | 2026-06-09T20:24:01-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:24:01.411333 | controller | 2026-06-09T20:24:01-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:24:01.414818 | controller | 2026-06-09T20:24:01-04:00 INFO 2026-06-09 20:24:01.414858 | controller | - Config: Waiting ... 2026-06-09 20:24:01.414869 | controller | - Gateway: OK 2026-06-09 20:24:01.414878 | controller | - GitServer: OK 2026-06-09 20:24:01.414886 | controller | - HoundSearch: OK 2026-06-09 20:24:01.414937 | controller | - LogJuicer: OK 2026-06-09 20:24:01.414969 | controller | - Logserver: OK 2026-06-09 20:24:01.414981 | controller | - MariaDB: OK 2026-06-09 20:24:01.415006 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:24:01.415018 | controller | - NodePoolLauncher: OK 2026-06-09 20:24:01.415029 | controller | - Zookeeper: OK 2026-06-09 20:24:01.415039 | controller | - Zuul: Waiting ... 2026-06-09 20:24:01.415047 | controller | - |_ Config Secrets: OK 2026-06-09 20:24:01.415055 | controller | - |_ Executor: OK 2026-06-09 20:24:01.415062 | controller | - |_ Merger: OK 2026-06-09 20:24:01.415070 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:24:01.415078 | controller | - |_ Web: Waiting ... 2026-06-09 20:24:01.415086 | controller | 2026-06-09 20:24:02.419570 | controller | 2026-06-09T20:24:02-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:24:02.446529 | controller | 2026-06-09T20:24:02-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:24:02.464536 | controller | 2026-06-09T20:24:02-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:24:03.024408 | controller | 2026-06-09T20:24:03-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:24:03.024464 | controller | 2026-06-09T20:24:03-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:24:03.026889 | controller | 2026-06-09T20:24:03-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:24:03.033758 | controller | 2026-06-09T20:24:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:24:03.033788 | controller | 2026-06-09T20:24:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:24:03.038779 | controller | 2026-06-09T20:24:03-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:24:03.048341 | controller | 2026-06-09T20:24:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:24:03.048393 | controller | 2026-06-09T20:24:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:24:03.432959 | controller | 2026-06-09T20:24:03-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:24:03.437097 | controller | 2026-06-09T20:24:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:24:03.838910 | controller | 2026-06-09T20:24:03-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:24:03.838994 | controller | 2026-06-09T20:24:03-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:24:04.411040 | controller | 2026-06-09T20:24:04-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:24:04.411113 | controller | 2026-06-09T20:24:04-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:24:05.210522 | controller | 2026-06-09T20:24:05-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:05.210585 | controller | 2026-06-09T20:24:05-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:24:10.224542 | controller | 2026-06-09T20:24:10-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:10.224645 | controller | 2026-06-09T20:24:10-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:24:15.235639 | controller | 2026-06-09T20:24:15-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:15.235729 | controller | 2026-06-09T20:24:15-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:24:20.246922 | controller | 2026-06-09T20:24:20-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:20.247004 | controller | 2026-06-09T20:24:20-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:24:25.257093 | controller | 2026-06-09T20:24:25-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:25.257177 | controller | 2026-06-09T20:24:25-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:24:30.267340 | controller | 2026-06-09T20:24:30-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:30.276694 | controller | 2026-06-09T20:24:30-04:00 INFO Creating object, name: zuul-weeder 2026-06-09 20:24:30.301197 | controller | 2026-06-09T20:24:30-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:24:30.318004 | controller | 2026-06-09T20:24:30-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:24:30.318067 | controller | 2026-06-09T20:24:30-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:24:30.693172 | controller | 2026-06-09T20:24:30-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:24:30.696331 | controller | 2026-06-09T20:24:30-04:00 INFO 2026-06-09 20:24:30.696391 | controller | - Config: Waiting ... 2026-06-09 20:24:30.696403 | controller | - Gateway: OK 2026-06-09 20:24:30.696411 | controller | - GitServer: OK 2026-06-09 20:24:30.696420 | controller | - HoundSearch: OK 2026-06-09 20:24:30.696427 | controller | - LogJuicer: OK 2026-06-09 20:24:30.696435 | controller | - Logserver: OK 2026-06-09 20:24:30.696443 | controller | - MariaDB: OK 2026-06-09 20:24:30.696451 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:24:30.696459 | controller | - NodePoolLauncher: OK 2026-06-09 20:24:30.696467 | controller | - Zookeeper: OK 2026-06-09 20:24:30.696475 | controller | - Zuul: Waiting ... 2026-06-09 20:24:30.696510 | controller | - |_ Config Secrets: OK 2026-06-09 20:24:30.696520 | controller | - |_ Executor: OK 2026-06-09 20:24:30.696528 | controller | - |_ Merger: OK 2026-06-09 20:24:30.696536 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:24:30.696543 | controller | - |_ Web: OK 2026-06-09 20:24:30.696551 | controller | 2026-06-09 20:24:31.701855 | controller | 2026-06-09T20:24:31-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:24:31.717459 | controller | 2026-06-09T20:24:31-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:24:31.727690 | controller | 2026-06-09T20:24:31-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:24:32.307614 | controller | 2026-06-09T20:24:32-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:24:32.307688 | controller | 2026-06-09T20:24:32-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:24:32.311303 | controller | 2026-06-09T20:24:32-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:24:32.322365 | controller | 2026-06-09T20:24:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:24:32.322414 | controller | 2026-06-09T20:24:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:24:32.331653 | controller | 2026-06-09T20:24:32-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:24:32.342854 | controller | 2026-06-09T20:24:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:24:32.342913 | controller | 2026-06-09T20:24:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:24:32.717645 | controller | 2026-06-09T20:24:32-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:24:32.721119 | controller | 2026-06-09T20:24:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:24:33.124428 | controller | 2026-06-09T20:24:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:24:33.124512 | controller | 2026-06-09T20:24:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:24:33.692469 | controller | 2026-06-09T20:24:33-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:24:34.489119 | controller | 2026-06-09T20:24:34-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:34.496985 | controller | 2026-06-09T20:24:34-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:24:34.497047 | controller | 2026-06-09T20:24:34-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:24:35.295466 | controller | 2026-06-09T20:24:35-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:24:36.087732 | controller | 2026-06-09T20:24:36-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:24:36.087839 | controller | 2026-06-09T20:24:36-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:24:36.891320 | controller | 2026-06-09T20:24:36-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:24:37.290910 | controller | 2026-06-09T20:24:37-04:00 INFO Creating base secret job 2026-06-09 20:24:37.291080 | controller | 2026-06-09T20:24:37-04:00 INFO Creating object, name: config-base-secret-1-2801-ns 2026-06-09 20:24:37.293726 | controller | 2026-06-09T20:24:37-04:00 INFO 2026-06-09 20:24:37.293758 | controller | - Config: Waiting ... 2026-06-09 20:24:37.293763 | controller | - Gateway: OK 2026-06-09 20:24:37.293767 | controller | - GitServer: OK 2026-06-09 20:24:37.293771 | controller | - HoundSearch: OK 2026-06-09 20:24:37.293774 | controller | - LogJuicer: OK 2026-06-09 20:24:37.293777 | controller | - Logserver: OK 2026-06-09 20:24:37.293781 | controller | - MariaDB: OK 2026-06-09 20:24:37.293784 | controller | - NodePoolBuilder: OK 2026-06-09 20:24:37.293788 | controller | - NodePoolLauncher: OK 2026-06-09 20:24:37.293791 | controller | - Zookeeper: OK 2026-06-09 20:24:37.293803 | controller | - Zuul: OK 2026-06-09 20:24:37.293807 | controller | 2026-06-09 20:24:38.300995 | controller | 2026-06-09T20:24:38-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:24:38.329020 | controller | 2026-06-09T20:24:38-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:24:38.345415 | controller | 2026-06-09T20:24:38-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:24:38.906478 | controller | 2026-06-09T20:24:38-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:24:38.906589 | controller | 2026-06-09T20:24:38-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:24:38.909405 | controller | 2026-06-09T20:24:38-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:24:38.920420 | controller | 2026-06-09T20:24:38-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:24:38.920487 | controller | 2026-06-09T20:24:38-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:24:38.930856 | controller | 2026-06-09T20:24:38-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:24:38.944404 | controller | 2026-06-09T20:24:38-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:24:38.944455 | controller | 2026-06-09T20:24:38-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:24:39.310045 | controller | 2026-06-09T20:24:39-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:24:39.312395 | controller | 2026-06-09T20:24:39-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:24:39.721771 | controller | 2026-06-09T20:24:39-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:24:39.721843 | controller | 2026-06-09T20:24:39-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:24:40.293953 | controller | 2026-06-09T20:24:40-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:24:41.090200 | controller | 2026-06-09T20:24:41-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:41.100436 | controller | 2026-06-09T20:24:41-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:24:41.100483 | controller | 2026-06-09T20:24:41-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:24:41.892157 | controller | 2026-06-09T20:24:41-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:24:42.687906 | controller | 2026-06-09T20:24:42-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:24:42.687980 | controller | 2026-06-09T20:24:42-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:24:43.495853 | controller | 2026-06-09T20:24:43-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:24:43.892145 | controller | 2026-06-09T20:24:43-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-09 20:24:44.083403 | controller | 2026-06-09T20:24:44-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-09 20:24:50.269012 | controller | 2026-06-09T20:24:50-04:00 INFO Deleting object, name: zuul-web 2026-06-09 20:24:50.273887 | controller | 2026-06-09T20:24:50-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-09 20:24:50.276692 | controller | 2026-06-09T20:24:50-04:00 INFO 2026-06-09 20:24:50.276737 | controller | - Config: Waiting ... 2026-06-09 20:24:50.276752 | controller | - Gateway: OK 2026-06-09 20:24:50.276764 | controller | - GitServer: OK 2026-06-09 20:24:50.276775 | controller | - HoundSearch: OK 2026-06-09 20:24:50.276785 | controller | - LogJuicer: OK 2026-06-09 20:24:50.276796 | controller | - Logserver: OK 2026-06-09 20:24:50.276807 | controller | - MariaDB: OK 2026-06-09 20:24:50.276818 | controller | - NodePoolBuilder: OK 2026-06-09 20:24:50.276829 | controller | - NodePoolLauncher: OK 2026-06-09 20:24:50.276840 | controller | - Zookeeper: OK 2026-06-09 20:24:50.276852 | controller | - Zuul: OK 2026-06-09 20:24:50.276880 | controller | 2026-06-09 20:24:51.282225 | controller | 2026-06-09T20:24:51-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:24:51.302557 | controller | 2026-06-09T20:24:51-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:24:51.314487 | controller | 2026-06-09T20:24:51-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:24:51.887613 | controller | 2026-06-09T20:24:51-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:24:51.887680 | controller | 2026-06-09T20:24:51-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:24:51.889316 | controller | 2026-06-09T20:24:51-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:24:51.896672 | controller | 2026-06-09T20:24:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:24:51.896732 | controller | 2026-06-09T20:24:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:24:51.902381 | controller | 2026-06-09T20:24:51-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:24:51.910175 | controller | 2026-06-09T20:24:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:24:51.910250 | controller | 2026-06-09T20:24:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:24:52.290316 | controller | 2026-06-09T20:24:52-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:24:52.292287 | controller | 2026-06-09T20:24:52-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:24:52.695053 | controller | 2026-06-09T20:24:52-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:24:52.695114 | controller | 2026-06-09T20:24:52-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:24:52.901009 | controller | 2026-06-09T20:24:52-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:24:53.701418 | controller | 2026-06-09T20:24:53-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:53.707528 | controller | 2026-06-09T20:24:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:24:53.707593 | controller | 2026-06-09T20:24:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:24:53.892682 | controller | 2026-06-09T20:24:53-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:24:54.687672 | controller | 2026-06-09T20:24:54-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:24:55.492476 | controller | 2026-06-09T20:24:55-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:24:55.494726 | controller | 2026-06-09T20:24:55-04:00 INFO 2026-06-09 20:24:55.494766 | controller | - Config: Waiting ... 2026-06-09 20:24:55.494777 | controller | - Gateway: OK 2026-06-09 20:24:55.494790 | controller | - GitServer: OK 2026-06-09 20:24:55.494802 | controller | - HoundSearch: OK 2026-06-09 20:24:55.494814 | controller | - LogJuicer: OK 2026-06-09 20:24:55.494826 | controller | - Logserver: OK 2026-06-09 20:24:55.494838 | controller | - MariaDB: OK 2026-06-09 20:24:55.494850 | controller | - NodePoolBuilder: OK 2026-06-09 20:24:55.494862 | controller | - NodePoolLauncher: OK 2026-06-09 20:24:55.494872 | controller | - Zookeeper: OK 2026-06-09 20:24:55.494880 | controller | - Zuul: Waiting ... 2026-06-09 20:24:55.494888 | controller | - |_ Config Secrets: OK 2026-06-09 20:24:55.494897 | controller | - |_ Executor: OK 2026-06-09 20:24:55.494905 | controller | - |_ Merger: OK 2026-06-09 20:24:55.494913 | controller | - |_ Scheduler: OK 2026-06-09 20:24:55.494921 | controller | - |_ Web: Waiting ... 2026-06-09 20:24:55.494930 | controller | 2026-06-09 20:24:56.500387 | controller | 2026-06-09T20:24:56-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:24:56.519726 | controller | 2026-06-09T20:24:56-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:24:56.533168 | controller | 2026-06-09T20:24:56-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:24:57.104215 | controller | 2026-06-09T20:24:57-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:24:57.104546 | controller | 2026-06-09T20:24:57-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:24:57.107478 | controller | 2026-06-09T20:24:57-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:24:57.117753 | controller | 2026-06-09T20:24:57-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:24:57.117784 | controller | 2026-06-09T20:24:57-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:24:57.127994 | controller | 2026-06-09T20:24:57-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:24:57.138184 | controller | 2026-06-09T20:24:57-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:24:57.138224 | controller | 2026-06-09T20:24:57-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:24:57.513515 | controller | 2026-06-09T20:24:57-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:24:57.516099 | controller | 2026-06-09T20:24:57-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:24:57.916224 | controller | 2026-06-09T20:24:57-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:24:57.916294 | controller | 2026-06-09T20:24:57-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:24:58.493537 | controller | 2026-06-09T20:24:58-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:24:59.286647 | controller | 2026-06-09T20:24:59-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:24:59.291772 | controller | 2026-06-09T20:24:59-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:24:59.291810 | controller | 2026-06-09T20:24:59-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:00.091780 | controller | 2026-06-09T20:25:00-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:00.886896 | controller | 2026-06-09T20:25:00-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:00.886984 | controller | 2026-06-09T20:25:00-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:01.692206 | controller | 2026-06-09T20:25:01-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:01.695013 | controller | 2026-06-09T20:25:01-04:00 INFO 2026-06-09 20:25:01.695051 | controller | - Config: Waiting ... 2026-06-09 20:25:01.695062 | controller | - Gateway: OK 2026-06-09 20:25:01.695070 | controller | - GitServer: OK 2026-06-09 20:25:01.695078 | controller | - HoundSearch: OK 2026-06-09 20:25:01.695086 | controller | - LogJuicer: OK 2026-06-09 20:25:01.695094 | controller | - Logserver: OK 2026-06-09 20:25:01.695102 | controller | - MariaDB: OK 2026-06-09 20:25:01.695115 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:01.695123 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:01.695131 | controller | - Zookeeper: OK 2026-06-09 20:25:01.695139 | controller | - Zuul: Waiting ... 2026-06-09 20:25:01.695146 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:01.695154 | controller | - |_ Executor: OK 2026-06-09 20:25:01.695162 | controller | - |_ Merger: OK 2026-06-09 20:25:01.695169 | controller | - |_ Scheduler: OK 2026-06-09 20:25:01.695177 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:01.695185 | controller | 2026-06-09 20:25:02.701889 | controller | 2026-06-09T20:25:02-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:02.728660 | controller | 2026-06-09T20:25:02-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:02.742254 | controller | 2026-06-09T20:25:02-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:03.304635 | controller | 2026-06-09T20:25:03-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:03.304692 | controller | 2026-06-09T20:25:03-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:03.306847 | controller | 2026-06-09T20:25:03-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:03.313511 | controller | 2026-06-09T20:25:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:03.313528 | controller | 2026-06-09T20:25:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:03.321306 | controller | 2026-06-09T20:25:03-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:03.329882 | controller | 2026-06-09T20:25:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:03.329902 | controller | 2026-06-09T20:25:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:03.712552 | controller | 2026-06-09T20:25:03-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:03.715469 | controller | 2026-06-09T20:25:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:04.121374 | controller | 2026-06-09T20:25:04-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:04.121436 | controller | 2026-06-09T20:25:04-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:04.692401 | controller | 2026-06-09T20:25:04-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:05.491697 | controller | 2026-06-09T20:25:05-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:05.502469 | controller | 2026-06-09T20:25:05-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:05.502531 | controller | 2026-06-09T20:25:05-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:06.292641 | controller | 2026-06-09T20:25:06-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:07.087636 | controller | 2026-06-09T20:25:07-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:07.087699 | controller | 2026-06-09T20:25:07-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:07.890686 | controller | 2026-06-09T20:25:07-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:07.893673 | controller | 2026-06-09T20:25:07-04:00 INFO 2026-06-09 20:25:07.893714 | controller | - Config: Waiting ... 2026-06-09 20:25:07.893729 | controller | - Gateway: OK 2026-06-09 20:25:07.893738 | controller | - GitServer: OK 2026-06-09 20:25:07.893746 | controller | - HoundSearch: OK 2026-06-09 20:25:07.893757 | controller | - LogJuicer: OK 2026-06-09 20:25:07.893767 | controller | - Logserver: OK 2026-06-09 20:25:07.893780 | controller | - MariaDB: OK 2026-06-09 20:25:07.893818 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:07.893830 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:07.893863 | controller | - Zookeeper: OK 2026-06-09 20:25:07.893889 | controller | - Zuul: Waiting ... 2026-06-09 20:25:07.893925 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:07.893936 | controller | - |_ Executor: OK 2026-06-09 20:25:07.893979 | controller | - |_ Merger: OK 2026-06-09 20:25:07.894020 | controller | - |_ Scheduler: OK 2026-06-09 20:25:07.894030 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:07.894038 | controller | 2026-06-09 20:25:08.899922 | controller | 2026-06-09T20:25:08-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:08.926161 | controller | 2026-06-09T20:25:08-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:08.942121 | controller | 2026-06-09T20:25:08-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:09.506684 | controller | 2026-06-09T20:25:09-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:09.506760 | controller | 2026-06-09T20:25:09-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:09.508508 | controller | 2026-06-09T20:25:09-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:09.515940 | controller | 2026-06-09T20:25:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:09.515976 | controller | 2026-06-09T20:25:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:09.521825 | controller | 2026-06-09T20:25:09-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:09.529662 | controller | 2026-06-09T20:25:09-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:09.529764 | controller | 2026-06-09T20:25:09-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:09.909559 | controller | 2026-06-09T20:25:09-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:09.912015 | controller | 2026-06-09T20:25:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:10.314024 | controller | 2026-06-09T20:25:10-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:10.314099 | controller | 2026-06-09T20:25:10-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:10.892673 | controller | 2026-06-09T20:25:10-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:11.688998 | controller | 2026-06-09T20:25:11-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:11.695713 | controller | 2026-06-09T20:25:11-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:11.695754 | controller | 2026-06-09T20:25:11-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:12.492668 | controller | 2026-06-09T20:25:12-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:13.287681 | controller | 2026-06-09T20:25:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:13.287812 | controller | 2026-06-09T20:25:13-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:14.091580 | controller | 2026-06-09T20:25:14-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:14.094566 | controller | 2026-06-09T20:25:14-04:00 INFO 2026-06-09 20:25:14.094612 | controller | - Config: Waiting ... 2026-06-09 20:25:14.094705 | controller | - Gateway: OK 2026-06-09 20:25:14.094753 | controller | - GitServer: OK 2026-06-09 20:25:14.094804 | controller | - HoundSearch: OK 2026-06-09 20:25:14.094862 | controller | - LogJuicer: OK 2026-06-09 20:25:14.094936 | controller | - Logserver: OK 2026-06-09 20:25:14.094993 | controller | - MariaDB: OK 2026-06-09 20:25:14.095058 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:14.095117 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:14.095184 | controller | - Zookeeper: OK 2026-06-09 20:25:14.095283 | controller | - Zuul: Waiting ... 2026-06-09 20:25:14.095344 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:14.095413 | controller | - |_ Executor: OK 2026-06-09 20:25:14.095469 | controller | - |_ Merger: OK 2026-06-09 20:25:14.095538 | controller | - |_ Scheduler: OK 2026-06-09 20:25:14.095602 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:14.095614 | controller | 2026-06-09 20:25:15.100530 | controller | 2026-06-09T20:25:15-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:15.119748 | controller | 2026-06-09T20:25:15-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:15.137049 | controller | 2026-06-09T20:25:15-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:15.706606 | controller | 2026-06-09T20:25:15-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:15.706686 | controller | 2026-06-09T20:25:15-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:15.709338 | controller | 2026-06-09T20:25:15-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:15.719140 | controller | 2026-06-09T20:25:15-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:15.719188 | controller | 2026-06-09T20:25:15-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:15.726618 | controller | 2026-06-09T20:25:15-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:15.737141 | controller | 2026-06-09T20:25:15-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:15.737187 | controller | 2026-06-09T20:25:15-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:16.115356 | controller | 2026-06-09T20:25:16-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:16.117536 | controller | 2026-06-09T20:25:16-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:16.519008 | controller | 2026-06-09T20:25:16-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:16.519081 | controller | 2026-06-09T20:25:16-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:17.093129 | controller | 2026-06-09T20:25:17-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:17.889823 | controller | 2026-06-09T20:25:17-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:17.898906 | controller | 2026-06-09T20:25:17-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:17.898949 | controller | 2026-06-09T20:25:17-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:18.698756 | controller | 2026-06-09T20:25:18-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:19.488474 | controller | 2026-06-09T20:25:19-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:19.488538 | controller | 2026-06-09T20:25:19-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:20.291424 | controller | 2026-06-09T20:25:20-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:20.294011 | controller | 2026-06-09T20:25:20-04:00 INFO 2026-06-09 20:25:20.294050 | controller | - Config: Waiting ... 2026-06-09 20:25:20.294060 | controller | - Gateway: OK 2026-06-09 20:25:20.294069 | controller | - GitServer: OK 2026-06-09 20:25:20.294077 | controller | - HoundSearch: OK 2026-06-09 20:25:20.294084 | controller | - LogJuicer: OK 2026-06-09 20:25:20.294092 | controller | - Logserver: OK 2026-06-09 20:25:20.294099 | controller | - MariaDB: OK 2026-06-09 20:25:20.294107 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:20.294114 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:20.294122 | controller | - Zookeeper: OK 2026-06-09 20:25:20.294129 | controller | - Zuul: Waiting ... 2026-06-09 20:25:20.294136 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:20.294144 | controller | - |_ Executor: OK 2026-06-09 20:25:20.294151 | controller | - |_ Merger: OK 2026-06-09 20:25:20.294159 | controller | - |_ Scheduler: OK 2026-06-09 20:25:20.294166 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:20.294174 | controller | 2026-06-09 20:25:21.299931 | controller | 2026-06-09T20:25:21-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:21.327328 | controller | 2026-06-09T20:25:21-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:21.345151 | controller | 2026-06-09T20:25:21-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:21.901759 | controller | 2026-06-09T20:25:21-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:21.901817 | controller | 2026-06-09T20:25:21-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:21.903732 | controller | 2026-06-09T20:25:21-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:21.912912 | controller | 2026-06-09T20:25:21-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:21.912936 | controller | 2026-06-09T20:25:21-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:21.921243 | controller | 2026-06-09T20:25:21-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:21.932110 | controller | 2026-06-09T20:25:21-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:21.932220 | controller | 2026-06-09T20:25:21-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:22.304997 | controller | 2026-06-09T20:25:22-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:22.306428 | controller | 2026-06-09T20:25:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:22.716106 | controller | 2026-06-09T20:25:22-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:22.716170 | controller | 2026-06-09T20:25:22-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:23.294033 | controller | 2026-06-09T20:25:23-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:24.091757 | controller | 2026-06-09T20:25:24-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:24.100393 | controller | 2026-06-09T20:25:24-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:24.100479 | controller | 2026-06-09T20:25:24-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:24.892128 | controller | 2026-06-09T20:25:24-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:25.688125 | controller | 2026-06-09T20:25:25-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:25.688198 | controller | 2026-06-09T20:25:25-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:26.496902 | controller | 2026-06-09T20:25:26-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:26.500573 | controller | 2026-06-09T20:25:26-04:00 INFO 2026-06-09 20:25:26.500660 | controller | - Config: Waiting ... 2026-06-09 20:25:26.500680 | controller | - Gateway: OK 2026-06-09 20:25:26.500735 | controller | - GitServer: OK 2026-06-09 20:25:26.500794 | controller | - HoundSearch: OK 2026-06-09 20:25:26.500862 | controller | - LogJuicer: OK 2026-06-09 20:25:26.500881 | controller | - Logserver: OK 2026-06-09 20:25:26.500935 | controller | - MariaDB: OK 2026-06-09 20:25:26.501017 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:26.501034 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:26.501109 | controller | - Zookeeper: OK 2026-06-09 20:25:26.501173 | controller | - Zuul: Waiting ... 2026-06-09 20:25:26.501222 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:26.501320 | controller | - |_ Executor: OK 2026-06-09 20:25:26.501345 | controller | - |_ Merger: OK 2026-06-09 20:25:26.501417 | controller | - |_ Scheduler: OK 2026-06-09 20:25:26.501496 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:26.501513 | controller | 2026-06-09 20:25:27.508005 | controller | 2026-06-09T20:25:27-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:27.533465 | controller | 2026-06-09T20:25:27-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:27.555523 | controller | 2026-06-09T20:25:27-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:28.114297 | controller | 2026-06-09T20:25:28-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:28.114385 | controller | 2026-06-09T20:25:28-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:28.117546 | controller | 2026-06-09T20:25:28-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:28.129186 | controller | 2026-06-09T20:25:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:28.129257 | controller | 2026-06-09T20:25:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:28.138800 | controller | 2026-06-09T20:25:28-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:28.152187 | controller | 2026-06-09T20:25:28-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:28.152288 | controller | 2026-06-09T20:25:28-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:28.519927 | controller | 2026-06-09T20:25:28-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:28.522032 | controller | 2026-06-09T20:25:28-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:28.927043 | controller | 2026-06-09T20:25:28-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:28.927130 | controller | 2026-06-09T20:25:28-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:29.490482 | controller | 2026-06-09T20:25:29-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:30.290893 | controller | 2026-06-09T20:25:30-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:30.299889 | controller | 2026-06-09T20:25:30-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:30.299950 | controller | 2026-06-09T20:25:30-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:31.096012 | controller | 2026-06-09T20:25:31-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:31.888163 | controller | 2026-06-09T20:25:31-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:31.888253 | controller | 2026-06-09T20:25:31-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:32.694595 | controller | 2026-06-09T20:25:32-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:32.698196 | controller | 2026-06-09T20:25:32-04:00 INFO 2026-06-09 20:25:32.698269 | controller | - Config: Waiting ... 2026-06-09 20:25:32.698289 | controller | - Gateway: OK 2026-06-09 20:25:32.698298 | controller | - GitServer: OK 2026-06-09 20:25:32.698306 | controller | - HoundSearch: OK 2026-06-09 20:25:32.698314 | controller | - LogJuicer: OK 2026-06-09 20:25:32.698321 | controller | - Logserver: OK 2026-06-09 20:25:32.698329 | controller | - MariaDB: OK 2026-06-09 20:25:32.698338 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:32.698348 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:32.698359 | controller | - Zookeeper: OK 2026-06-09 20:25:32.698369 | controller | - Zuul: Waiting ... 2026-06-09 20:25:32.698380 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:32.698390 | controller | - |_ Executor: OK 2026-06-09 20:25:32.698401 | controller | - |_ Merger: OK 2026-06-09 20:25:32.698411 | controller | - |_ Scheduler: OK 2026-06-09 20:25:32.698421 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:32.698431 | controller | 2026-06-09 20:25:33.704082 | controller | 2026-06-09T20:25:33-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:33.730132 | controller | 2026-06-09T20:25:33-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:33.744947 | controller | 2026-06-09T20:25:33-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:34.308211 | controller | 2026-06-09T20:25:34-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:34.308331 | controller | 2026-06-09T20:25:34-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:34.310374 | controller | 2026-06-09T20:25:34-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:34.320477 | controller | 2026-06-09T20:25:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:34.320533 | controller | 2026-06-09T20:25:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:34.330550 | controller | 2026-06-09T20:25:34-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:34.349639 | controller | 2026-06-09T20:25:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:34.349711 | controller | 2026-06-09T20:25:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:34.714959 | controller | 2026-06-09T20:25:34-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:34.717224 | controller | 2026-06-09T20:25:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:35.122477 | controller | 2026-06-09T20:25:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:35.122565 | controller | 2026-06-09T20:25:35-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:35.692176 | controller | 2026-06-09T20:25:35-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:36.489743 | controller | 2026-06-09T20:25:36-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:36.502649 | controller | 2026-06-09T20:25:36-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:36.502728 | controller | 2026-06-09T20:25:36-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:37.293610 | controller | 2026-06-09T20:25:37-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:38.087651 | controller | 2026-06-09T20:25:38-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:38.087744 | controller | 2026-06-09T20:25:38-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:38.893869 | controller | 2026-06-09T20:25:38-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:38.897179 | controller | 2026-06-09T20:25:38-04:00 INFO 2026-06-09 20:25:38.897267 | controller | - Config: Waiting ... 2026-06-09 20:25:38.897283 | controller | - Gateway: OK 2026-06-09 20:25:38.897293 | controller | - GitServer: OK 2026-06-09 20:25:38.897301 | controller | - HoundSearch: OK 2026-06-09 20:25:38.897310 | controller | - LogJuicer: OK 2026-06-09 20:25:38.897318 | controller | - Logserver: OK 2026-06-09 20:25:38.897326 | controller | - MariaDB: OK 2026-06-09 20:25:38.897334 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:38.897343 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:38.897351 | controller | - Zookeeper: OK 2026-06-09 20:25:38.897359 | controller | - Zuul: Waiting ... 2026-06-09 20:25:38.897513 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:38.897522 | controller | - |_ Executor: OK 2026-06-09 20:25:38.897530 | controller | - |_ Merger: OK 2026-06-09 20:25:38.897538 | controller | - |_ Scheduler: OK 2026-06-09 20:25:38.897546 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:38.897555 | controller | 2026-06-09 20:25:39.903658 | controller | 2026-06-09T20:25:39-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:39.927533 | controller | 2026-06-09T20:25:39-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:39.947197 | controller | 2026-06-09T20:25:39-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:40.508809 | controller | 2026-06-09T20:25:40-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:40.508879 | controller | 2026-06-09T20:25:40-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:40.511659 | controller | 2026-06-09T20:25:40-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:40.524201 | controller | 2026-06-09T20:25:40-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:40.524276 | controller | 2026-06-09T20:25:40-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:40.534427 | controller | 2026-06-09T20:25:40-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:40.549975 | controller | 2026-06-09T20:25:40-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:40.550024 | controller | 2026-06-09T20:25:40-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:40.918412 | controller | 2026-06-09T20:25:40-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:40.921471 | controller | 2026-06-09T20:25:40-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:41.323768 | controller | 2026-06-09T20:25:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:41.323845 | controller | 2026-06-09T20:25:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:41.889401 | controller | 2026-06-09T20:25:41-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:42.694440 | controller | 2026-06-09T20:25:42-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:42.702465 | controller | 2026-06-09T20:25:42-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:42.702525 | controller | 2026-06-09T20:25:42-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:43.492682 | controller | 2026-06-09T20:25:43-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:44.290104 | controller | 2026-06-09T20:25:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:44.290188 | controller | 2026-06-09T20:25:44-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:45.094379 | controller | 2026-06-09T20:25:45-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:45.098023 | controller | 2026-06-09T20:25:45-04:00 INFO 2026-06-09 20:25:45.098070 | controller | - Config: Waiting ... 2026-06-09 20:25:45.098086 | controller | - Gateway: OK 2026-06-09 20:25:45.098099 | controller | - GitServer: OK 2026-06-09 20:25:45.098108 | controller | - HoundSearch: OK 2026-06-09 20:25:45.098116 | controller | - LogJuicer: OK 2026-06-09 20:25:45.098124 | controller | - Logserver: OK 2026-06-09 20:25:45.098132 | controller | - MariaDB: OK 2026-06-09 20:25:45.098141 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:45.098149 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:45.098157 | controller | - Zookeeper: OK 2026-06-09 20:25:45.098165 | controller | - Zuul: Waiting ... 2026-06-09 20:25:45.098174 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:45.098182 | controller | - |_ Executor: OK 2026-06-09 20:25:45.098190 | controller | - |_ Merger: OK 2026-06-09 20:25:45.098198 | controller | - |_ Scheduler: OK 2026-06-09 20:25:45.098207 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:45.098215 | controller | 2026-06-09 20:25:46.104524 | controller | 2026-06-09T20:25:46-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:46.131271 | controller | 2026-06-09T20:25:46-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:46.150871 | controller | 2026-06-09T20:25:46-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:46.711771 | controller | 2026-06-09T20:25:46-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:46.711847 | controller | 2026-06-09T20:25:46-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:46.715151 | controller | 2026-06-09T20:25:46-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:46.728178 | controller | 2026-06-09T20:25:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:46.728220 | controller | 2026-06-09T20:25:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:46.738651 | controller | 2026-06-09T20:25:46-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:46.752412 | controller | 2026-06-09T20:25:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:46.752499 | controller | 2026-06-09T20:25:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:47.119874 | controller | 2026-06-09T20:25:47-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:47.122693 | controller | 2026-06-09T20:25:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:47.523290 | controller | 2026-06-09T20:25:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:47.523367 | controller | 2026-06-09T20:25:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:48.091628 | controller | 2026-06-09T20:25:48-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:48.890410 | controller | 2026-06-09T20:25:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:48.898632 | controller | 2026-06-09T20:25:48-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:48.898679 | controller | 2026-06-09T20:25:48-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:49.693142 | controller | 2026-06-09T20:25:49-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:50.485606 | controller | 2026-06-09T20:25:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:50.485661 | controller | 2026-06-09T20:25:50-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:25:51.291775 | controller | 2026-06-09T20:25:51-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:51.294816 | controller | 2026-06-09T20:25:51-04:00 INFO 2026-06-09 20:25:51.294894 | controller | - Config: Waiting ... 2026-06-09 20:25:51.294905 | controller | - Gateway: OK 2026-06-09 20:25:51.294914 | controller | - GitServer: OK 2026-06-09 20:25:51.294922 | controller | - HoundSearch: OK 2026-06-09 20:25:51.294930 | controller | - LogJuicer: OK 2026-06-09 20:25:51.294938 | controller | - Logserver: OK 2026-06-09 20:25:51.294946 | controller | - MariaDB: OK 2026-06-09 20:25:51.294953 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:51.294961 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:51.294969 | controller | - Zookeeper: OK 2026-06-09 20:25:51.294988 | controller | - Zuul: Waiting ... 2026-06-09 20:25:51.294996 | controller | - |_ Config Secrets: OK 2026-06-09 20:25:51.295004 | controller | - |_ Executor: OK 2026-06-09 20:25:51.295011 | controller | - |_ Merger: OK 2026-06-09 20:25:51.295019 | controller | - |_ Scheduler: OK 2026-06-09 20:25:51.295027 | controller | - |_ Web: Waiting ... 2026-06-09 20:25:51.295035 | controller | 2026-06-09 20:25:52.299988 | controller | 2026-06-09T20:25:52-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:52.319955 | controller | 2026-06-09T20:25:52-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:52.329950 | controller | 2026-06-09T20:25:52-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:25:52.906167 | controller | 2026-06-09T20:25:52-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:25:52.906253 | controller | 2026-06-09T20:25:52-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:25:52.908992 | controller | 2026-06-09T20:25:52-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:25:52.919124 | controller | 2026-06-09T20:25:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:25:52.919180 | controller | 2026-06-09T20:25:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:25:52.929301 | controller | 2026-06-09T20:25:52-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:25:52.940785 | controller | 2026-06-09T20:25:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:25:52.940832 | controller | 2026-06-09T20:25:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:25:53.318603 | controller | 2026-06-09T20:25:53-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:25:53.321211 | controller | 2026-06-09T20:25:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:25:53.720826 | controller | 2026-06-09T20:25:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:25:53.720898 | controller | 2026-06-09T20:25:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:25:54.294299 | controller | 2026-06-09T20:25:54-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:25:55.090444 | controller | 2026-06-09T20:25:55-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:25:55.100013 | controller | 2026-06-09T20:25:55-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:25:55.100064 | controller | 2026-06-09T20:25:55-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:25:55.894157 | controller | 2026-06-09T20:25:55-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:25:56.686697 | controller | 2026-06-09T20:25:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:25:56.686775 | controller | 2026-06-09T20:25:56-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:25:57.490397 | controller | 2026-06-09T20:25:57-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:25:57.896190 | controller | 2026-06-09T20:25:57-04:00 INFO 2026-06-09 20:25:57.896297 | controller | - Config: OK 2026-06-09 20:25:57.896309 | controller | - Gateway: OK 2026-06-09 20:25:57.896317 | controller | - GitServer: OK 2026-06-09 20:25:57.896325 | controller | - HoundSearch: OK 2026-06-09 20:25:57.896333 | controller | - LogJuicer: OK 2026-06-09 20:25:57.896341 | controller | - Logserver: OK 2026-06-09 20:25:57.896348 | controller | - MariaDB: OK 2026-06-09 20:25:57.896356 | controller | - NodePoolBuilder: OK 2026-06-09 20:25:57.896364 | controller | - NodePoolLauncher: OK 2026-06-09 20:25:57.896371 | controller | - Zookeeper: OK 2026-06-09 20:25:57.896380 | controller | - Zuul: OK 2026-06-09 20:25:57.896388 | controller | 2026-06-09 20:25:57.896404 | controller | STEP: Validating secret changed @ 06/09/26 20:25:57.896 2026-06-09 20:25:57.907703 | controller | 2026-06-09T20:25:57-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /etc/zuul/zuul.conf] 2026-06-09 20:25:58.064210 | controller | STEP: Checking the secret is in the zuul.conf @ 06/09/26 20:25:58.064 2026-06-09 20:25:58.067016 | controller | STEP: Checking zuul-client works @ 06/09/26 20:25:58.066 2026-06-09 20:25:58.067059 | controller | 2026-06-09T20:25:58-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c zuul-client -v autohold --tenant internal --project system-config --job testytest --reason testy 2>&1] 2026-06-09 20:25:58.893306 | controller | • [275.111 seconds] 2026-06-09 20:25:58.893392 | controller | ------------------------------ 2026-06-09 20:25:58.893457 | controller | Secret Rotations zuul-db works 2026-06-09 20:25:58.893481 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:71 2026-06-09 20:25:58.893491 | controller | STEP: Checking the secret is in the zuul.conf @ 06/09/26 20:25:58.893 2026-06-09 20:25:58.896529 | controller | STEP: Checking zuul-web works @ 06/09/26 20:25:58.896 2026-06-09 20:25:58.896569 | controller | 2026-06-09T20:25:58-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl zuul-web:9000/api/tenant/demo-tenant/builds] 2026-06-09 20:25:59.072941 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-09 20:25:59.073022 | controller | Dload Upload Total Spent Left Speed 2026-06-09 20:25:59.164177 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 1854 100 1854 0 0 20373 0 --:--:-- --:--:-- --:--:-- 20373 2026-06-09 20:25:59.167308 | controller | • [0.274 seconds] 2026-06-09 20:25:59.167368 | controller | ------------------------------ 2026-06-09 20:25:59.167374 | controller | Secret Rotations zuul-zk works 2026-06-09 20:25:59.167378 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:81 2026-06-09 20:25:59.167389 | controller | STEP: Checking the key in zuul pod @ 06/09/26 20:25:59.167 2026-06-09 20:25:59.167394 | controller | 2026-06-09T20:25:59-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /tls/client/tls.key] 2026-06-09 20:25:59.285211 | controller | STEP: Connecting with kazoo @ 06/09/26 20:25:59.285 2026-06-09 20:25:59.285291 | controller | 2026-06-09T20:25:59-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [python3 -c import os, kazoo.client; os.chdir('/tls/client'); hosts=list(filter(lambda l: l.startswith('hosts'), open('/etc/zuul/zuul.conf').readlines()))[0].split('=')[1].strip(); client = kazoo.client.KazooClient(hosts, use_ssl=True, keyfile='tls.key', certfile='tls.crt', ca='ca.crt'); client.start(); print(client.get_children('/keystorage2'))] 2026-06-09 20:25:59.672130 | controller | • [0.505 seconds] 2026-06-09 20:25:59.672210 | controller | ------------------------------ 2026-06-09 20:25:59.672258 | controller | gateway tests When setting up extra gateway configuration for a maintenance Should reconcile 2026-06-09 20:25:59.672272 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:28 2026-06-09 20:25:59.672311 | controller | 2026-06-09T20:25:59-04:00 INFO Creating object, name: maintenance-config 2026-06-09 20:25:59.675991 | controller | 2026-06-09T20:25:59-04:00 INFO Creating object, name: maintenance-static-files 2026-06-09 20:25:59.683150 | controller | 2026-06-09T20:25:59-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:25:59.705913 | controller | 2026-06-09T20:25:59-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:25:59.723196 | controller | 2026-06-09T20:25:59-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:26:00.302428 | controller | 2026-06-09T20:26:00-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:26:00.302508 | controller | 2026-06-09T20:26:00-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:26:00.305559 | controller | 2026-06-09T20:26:00-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:26:00.312524 | controller | 2026-06-09T20:26:00-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:26:00.422988 | controller | 2026-06-09T20:26:00-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:26:00.434938 | controller | 2026-06-09T20:26:00-04:00 INFO Statefulset "gateway" configuration changed, applying... 2026-06-09 20:26:00.434975 | controller | 2026-06-09T20:26:00-04:00 DEBUG Reason: annotation 'config-hash' changed from '90a6261c7272ad73ee593bd22fc08aa0b50c9e09eb380107181e1e58b851202b' to 'c04b8e5064e2a2985ef09fcc1ab4c2dda05fbcb2cd18006ff3d01f2b963e17d8' 2026-06-09 20:26:00.434983 | controller | 2026-06-09T20:26:00-04:00 INFO Updating object, name: gateway 2026-06-09 20:26:00.447221 | controller | 2026-06-09T20:26:00-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:26:00.460142 | controller | 2026-06-09T20:26:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:26:00.460192 | controller | 2026-06-09T20:26:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:26:00.711973 | controller | 2026-06-09T20:26:00-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:26:00.716392 | controller | 2026-06-09T20:26:00-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:26:01.115499 | controller | 2026-06-09T20:26:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:26:01.115571 | controller | 2026-06-09T20:26:01-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:26:01.690007 | controller | 2026-06-09T20:26:01-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:26:02.489956 | controller | 2026-06-09T20:26:02-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:26:02.498925 | controller | 2026-06-09T20:26:02-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:26:02.498959 | controller | 2026-06-09T20:26:02-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:26:03.294723 | controller | 2026-06-09T20:26:03-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:26:04.086960 | controller | 2026-06-09T20:26:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:26:04.087092 | controller | 2026-06-09T20:26:04-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:26:04.898554 | controller | 2026-06-09T20:26:04-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:26:05.299448 | controller | 2026-06-09T20:26:05-04:00 INFO 2026-06-09 20:26:05.299564 | controller | - Config: OK 2026-06-09 20:26:05.299576 | controller | - Gateway: Waiting ... 2026-06-09 20:26:05.299586 | controller | - GitServer: OK 2026-06-09 20:26:05.299595 | controller | - HoundSearch: OK 2026-06-09 20:26:05.299603 | controller | - LogJuicer: OK 2026-06-09 20:26:05.299612 | controller | - Logserver: OK 2026-06-09 20:26:05.299620 | controller | - MariaDB: OK 2026-06-09 20:26:05.299629 | controller | - NodePoolBuilder: OK 2026-06-09 20:26:05.299637 | controller | - NodePoolLauncher: OK 2026-06-09 20:26:05.299645 | controller | - Zookeeper: OK 2026-06-09 20:26:05.299654 | controller | - Zuul: OK 2026-06-09 20:26:05.299663 | controller | 2026-06-09 20:26:06.307008 | controller | 2026-06-09T20:26:06-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:26:06.336862 | controller | 2026-06-09T20:26:06-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:26:06.360266 | controller | 2026-06-09T20:26:06-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:26:06.910098 | controller | 2026-06-09T20:26:06-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:26:06.910227 | controller | 2026-06-09T20:26:06-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:26:06.913202 | controller | 2026-06-09T20:26:06-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:26:06.932342 | controller | 2026-06-09T20:26:06-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:26:06.932446 | controller | 2026-06-09T20:26:06-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:26:06.941636 | controller | 2026-06-09T20:26:06-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:26:07.092634 | controller | 2026-06-09T20:26:07-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:26:07.092754 | controller | 2026-06-09T20:26:07-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:26:07.317277 | controller | 2026-06-09T20:26:07-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:26:07.320643 | controller | 2026-06-09T20:26:07-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:26:08.093036 | controller | 2026-06-09T20:26:08-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:26:08.093142 | controller | 2026-06-09T20:26:08-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:26:08.695994 | controller | 2026-06-09T20:26:08-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:26:09.487925 | controller | 2026-06-09T20:26:09-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:26:09.493197 | controller | 2026-06-09T20:26:09-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:26:09.493254 | controller | 2026-06-09T20:26:09-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:26:10.294410 | controller | 2026-06-09T20:26:10-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:26:11.090703 | controller | 2026-06-09T20:26:11-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:26:11.090795 | controller | 2026-06-09T20:26:11-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:26:11.893815 | controller | 2026-06-09T20:26:11-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:26:12.297860 | controller | 2026-06-09T20:26:12-04:00 INFO 2026-06-09 20:26:12.297951 | controller | - Config: OK 2026-06-09 20:26:12.297962 | controller | - Gateway: OK 2026-06-09 20:26:12.297980 | controller | - GitServer: OK 2026-06-09 20:26:12.297989 | controller | - HoundSearch: OK 2026-06-09 20:26:12.297997 | controller | - LogJuicer: OK 2026-06-09 20:26:12.298005 | controller | - Logserver: OK 2026-06-09 20:26:12.298012 | controller | - MariaDB: OK 2026-06-09 20:26:12.298020 | controller | - NodePoolBuilder: OK 2026-06-09 20:26:12.298028 | controller | - NodePoolLauncher: OK 2026-06-09 20:26:12.298035 | controller | - Zookeeper: OK 2026-06-09 20:26:12.298043 | controller | - Zuul: OK 2026-06-09 20:26:12.298052 | controller | 2026-06-09 20:26:12.298067 | controller | • [12.626 seconds] 2026-06-09 20:26:12.298075 | controller | ------------------------------ 2026-06-09 20:26:12.298087 | controller | gateway tests When setting up extra gateway configuration for a maintenance Should serve HTTP 503 and the expected content 2026-06-09 20:26:12.298095 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:55 2026-06-09 20:26:12.298187 | controller | 2026-06-09T20:26:12-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -o - -I gateway:8080/zuul/] 2026-06-09 20:26:12.387624 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-09 20:26:12.387681 | controller | Dload Upload Total Spent Left Speed 2026-06-09 20:26:12.389530 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 17 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2026-06-09 20:26:12.392043 | controller | 2026-06-09T20:26:12-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl gateway:8080/zuul/api/components] 2026-06-09 20:26:12.473204 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-09 20:26:12.473275 | controller | Dload Upload Total Spent Left Speed 2026-06-09 20:26:12.474509 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 17 100 17 0 0 17000 0 --:--:-- --:--:-- --:--:-- 17000 2026-06-09 20:26:12.476769 | controller | • [0.179 seconds] 2026-06-09 20:26:12.476785 | controller | ------------------------------ 2026-06-09 20:26:12.476792 | controller | gateway tests When removing the gateway's extra config Should reconcile 2026-06-09 20:26:12.476796 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:65 2026-06-09 20:26:12.483257 | controller | 2026-06-09T20:26:12-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:26:12.898969 | controller | 2026-06-09T20:26:12-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:26:12.916745 | controller | 2026-06-09T20:26:12-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:26:13.103582 | controller | 2026-06-09T20:26:13-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:26:13.103668 | controller | 2026-06-09T20:26:13-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:26:13.107312 | controller | 2026-06-09T20:26:13-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:26:13.113629 | controller | 2026-06-09T20:26:13-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:26:13.266888 | controller | 2026-06-09T20:26:13-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:26:13.285004 | controller | 2026-06-09T20:26:13-04:00 INFO Statefulset "gateway" configuration changed, applying... 2026-06-09 20:26:13.285067 | controller | 2026-06-09T20:26:13-04:00 DEBUG Reason: annotation 'config-hash' changed from 'c04b8e5064e2a2985ef09fcc1ab4c2dda05fbcb2cd18006ff3d01f2b963e17d8' to '90a6261c7272ad73ee593bd22fc08aa0b50c9e09eb380107181e1e58b851202b' 2026-06-09 20:26:13.285089 | controller | 2026-06-09T20:26:13-04:00 INFO Updating object, name: gateway 2026-06-09 20:26:13.490839 | controller | 2026-06-09T20:26:13-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:26:13.687509 | controller | 2026-06-09T20:26:13-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:26:13.687609 | controller | 2026-06-09T20:26:13-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:26:13.896021 | controller | 2026-06-09T20:26:13-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:26:13.899221 | controller | 2026-06-09T20:26:13-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:26:14.694160 | controller | 2026-06-09T20:26:14-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:26:14.694274 | controller | 2026-06-09T20:26:14-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:26:15.291916 | controller | 2026-06-09T20:26:15-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:26:16.093666 | controller | 2026-06-09T20:26:16-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:26:16.102625 | controller | 2026-06-09T20:26:16-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:26:16.102688 | controller | 2026-06-09T20:26:16-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:26:16.891658 | controller | 2026-06-09T20:26:16-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:26:17.688456 | controller | 2026-06-09T20:26:17-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:26:17.688533 | controller | 2026-06-09T20:26:17-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:26:18.493389 | controller | 2026-06-09T20:26:18-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:26:18.894513 | controller | 2026-06-09T20:26:18-04:00 INFO 2026-06-09 20:26:18.894592 | controller | - Config: OK 2026-06-09 20:26:18.894608 | controller | - Gateway: Waiting ... 2026-06-09 20:26:18.894620 | controller | - GitServer: OK 2026-06-09 20:26:18.894632 | controller | - HoundSearch: OK 2026-06-09 20:26:18.894640 | controller | - LogJuicer: OK 2026-06-09 20:26:18.894647 | controller | - Logserver: OK 2026-06-09 20:26:18.894655 | controller | - MariaDB: OK 2026-06-09 20:26:18.894663 | controller | - NodePoolBuilder: OK 2026-06-09 20:26:18.894670 | controller | - NodePoolLauncher: OK 2026-06-09 20:26:18.894678 | controller | - Zookeeper: OK 2026-06-09 20:26:18.894686 | controller | - Zuul: OK 2026-06-09 20:26:18.894694 | controller | 2026-06-09 20:26:19.899418 | controller | 2026-06-09T20:26:19-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:26:19.920221 | controller | 2026-06-09T20:26:19-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:26:19.937800 | controller | 2026-06-09T20:26:19-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:26:20.506922 | controller | 2026-06-09T20:26:20-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-09 20:26:20.506985 | controller | 2026-06-09T20:26:20-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-09 20:26:20.509595 | controller | 2026-06-09T20:26:20-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:26:20.520738 | controller | 2026-06-09T20:26:20-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:26:20.520803 | controller | 2026-06-09T20:26:20-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:26:20.531137 | controller | 2026-06-09T20:26:20-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:26:20.543662 | controller | 2026-06-09T20:26:20-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:26:20.543747 | controller | 2026-06-09T20:26:20-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:26:20.910888 | controller | 2026-06-09T20:26:20-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:26:20.913009 | controller | 2026-06-09T20:26:20-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:26:21.321028 | controller | 2026-06-09T20:26:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:26:21.321113 | controller | 2026-06-09T20:26:21-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:26:21.887545 | controller | 2026-06-09T20:26:21-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:26:22.691646 | controller | 2026-06-09T20:26:22-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:26:22.700370 | controller | 2026-06-09T20:26:22-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:26:22.700503 | controller | 2026-06-09T20:26:22-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:26:23.492909 | controller | 2026-06-09T20:26:23-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:26:24.287092 | controller | 2026-06-09T20:26:24-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:26:24.287179 | controller | 2026-06-09T20:26:24-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:26:25.093995 | controller | 2026-06-09T20:26:25-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:26:25.493449 | controller | 2026-06-09T20:26:25-04:00 INFO 2026-06-09 20:26:25.493519 | controller | - Config: OK 2026-06-09 20:26:25.493530 | controller | - Gateway: OK 2026-06-09 20:26:25.493538 | controller | - GitServer: OK 2026-06-09 20:26:25.493546 | controller | - HoundSearch: OK 2026-06-09 20:26:25.493554 | controller | - LogJuicer: OK 2026-06-09 20:26:25.493562 | controller | - Logserver: OK 2026-06-09 20:26:25.493569 | controller | - MariaDB: OK 2026-06-09 20:26:25.493577 | controller | - NodePoolBuilder: OK 2026-06-09 20:26:25.493585 | controller | - NodePoolLauncher: OK 2026-06-09 20:26:25.493592 | controller | - Zookeeper: OK 2026-06-09 20:26:25.493600 | controller | - Zuul: OK 2026-06-09 20:26:25.493609 | controller | 2026-06-09 20:26:25.493617 | controller | • [13.017 seconds] 2026-06-09 20:26:25.493625 | controller | ------------------------------ 2026-06-09 20:26:25.493675 | controller | gateway tests When removing the gateway's extra config Should serve the regular content 2026-06-09 20:26:25.493688 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:71 2026-06-09 20:26:25.493696 | controller | 2026-06-09T20:26:25-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -o - -I gateway:8080/zuul/api/components] 2026-06-09 20:26:25.667054 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-09 20:26:25.667118 | controller | Dload Upload Total Spent Left Speed 2026-06-09 20:26:25.672533 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 537 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2026-06-09 20:26:25.675756 | controller | 2026-06-09T20:26:25-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl gateway:8080/zuul/api/components] 2026-06-09 20:26:25.790460 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-09 20:26:25.790514 | controller | Dload Upload Total Spent Left Speed 2026-06-09 20:26:25.794310 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 537 100 537 0 0 131k 0 --:--:-- --:--:-- --:--:-- 131k 2026-06-09 20:26:25.797145 | controller | • [0.304 seconds] 2026-06-09 20:26:25.797191 | controller | ------------------------------ 2026-06-09 20:26:25.797224 | controller | Test Env Has namespace 2026-06-09 20:26:25.797259 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/main_test.go:167 2026-06-09 20:26:25.797265 | controller | • [0.000 seconds] 2026-06-09 20:26:25.797298 | controller | ------------------------------ 2026-06-09 20:26:25.797306 | controller | 2026-06-09 20:26:25.797341 | controller | Ran 16 of 16 Specs in 473.552 seconds 2026-06-09 20:26:25.797391 | controller | SUCCESS! -- 16 Passed | 0 Failed | 0 Pending | 0 Skipped 2026-06-09 20:26:25.797431 | controller | --- PASS: TestSFOP (473.55s) 2026-06-09 20:26:25.797465 | controller | PASS 2026-06-09 20:26:25.803697 | controller | ok github.com/softwarefactory-project/sf-operator/tests 473.580s 2026-06-10 00:26:26.332001 | controller | ok: Runtime: 0:08:02.923397 2026-06-10 00:26:26.381072 | 2026-06-10 00:26:26.381278 | TASK [health-check/test-standalone-update : Create a temporary directory for generated CRs] 2026-06-10 00:26:26.739953 | controller | changed 2026-06-10 00:26:26.749444 | 2026-06-10 00:26:26.749632 | TASK [health-check/test-standalone-update : Generate initial CR] 2026-06-10 00:26:27.636209 | controller | changed 2026-06-10 00:26:27.645112 | 2026-06-10 00:26:27.645245 | TASK [Run standalone operator with initial config] 2026-06-10 00:26:27.681076 | controller | ok 2026-06-10 00:26:27.713698 | 2026-06-10 00:26:27.713850 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:26:27.957186 | controller | ok 2026-06-10 00:26:27.961714 | 2026-06-10 00:26:27.961779 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:26:28.212025 | controller | changed 2026-06-10 00:26:28.220978 | 2026-06-10 00:26:28.221130 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:26:28.897600 | controller | 2026-06-09T20:26:28-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:26:28.898104 | controller | 2026-06-09 20:26:28.909113 | controller | 2026-06-09T20:26:28-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:26:28.909192 | controller | 2026-06-09 20:26:28.929393 | controller | 2026-06-09T20:26:28-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:26:28.940797 | controller | 2026-06-09T20:26:28-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-09 20:26:28.940870 | controller | 2026-06-09T20:26:28-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-09 20:26:28.954853 | controller | 2026-06-09T20:26:28-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-09 20:26:28.954919 | controller | 2026-06-09T20:26:28-04:00 INFO Updating object, name: git-server 2026-06-09 20:26:28.987018 | controller | 2026-06-09T20:26:28-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:26:29.532821 | controller | 2026-06-09T20:26:29-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-09 20:26:29.532942 | controller | 2026-06-09T20:26:29-04:00 INFO Updating object, name: logserver 2026-06-09 20:26:29.555494 | controller | 2026-06-09T20:26:29-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:26:29.555666 | controller | 2026-06-09T20:26:29-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:26:29.571109 | controller | 2026-06-09T20:26:29-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:26:29.586996 | controller | 2026-06-09T20:26:29-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:26:29.587100 | controller | 2026-06-09T20:26:29-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:26:29.942285 | controller | 2026-06-09T20:26:29-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:26:29.944376 | controller | 2026-06-09T20:26:29-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 20:26:29.944405 | controller | 2026-06-09T20:26:29-04:00 INFO 2026-06-09 20:26:29.944416 | controller | - Config: Waiting ... 2026-06-09 20:26:29.944426 | controller | - Gateway: OK 2026-06-09 20:26:29.944435 | controller | - GitServer: Waiting ... 2026-06-09 20:26:29.944443 | controller | - HoundSearch: OK 2026-06-09 20:26:29.944451 | controller | - LogJuicer: OK 2026-06-09 20:26:29.944459 | controller | - Logserver: Waiting ... 2026-06-09 20:26:29.944467 | controller | - MariaDB: OK 2026-06-09 20:26:29.944475 | controller | - Zookeeper: OK 2026-06-09 20:26:29.944483 | controller | 2026-06-09 20:26:29.944497 | controller | 2026-06-09T20:26:29-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:26:34.962912 | controller | 2026-06-09T20:26:34-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:26:35.005184 | controller | 2026-06-09T20:26:35-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:26:35.005332 | controller | 2026-06-09T20:26:35-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:26:40.015684 | controller | 2026-06-09T20:26:40-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:26:40.015737 | controller | 2026-06-09T20:26:40-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:26:45.025935 | controller | 2026-06-09T20:26:45-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:26:45.026028 | controller | 2026-06-09T20:26:45-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:26:50.039689 | controller | 2026-06-09T20:26:50-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:26:50.039769 | controller | 2026-06-09T20:26:50-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:26:55.052315 | controller | 2026-06-09T20:26:55-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:26:55.076706 | controller | 2026-06-09T20:26:55-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:26:55.116504 | controller | 2026-06-09T20:26:55-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:26:55.116629 | controller | 2026-06-09T20:26:55-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:27:00.123636 | controller | 2026-06-09T20:27:00-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:27:00.123710 | controller | 2026-06-09T20:27:00-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:27:05.135836 | controller | 2026-06-09T20:27:05-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:27:05.135973 | controller | 2026-06-09T20:27:05-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:27:10.143114 | controller | 2026-06-09T20:27:10-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:27:10.143267 | controller | 2026-06-09T20:27:10-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:27:15.154883 | controller | 2026-06-09T20:27:15-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:27:15.154927 | controller | 2026-06-09T20:27:15-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:27:20.165860 | controller | 2026-06-09T20:27:20-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:27:20.165945 | controller | 2026-06-09T20:27:20-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:27:25.169700 | controller | 2026-06-09T20:27:25-04:00 INFO Statefulset "logserver" not ready after 30s 2026-06-09 20:27:25.177355 | controller | 2026-06-09T20:27:25-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:27:25.177407 | controller | 2026-06-09T20:27:25-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:27:25.185600 | controller | 2026-06-09T20:27:25-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:27:25.196355 | controller | 2026-06-09T20:27:25-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:27:25.196413 | controller | 2026-06-09T20:27:25-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:27:25.212775 | controller | 2026-06-09T20:27:25-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:27:25.216080 | controller | 2026-06-09T20:27:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:27:25.238437 | controller | 2026-06-09T20:27:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:27:25.238494 | controller | 2026-06-09T20:27:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:27:25.385518 | controller | 2026-06-09T20:27:25-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:27:25.396604 | controller | 2026-06-09T20:27:25-04:00 INFO Updating secret, name: zuul-config 2026-06-09 20:27:25.396652 | controller | 2026-06-09T20:27:25-04:00 INFO Updating object, name: zuul-config 2026-06-09 20:27:26.179354 | controller | 2026-06-09T20:27:26-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 20:27:26.179419 | controller | 2026-06-09T20:27:26-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 20:27:26.986654 | controller | 2026-06-09T20:27:26-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:27:27.779793 | controller | 2026-06-09T20:27:27-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 20:27:27.779856 | controller | 2026-06-09T20:27:27-04:00 INFO Updating object, name: zuul-web 2026-06-09 20:27:28.589891 | controller | 2026-06-09T20:27:28-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:27:28.594113 | controller | 2026-06-09T20:27:28-04:00 INFO 2026-06-09 20:27:28.594158 | controller | - Config: Waiting ... 2026-06-09 20:27:28.594170 | controller | - Gateway: OK 2026-06-09 20:27:28.594178 | controller | - GitServer: OK 2026-06-09 20:27:28.594186 | controller | - HoundSearch: OK 2026-06-09 20:27:28.594195 | controller | - LogJuicer: OK 2026-06-09 20:27:28.594203 | controller | - Logserver: Waiting ... 2026-06-09 20:27:28.594210 | controller | - MariaDB: OK 2026-06-09 20:27:28.594218 | controller | - NodePoolBuilder: OK 2026-06-09 20:27:28.594226 | controller | - NodePoolLauncher: OK 2026-06-09 20:27:28.594273 | controller | - Zookeeper: OK 2026-06-09 20:27:28.594282 | controller | - Zuul: Waiting ... 2026-06-09 20:27:28.594290 | controller | - |_ Config Secrets: OK 2026-06-09 20:27:28.594298 | controller | - |_ Executor: OK 2026-06-09 20:27:28.594306 | controller | - |_ Merger: OK 2026-06-09 20:27:28.594314 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:27:28.594322 | controller | - |_ Web: Waiting ... 2026-06-09 20:27:28.594341 | controller | 2026-06-09 20:27:28.594351 | controller | 2026-06-09T20:27:28-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:27:33.601109 | controller | 2026-06-09T20:27:33-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:27:33.633725 | controller | 2026-06-09T20:27:33-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:27:33.656779 | controller | 2026-06-09T20:27:33-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:27:34.210053 | controller | 2026-06-09T20:27:34-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:27:34.216885 | controller | 2026-06-09T20:27:34-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:27:34.371209 | controller | 2026-06-09T20:27:34-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:27:34.382898 | controller | 2026-06-09T20:27:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:27:34.382955 | controller | 2026-06-09T20:27:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:27:34.392503 | controller | 2026-06-09T20:27:34-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:27:34.404856 | controller | 2026-06-09T20:27:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:27:34.404906 | controller | 2026-06-09T20:27:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:27:34.608616 | controller | 2026-06-09T20:27:34-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:27:34.610951 | controller | 2026-06-09T20:27:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:27:35.023816 | controller | 2026-06-09T20:27:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:27:35.023897 | controller | 2026-06-09T20:27:35-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:27:35.219649 | controller | 2026-06-09T20:27:35-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:27:36.020181 | controller | 2026-06-09T20:27:36-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:27:36.020280 | controller | 2026-06-09T20:27:36-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:27:41.031422 | controller | 2026-06-09T20:27:41-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:27:41.031494 | controller | 2026-06-09T20:27:41-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:27:46.043339 | controller | 2026-06-09T20:27:46-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:27:46.043451 | controller | 2026-06-09T20:27:46-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:27:51.053476 | controller | 2026-06-09T20:27:51-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:27:51.053557 | controller | 2026-06-09T20:27:51-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:27:56.062831 | controller | 2026-06-09T20:27:56-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:27:56.062886 | controller | 2026-06-09T20:27:56-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:28:01.076321 | controller | 2026-06-09T20:28:01-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:28:01.076399 | controller | 2026-06-09T20:28:01-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:28:06.079953 | controller | 2026-06-09T20:28:06-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:28:06.100782 | controller | 2026-06-09T20:28:06-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:28:06.115932 | controller | 2026-06-09T20:28:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:28:06.115993 | controller | 2026-06-09T20:28:06-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:28:06.492745 | controller | 2026-06-09T20:28:06-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:28:06.495589 | controller | 2026-06-09T20:28:06-04:00 INFO 2026-06-09 20:28:06.495638 | controller | - Config: Waiting ... 2026-06-09 20:28:06.495649 | controller | - Gateway: OK 2026-06-09 20:28:06.495658 | controller | - GitServer: OK 2026-06-09 20:28:06.495668 | controller | - HoundSearch: OK 2026-06-09 20:28:06.495679 | controller | - LogJuicer: OK 2026-06-09 20:28:06.495690 | controller | - Logserver: OK 2026-06-09 20:28:06.495701 | controller | - MariaDB: OK 2026-06-09 20:28:06.495712 | controller | - NodePoolBuilder: OK 2026-06-09 20:28:06.495723 | controller | - NodePoolLauncher: OK 2026-06-09 20:28:06.495734 | controller | - Zookeeper: OK 2026-06-09 20:28:06.495744 | controller | - Zuul: Waiting ... 2026-06-09 20:28:06.495755 | controller | - |_ Config Secrets: OK 2026-06-09 20:28:06.495766 | controller | - |_ Executor: OK 2026-06-09 20:28:06.495777 | controller | - |_ Merger: OK 2026-06-09 20:28:06.495788 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:28:06.495799 | controller | - |_ Web: Waiting ... 2026-06-09 20:28:06.495810 | controller | 2026-06-09 20:28:06.495896 | controller | 2026-06-09T20:28:06-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:28:11.500814 | controller | 2026-06-09T20:28:11-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:28:11.526486 | controller | 2026-06-09T20:28:11-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:28:11.542615 | controller | 2026-06-09T20:28:11-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:28:12.109139 | controller | 2026-06-09T20:28:12-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:28:12.123178 | controller | 2026-06-09T20:28:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:28:12.123208 | controller | 2026-06-09T20:28:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:28:12.132261 | controller | 2026-06-09T20:28:12-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:28:12.147305 | controller | 2026-06-09T20:28:12-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:28:12.147373 | controller | 2026-06-09T20:28:12-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:28:12.514001 | controller | 2026-06-09T20:28:12-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:28:12.517189 | controller | 2026-06-09T20:28:12-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:28:12.922663 | controller | 2026-06-09T20:28:12-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:28:12.922740 | controller | 2026-06-09T20:28:12-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:28:13.116161 | controller | 2026-06-09T20:28:13-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:28:13.924694 | controller | 2026-06-09T20:28:13-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:28:13.924776 | controller | 2026-06-09T20:28:13-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:28:18.935143 | controller | 2026-06-09T20:28:18-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:28:18.935218 | controller | 2026-06-09T20:28:18-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:28:23.945128 | controller | 2026-06-09T20:28:23-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:28:23.945189 | controller | 2026-06-09T20:28:23-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:28:28.954985 | controller | 2026-06-09T20:28:28-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:28:28.955050 | controller | 2026-06-09T20:28:28-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:28:33.964108 | controller | 2026-06-09T20:28:33-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:28:33.974856 | controller | 2026-06-09T20:28:33-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:28:33.974914 | controller | 2026-06-09T20:28:33-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:28:33.990298 | controller | 2026-06-09T20:28:33-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:28:34.004703 | controller | 2026-06-09T20:28:34-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:28:34.004756 | controller | 2026-06-09T20:28:34-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:28:34.389152 | controller | 2026-06-09T20:28:34-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:28:34.793309 | controller | 2026-06-09T20:28:34-04:00 INFO 2026-06-09 20:28:34.793385 | controller | - Config: OK 2026-06-09 20:28:34.793396 | controller | - Gateway: OK 2026-06-09 20:28:34.793405 | controller | - GitServer: OK 2026-06-09 20:28:34.793436 | controller | - HoundSearch: OK 2026-06-09 20:28:34.793444 | controller | - LogJuicer: OK 2026-06-09 20:28:34.793451 | controller | - Logserver: OK 2026-06-09 20:28:34.793459 | controller | - MariaDB: OK 2026-06-09 20:28:34.793467 | controller | - NodePoolBuilder: OK 2026-06-09 20:28:34.793475 | controller | - NodePoolLauncher: OK 2026-06-09 20:28:34.793483 | controller | - Zookeeper: OK 2026-06-09 20:28:34.793491 | controller | - Zuul: OK 2026-06-09 20:28:34.793499 | controller | 2026-06-09 20:28:34.793514 | controller | 2026-06-09T20:28:34-04:00 INFO Updating controller configmap ... 2026-06-09 20:28:35.179656 | controller | 2026-06-09T20:28:35-04:00 INFO Standalone reconcile done. 2026-06-10 00:28:35.436800 | controller | ok: Runtime: 0:02:06.663904 2026-06-10 00:28:35.449863 | 2026-06-10 00:28:35.450081 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:28:35.997225 | controller | ok: Runtime: 0:00:00.008331 2026-06-10 00:28:36.015427 | 2026-06-10 00:28:36.015637 | TASK [health-check/test-standalone-update : Get standalone owner configmap after initial creation] 2026-06-09 20:28:36.340470 | controller | fqdn: test-initial.sf.io 2026-06-09 20:28:36.340590 | controller | fluentbitlogforwarding: null 2026-06-09 20:28:36.340613 | controller | storagedefault: 2026-06-09 20:28:36.340628 | controller | classname: "" 2026-06-09 20:28:36.340641 | controller | extraannotations: {} 2026-06-09 20:28:36.340653 | controller | extralabels: {} 2026-06-09 20:28:36.340665 | controller | configrepositorylocation: 2026-06-09 20:28:36.340681 | controller | name: config 2026-06-09 20:28:36.340696 | controller | branch: "" 2026-06-09 20:28:36.340710 | controller | zuulconnectionname: gerrit 2026-06-09 20:28:36.340722 | controller | clusterapiurl: "" 2026-06-09 20:28:36.340737 | controller | logserverhost: "" 2026-06-09 20:28:36.340750 | controller | zuul: 2026-06-09 20:28:36.340762 | controller | oidcauthenticators: [] 2026-06-09 20:28:36.340774 | controller | defaultauthenticator: "" 2026-06-09 20:28:36.340787 | controller | gerritconns: 2026-06-09 20:28:36.340799 | controller | - name: gerrit 2026-06-09 20:28:36.340818 | controller | hostname: gerrit-sshd.sf 2026-06-09 20:28:36.340831 | controller | port: 0 2026-06-09 20:28:36.340846 | controller | puburl: https://gerrit.sfop.me 2026-06-09 20:28:36.340859 | controller | username: zuul 2026-06-09 20:28:36.340872 | controller | canonicalhostname: "" 2026-06-09 20:28:36.340885 | controller | password: "" 2026-06-09 20:28:36.340898 | controller | sshkey: "" 2026-06-09 20:28:36.340909 | controller | gitoverssh: false 2026-06-09 20:28:36.340921 | controller | verifyssl: false 2026-06-09 20:28:36.340934 | controller | streamevents: null 2026-06-09 20:28:36.340945 | controller | githubconns: [] 2026-06-09 20:28:36.340957 | controller | gitlabconns: [] 2026-06-09 20:28:36.340969 | controller | gitconns: [] 2026-06-09 20:28:36.340981 | controller | pagureconns: [] 2026-06-09 20:28:36.340993 | controller | elasticsearchconns: [] 2026-06-09 20:28:36.341005 | controller | smtpconns: [] 2026-06-09 20:28:36.341018 | controller | executor: 2026-06-09 20:28:36.341031 | controller | storage: 2026-06-09 20:28:36.341043 | controller | size: 2026-06-09 20:28:36.341055 | controller | format: "" 2026-06-09 20:28:36.341067 | controller | classname: "" 2026-06-09 20:28:36.341079 | controller | loglevel: DEBUG 2026-06-09 20:28:36.341091 | controller | enabled: true 2026-06-09 20:28:36.341103 | controller | standalone: null 2026-06-09 20:28:36.341115 | controller | limits: null 2026-06-09 20:28:36.341127 | controller | disklimitperjob: 0 2026-06-09 20:28:36.341139 | controller | ansiblesetuptimeout: 0 2026-06-09 20:28:36.341159 | controller | terminationgraceperiodseconds: 0 2026-06-09 20:28:36.341172 | controller | scheduler: 2026-06-09 20:28:36.341184 | controller | storage: 2026-06-09 20:28:36.341196 | controller | size: 2026-06-09 20:28:36.341208 | controller | format: "" 2026-06-09 20:28:36.341219 | controller | classname: "" 2026-06-09 20:28:36.341278 | controller | statsdtarget: "" 2026-06-09 20:28:36.341299 | controller | loglevel: DEBUG 2026-06-09 20:28:36.341311 | controller | limits: null 2026-06-09 20:28:36.341322 | controller | defaultholdexpiration: null 2026-06-09 20:28:36.341333 | controller | maxholdexpiration: null 2026-06-09 20:28:36.341344 | controller | web: 2026-06-09 20:28:36.341355 | controller | loglevel: DEBUG 2026-06-09 20:28:36.341366 | controller | limits: null 2026-06-09 20:28:36.341377 | controller | merger: 2026-06-09 20:28:36.341389 | controller | gitusername: "" 2026-06-09 20:28:36.341400 | controller | gituseremail: "" 2026-06-09 20:28:36.341412 | controller | githttplowspeedlimit: 0 2026-06-09 20:28:36.341424 | controller | githttplowspeedtime: 0 2026-06-09 20:28:36.341436 | controller | gittimeout: 0 2026-06-09 20:28:36.341449 | controller | storage: 2026-06-09 20:28:36.341461 | controller | size: 2026-06-09 20:28:36.341473 | controller | format: "" 2026-06-09 20:28:36.341486 | controller | classname: "" 2026-06-09 20:28:36.341498 | controller | loglevel: DEBUG 2026-06-09 20:28:36.341510 | controller | limits: null 2026-06-09 20:28:36.341521 | controller | nodepool: 2026-06-09 20:28:36.341532 | controller | launcher: 2026-06-09 20:28:36.341544 | controller | loglevel: "" 2026-06-09 20:28:36.341556 | controller | limits: null 2026-06-09 20:28:36.341567 | controller | builder: 2026-06-09 20:28:36.341580 | controller | storage: 2026-06-09 20:28:36.341592 | controller | size: 2026-06-09 20:28:36.341603 | controller | format: "" 2026-06-09 20:28:36.341616 | controller | classname: "" 2026-06-09 20:28:36.341628 | controller | loglevel: "" 2026-06-09 20:28:36.341655 | controller | limits: null 2026-06-09 20:28:36.341668 | controller | statsdtarget: "" 2026-06-09 20:28:36.341680 | controller | zookeeper: 2026-06-09 20:28:36.341693 | controller | storage: 2026-06-09 20:28:36.341705 | controller | size: 2026-06-09 20:28:36.341717 | controller | format: "" 2026-06-09 20:28:36.341729 | controller | classname: "" 2026-06-09 20:28:36.341741 | controller | limits: null 2026-06-09 20:28:36.341753 | controller | logserver: 2026-06-09 20:28:36.341765 | controller | retentiondays: 30 2026-06-09 20:28:36.341776 | controller | loopdelay: 3600 2026-06-09 20:28:36.341788 | controller | storage: 2026-06-09 20:28:36.341801 | controller | size: 2026-06-09 20:28:36.341813 | controller | format: BinarySI 2026-06-09 20:28:36.341825 | controller | classname: "" 2026-06-09 20:28:36.341837 | controller | podannotations: {} 2026-06-09 20:28:36.341849 | controller | logjuicer: 2026-06-09 20:28:36.341860 | controller | size: 2026-06-09 20:28:36.341872 | controller | format: "" 2026-06-09 20:28:36.341883 | controller | classname: "" 2026-06-09 20:28:36.341895 | controller | mariadb: 2026-06-09 20:28:36.341906 | controller | dbstorage: 2026-06-09 20:28:36.341918 | controller | size: 2026-06-09 20:28:36.341929 | controller | format: "" 2026-06-09 20:28:36.341941 | controller | classname: "" 2026-06-09 20:28:36.341953 | controller | logstorage: 2026-06-09 20:28:36.341965 | controller | size: 2026-06-09 20:28:36.341977 | controller | format: "" 2026-06-09 20:28:36.341989 | controller | classname: "" 2026-06-09 20:28:36.342001 | controller | limits: null 2026-06-09 20:28:36.342013 | controller | gitserver: 2026-06-09 20:28:36.342025 | controller | storage: 2026-06-09 20:28:36.342037 | controller | size: 2026-06-09 20:28:36.342048 | controller | format: "" 2026-06-09 20:28:36.342060 | controller | classname: "" 2026-06-09 20:28:36.342073 | controller | codesearch: 2026-06-09 20:28:36.342085 | controller | storage: 2026-06-09 20:28:36.342097 | controller | size: 2026-06-09 20:28:36.342109 | controller | format: "" 2026-06-09 20:28:36.342121 | controller | classname: "" 2026-06-09 20:28:36.342134 | controller | limits: null 2026-06-09 20:28:36.342146 | controller | enabled: null 2026-06-09 20:28:36.342157 | controller | hostaliases: [] 2026-06-09 20:28:36.342168 | controller | gateway: null 2026-06-10 00:28:36.569227 | controller | ok: Runtime: 0:00:00.093600 2026-06-10 00:28:36.582259 | 2026-06-10 00:28:36.582515 | TASK [health-check/test-standalone-update : Verify initial spec in configmap] 2026-06-10 00:28:36.626914 | controller | ok: All assertions passed 2026-06-10 00:28:36.633047 | 2026-06-10 00:28:36.633140 | TASK [health-check/test-standalone-update : Get standalone owner configmap annotation after initial creation] 2026-06-10 00:28:37.167920 | controller | ok: Runtime: 0:00:00.067170 2026-06-10 00:28:37.185816 | 2026-06-10 00:28:37.185935 | TASK [health-check/test-standalone-update : Pause for 2 seconds to ensure timestamp difference] 2026-06-10 00:28:39.217563 | controller | ok 2026-06-10 00:28:39.226973 | 2026-06-10 00:28:39.227111 | TASK [health-check/test-standalone-update : Generate updated CR] 2026-06-10 00:28:39.954835 | controller | changed 2026-06-10 00:28:39.965083 | 2026-06-10 00:28:39.965244 | TASK [Run standalone operator with updated config] 2026-06-10 00:28:40.000720 | controller | ok 2026-06-10 00:28:40.040655 | 2026-06-10 00:28:40.040831 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:28:40.318944 | controller | ok 2026-06-10 00:28:40.334542 | 2026-06-10 00:28:40.334800 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:28:40.597521 | controller | changed 2026-06-10 00:28:40.606605 | 2026-06-10 00:28:40.606746 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:28:41.173025 | controller | 2026-06-09T20:28:41-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:28:41.173124 | controller | 2026-06-09 20:28:41.181916 | controller | 2026-06-09T20:28:41-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:28:41.181969 | controller | 2026-06-09 20:28:41.194405 | controller | 2026-06-09T20:28:41-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:28:41.203069 | controller | 2026-06-09T20:28:41-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-09 20:28:41.203123 | controller | 2026-06-09T20:28:41-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-09 20:28:41.214405 | controller | 2026-06-09T20:28:41-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-09 20:28:41.214461 | controller | 2026-06-09T20:28:41-04:00 INFO Updating object, name: git-server 2026-06-09 20:28:41.237718 | controller | 2026-06-09T20:28:41-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:28:41.800523 | controller | 2026-06-09T20:28:41-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-09 20:28:41.800586 | controller | 2026-06-09T20:28:41-04:00 INFO Updating object, name: logserver 2026-06-09 20:28:41.817293 | controller | 2026-06-09T20:28:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:28:41.817352 | controller | 2026-06-09T20:28:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:28:41.834541 | controller | 2026-06-09T20:28:41-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:28:41.849893 | controller | 2026-06-09T20:28:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:28:41.849950 | controller | 2026-06-09T20:28:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:28:42.206705 | controller | 2026-06-09T20:28:42-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:28:42.211662 | controller | 2026-06-09T20:28:42-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 20:28:42.211703 | controller | 2026-06-09T20:28:42-04:00 INFO 2026-06-09 20:28:42.211714 | controller | - Config: Waiting ... 2026-06-09 20:28:42.211723 | controller | - Gateway: OK 2026-06-09 20:28:42.211731 | controller | - GitServer: Waiting ... 2026-06-09 20:28:42.211739 | controller | - HoundSearch: OK 2026-06-09 20:28:42.211747 | controller | - LogJuicer: OK 2026-06-09 20:28:42.211755 | controller | - Logserver: Waiting ... 2026-06-09 20:28:42.211763 | controller | - MariaDB: OK 2026-06-09 20:28:42.211771 | controller | - Zookeeper: OK 2026-06-09 20:28:42.211779 | controller | 2026-06-09 20:28:42.211792 | controller | 2026-06-09T20:28:42-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:28:47.222256 | controller | 2026-06-09T20:28:47-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:28:47.265829 | controller | 2026-06-09T20:28:47-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:28:47.265931 | controller | 2026-06-09T20:28:47-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:28:52.277597 | controller | 2026-06-09T20:28:52-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:28:52.277661 | controller | 2026-06-09T20:28:52-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:28:57.287016 | controller | 2026-06-09T20:28:57-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:28:57.287091 | controller | 2026-06-09T20:28:57-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:29:02.296711 | controller | 2026-06-09T20:29:02-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:29:02.296775 | controller | 2026-06-09T20:29:02-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:29:07.307746 | controller | 2026-06-09T20:29:07-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:29:07.325149 | controller | 2026-06-09T20:29:07-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:29:07.353903 | controller | 2026-06-09T20:29:07-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:29:07.353959 | controller | 2026-06-09T20:29:07-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:29:12.364194 | controller | 2026-06-09T20:29:12-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:29:12.364343 | controller | 2026-06-09T20:29:12-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:29:17.377417 | controller | 2026-06-09T20:29:17-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:29:17.377503 | controller | 2026-06-09T20:29:17-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:29:22.385131 | controller | 2026-06-09T20:29:22-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:29:22.385288 | controller | 2026-06-09T20:29:22-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:29:27.392711 | controller | 2026-06-09T20:29:27-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:29:27.392778 | controller | 2026-06-09T20:29:27-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:29:32.401587 | controller | 2026-06-09T20:29:32-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:29:32.401677 | controller | 2026-06-09T20:29:32-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:29:37.407169 | controller | 2026-06-09T20:29:37-04:00 INFO Statefulset "logserver" not ready after 30s 2026-06-09 20:29:37.415697 | controller | 2026-06-09T20:29:37-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:29:37.415757 | controller | 2026-06-09T20:29:37-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:29:37.424112 | controller | 2026-06-09T20:29:37-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:29:37.440217 | controller | 2026-06-09T20:29:37-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:29:37.440290 | controller | 2026-06-09T20:29:37-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:29:37.455755 | controller | 2026-06-09T20:29:37-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:29:37.457674 | controller | 2026-06-09T20:29:37-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:29:37.477530 | controller | 2026-06-09T20:29:37-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:29:37.477636 | controller | 2026-06-09T20:29:37-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:29:37.625625 | controller | 2026-06-09T20:29:37-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:29:37.635294 | controller | 2026-06-09T20:29:37-04:00 INFO Updating secret, name: zuul-config 2026-06-09 20:29:37.635351 | controller | 2026-06-09T20:29:37-04:00 INFO Updating object, name: zuul-config 2026-06-09 20:29:38.418870 | controller | 2026-06-09T20:29:38-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 20:29:38.419000 | controller | 2026-06-09T20:29:38-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 20:29:39.222016 | controller | 2026-06-09T20:29:39-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:29:40.019536 | controller | 2026-06-09T20:29:40-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 20:29:40.019668 | controller | 2026-06-09T20:29:40-04:00 INFO Updating object, name: zuul-web 2026-06-09 20:29:40.827764 | controller | 2026-06-09T20:29:40-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:29:40.831757 | controller | 2026-06-09T20:29:40-04:00 INFO 2026-06-09 20:29:40.831841 | controller | - Config: Waiting ... 2026-06-09 20:29:40.831851 | controller | - Gateway: OK 2026-06-09 20:29:40.831860 | controller | - GitServer: OK 2026-06-09 20:29:40.831868 | controller | - HoundSearch: OK 2026-06-09 20:29:40.831876 | controller | - LogJuicer: OK 2026-06-09 20:29:40.831884 | controller | - Logserver: Waiting ... 2026-06-09 20:29:40.831892 | controller | - MariaDB: OK 2026-06-09 20:29:40.831900 | controller | - NodePoolBuilder: OK 2026-06-09 20:29:40.831908 | controller | - NodePoolLauncher: OK 2026-06-09 20:29:40.831916 | controller | - Zookeeper: OK 2026-06-09 20:29:40.831924 | controller | - Zuul: Waiting ... 2026-06-09 20:29:40.831932 | controller | - |_ Config Secrets: OK 2026-06-09 20:29:40.831940 | controller | - |_ Executor: OK 2026-06-09 20:29:40.831947 | controller | - |_ Merger: OK 2026-06-09 20:29:40.831955 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:29:40.831963 | controller | - |_ Web: Waiting ... 2026-06-09 20:29:40.831971 | controller | 2026-06-09 20:29:40.831982 | controller | 2026-06-09T20:29:40-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:29:45.841262 | controller | 2026-06-09T20:29:45-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:29:45.868267 | controller | 2026-06-09T20:29:45-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:29:45.887698 | controller | 2026-06-09T20:29:45-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:29:46.451697 | controller | 2026-06-09T20:29:46-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:29:46.458509 | controller | 2026-06-09T20:29:46-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:29:46.574475 | controller | 2026-06-09T20:29:46-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:29:46.580877 | controller | 2026-06-09T20:29:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:29:46.580925 | controller | 2026-06-09T20:29:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:29:46.588299 | controller | 2026-06-09T20:29:46-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:29:46.608868 | controller | 2026-06-09T20:29:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:29:46.608952 | controller | 2026-06-09T20:29:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:29:46.855065 | controller | 2026-06-09T20:29:46-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:29:46.858741 | controller | 2026-06-09T20:29:46-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:29:47.260516 | controller | 2026-06-09T20:29:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:29:47.260633 | controller | 2026-06-09T20:29:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:29:47.455318 | controller | 2026-06-09T20:29:47-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:29:48.262798 | controller | 2026-06-09T20:29:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:29:48.262908 | controller | 2026-06-09T20:29:48-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:29:53.273287 | controller | 2026-06-09T20:29:53-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:29:53.273328 | controller | 2026-06-09T20:29:53-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:29:58.285754 | controller | 2026-06-09T20:29:58-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:29:58.285853 | controller | 2026-06-09T20:29:58-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:30:03.296339 | controller | 2026-06-09T20:30:03-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:30:03.296391 | controller | 2026-06-09T20:30:03-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:30:08.305616 | controller | 2026-06-09T20:30:08-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:30:08.305679 | controller | 2026-06-09T20:30:08-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:30:13.314067 | controller | 2026-06-09T20:30:13-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:30:13.314166 | controller | 2026-06-09T20:30:13-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:30:18.319137 | controller | 2026-06-09T20:30:18-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:30:18.336801 | controller | 2026-06-09T20:30:18-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:30:18.352145 | controller | 2026-06-09T20:30:18-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:30:18.352196 | controller | 2026-06-09T20:30:18-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:30:18.731135 | controller | 2026-06-09T20:30:18-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:30:18.734094 | controller | 2026-06-09T20:30:18-04:00 INFO 2026-06-09 20:30:18.734127 | controller | - Config: Waiting ... 2026-06-09 20:30:18.734138 | controller | - Gateway: OK 2026-06-09 20:30:18.734147 | controller | - GitServer: OK 2026-06-09 20:30:18.734155 | controller | - HoundSearch: OK 2026-06-09 20:30:18.734163 | controller | - LogJuicer: OK 2026-06-09 20:30:18.734171 | controller | - Logserver: OK 2026-06-09 20:30:18.734178 | controller | - MariaDB: OK 2026-06-09 20:30:18.734186 | controller | - NodePoolBuilder: OK 2026-06-09 20:30:18.734194 | controller | - NodePoolLauncher: OK 2026-06-09 20:30:18.734202 | controller | - Zookeeper: OK 2026-06-09 20:30:18.734210 | controller | - Zuul: Waiting ... 2026-06-09 20:30:18.734218 | controller | - |_ Config Secrets: OK 2026-06-09 20:30:18.734225 | controller | - |_ Executor: OK 2026-06-09 20:30:18.734268 | controller | - |_ Merger: OK 2026-06-09 20:30:18.734277 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:30:18.734285 | controller | - |_ Web: Waiting ... 2026-06-09 20:30:18.734294 | controller | 2026-06-09 20:30:18.734302 | controller | 2026-06-09T20:30:18-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:30:23.743186 | controller | 2026-06-09T20:30:23-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:30:23.766847 | controller | 2026-06-09T20:30:23-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:30:23.780821 | controller | 2026-06-09T20:30:23-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:30:24.346705 | controller | 2026-06-09T20:30:24-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:30:24.355415 | controller | 2026-06-09T20:30:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:30:24.355446 | controller | 2026-06-09T20:30:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:30:24.364199 | controller | 2026-06-09T20:30:24-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:30:24.373283 | controller | 2026-06-09T20:30:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:30:24.373306 | controller | 2026-06-09T20:30:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:30:24.756689 | controller | 2026-06-09T20:30:24-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:30:24.759595 | controller | 2026-06-09T20:30:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:30:25.162585 | controller | 2026-06-09T20:30:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:30:25.162630 | controller | 2026-06-09T20:30:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:30:25.358837 | controller | 2026-06-09T20:30:25-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:30:26.168607 | controller | 2026-06-09T20:30:26-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:30:26.168706 | controller | 2026-06-09T20:30:26-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:30:31.179381 | controller | 2026-06-09T20:30:31-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:30:31.179486 | controller | 2026-06-09T20:30:31-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:30:36.187920 | controller | 2026-06-09T20:30:36-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:30:36.187997 | controller | 2026-06-09T20:30:36-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:30:41.198645 | controller | 2026-06-09T20:30:41-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:30:41.198736 | controller | 2026-06-09T20:30:41-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:30:46.209215 | controller | 2026-06-09T20:30:46-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:30:46.217144 | controller | 2026-06-09T20:30:46-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:30:46.217181 | controller | 2026-06-09T20:30:46-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:30:46.237830 | controller | 2026-06-09T20:30:46-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:30:46.252269 | controller | 2026-06-09T20:30:46-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:30:46.252312 | controller | 2026-06-09T20:30:46-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:30:46.629072 | controller | 2026-06-09T20:30:46-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:30:47.034960 | controller | 2026-06-09T20:30:47-04:00 INFO 2026-06-09 20:30:47.035042 | controller | - Config: OK 2026-06-09 20:30:47.035059 | controller | - Gateway: OK 2026-06-09 20:30:47.035073 | controller | - GitServer: OK 2026-06-09 20:30:47.035084 | controller | - HoundSearch: OK 2026-06-09 20:30:47.035092 | controller | - LogJuicer: OK 2026-06-09 20:30:47.035101 | controller | - Logserver: OK 2026-06-09 20:30:47.035112 | controller | - MariaDB: OK 2026-06-09 20:30:47.035124 | controller | - NodePoolBuilder: OK 2026-06-09 20:30:47.035137 | controller | - NodePoolLauncher: OK 2026-06-09 20:30:47.035149 | controller | - Zookeeper: OK 2026-06-09 20:30:47.035162 | controller | - Zuul: OK 2026-06-09 20:30:47.035175 | controller | 2026-06-09 20:30:47.035185 | controller | 2026-06-09T20:30:47-04:00 INFO Updating controller configmap ... 2026-06-09 20:30:47.423101 | controller | 2026-06-09T20:30:47-04:00 INFO Standalone reconcile done. 2026-06-10 00:30:47.814433 | controller | ok: Runtime: 0:02:06.559132 2026-06-10 00:30:47.829867 | 2026-06-10 00:30:47.830068 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:30:48.372421 | controller | ok: Runtime: 0:00:00.004795 2026-06-10 00:30:48.388765 | 2026-06-10 00:30:48.388918 | TASK [health-check/test-standalone-update : Get standalone owner configmap after update] 2026-06-09 20:30:48.782345 | controller | fqdn: test-updated.sf.io 2026-06-09 20:30:48.782449 | controller | fluentbitlogforwarding: null 2026-06-09 20:30:48.782464 | controller | storagedefault: 2026-06-09 20:30:48.782473 | controller | classname: "" 2026-06-09 20:30:48.782482 | controller | extraannotations: {} 2026-06-09 20:30:48.782491 | controller | extralabels: {} 2026-06-09 20:30:48.782499 | controller | configrepositorylocation: 2026-06-09 20:30:48.782511 | controller | name: config 2026-06-09 20:30:48.782521 | controller | branch: "" 2026-06-09 20:30:48.782531 | controller | zuulconnectionname: gerrit 2026-06-09 20:30:48.782540 | controller | clusterapiurl: "" 2026-06-09 20:30:48.782550 | controller | logserverhost: "" 2026-06-09 20:30:48.782559 | controller | zuul: 2026-06-09 20:30:48.782566 | controller | oidcauthenticators: [] 2026-06-09 20:30:48.782575 | controller | defaultauthenticator: "" 2026-06-09 20:30:48.782583 | controller | gerritconns: 2026-06-09 20:30:48.782590 | controller | - name: gerrit 2026-06-09 20:30:48.782603 | controller | hostname: gerrit-sshd.sf 2026-06-09 20:30:48.782612 | controller | port: 0 2026-06-09 20:30:48.782623 | controller | puburl: https://gerrit.sfop.me 2026-06-09 20:30:48.782631 | controller | username: zuul 2026-06-09 20:30:48.782639 | controller | canonicalhostname: "" 2026-06-09 20:30:48.782648 | controller | password: "" 2026-06-09 20:30:48.782655 | controller | sshkey: "" 2026-06-09 20:30:48.782663 | controller | gitoverssh: false 2026-06-09 20:30:48.782671 | controller | verifyssl: false 2026-06-09 20:30:48.782678 | controller | streamevents: null 2026-06-09 20:30:48.782686 | controller | githubconns: [] 2026-06-09 20:30:48.782693 | controller | gitlabconns: [] 2026-06-09 20:30:48.782701 | controller | gitconns: [] 2026-06-09 20:30:48.782709 | controller | pagureconns: [] 2026-06-09 20:30:48.782716 | controller | elasticsearchconns: [] 2026-06-09 20:30:48.782724 | controller | smtpconns: [] 2026-06-09 20:30:48.782732 | controller | executor: 2026-06-09 20:30:48.782739 | controller | storage: 2026-06-09 20:30:48.782747 | controller | size: 2026-06-09 20:30:48.782755 | controller | format: "" 2026-06-09 20:30:48.782762 | controller | classname: "" 2026-06-09 20:30:48.782770 | controller | loglevel: DEBUG 2026-06-09 20:30:48.782778 | controller | enabled: true 2026-06-09 20:30:48.782785 | controller | standalone: null 2026-06-09 20:30:48.782793 | controller | limits: null 2026-06-09 20:30:48.782800 | controller | disklimitperjob: 0 2026-06-09 20:30:48.782808 | controller | ansiblesetuptimeout: 0 2026-06-09 20:30:48.782820 | controller | terminationgraceperiodseconds: 0 2026-06-09 20:30:48.782828 | controller | scheduler: 2026-06-09 20:30:48.782835 | controller | storage: 2026-06-09 20:30:48.782843 | controller | size: 2026-06-09 20:30:48.782851 | controller | format: "" 2026-06-09 20:30:48.782858 | controller | classname: "" 2026-06-09 20:30:48.782866 | controller | statsdtarget: "" 2026-06-09 20:30:48.782873 | controller | loglevel: DEBUG 2026-06-09 20:30:48.782881 | controller | limits: null 2026-06-09 20:30:48.782889 | controller | defaultholdexpiration: null 2026-06-09 20:30:48.782896 | controller | maxholdexpiration: null 2026-06-09 20:30:48.782904 | controller | web: 2026-06-09 20:30:48.782912 | controller | loglevel: DEBUG 2026-06-09 20:30:48.782919 | controller | limits: null 2026-06-09 20:30:48.782927 | controller | merger: 2026-06-09 20:30:48.782935 | controller | gitusername: "" 2026-06-09 20:30:48.782943 | controller | gituseremail: "" 2026-06-09 20:30:48.782950 | controller | githttplowspeedlimit: 0 2026-06-09 20:30:48.782958 | controller | githttplowspeedtime: 0 2026-06-09 20:30:48.782966 | controller | gittimeout: 0 2026-06-09 20:30:48.782973 | controller | storage: 2026-06-09 20:30:48.782981 | controller | size: 2026-06-09 20:30:48.782988 | controller | format: "" 2026-06-09 20:30:48.782996 | controller | classname: "" 2026-06-09 20:30:48.783004 | controller | loglevel: DEBUG 2026-06-09 20:30:48.783011 | controller | limits: null 2026-06-09 20:30:48.783019 | controller | nodepool: 2026-06-09 20:30:48.783027 | controller | launcher: 2026-06-09 20:30:48.783034 | controller | loglevel: "" 2026-06-09 20:30:48.783042 | controller | limits: null 2026-06-09 20:30:48.783049 | controller | builder: 2026-06-09 20:30:48.783057 | controller | storage: 2026-06-09 20:30:48.783065 | controller | size: 2026-06-09 20:30:48.783072 | controller | format: "" 2026-06-09 20:30:48.783080 | controller | classname: "" 2026-06-09 20:30:48.783088 | controller | loglevel: "" 2026-06-09 20:30:48.783105 | controller | limits: null 2026-06-09 20:30:48.783113 | controller | statsdtarget: "" 2026-06-09 20:30:48.783120 | controller | zookeeper: 2026-06-09 20:30:48.783128 | controller | storage: 2026-06-09 20:30:48.783136 | controller | size: 2026-06-09 20:30:48.783143 | controller | format: "" 2026-06-09 20:30:48.783151 | controller | classname: "" 2026-06-09 20:30:48.783159 | controller | limits: null 2026-06-09 20:30:48.783166 | controller | logserver: 2026-06-09 20:30:48.783174 | controller | retentiondays: 30 2026-06-09 20:30:48.783182 | controller | loopdelay: 3600 2026-06-09 20:30:48.783189 | controller | storage: 2026-06-09 20:30:48.783197 | controller | size: 2026-06-09 20:30:48.783205 | controller | format: BinarySI 2026-06-09 20:30:48.783213 | controller | classname: "" 2026-06-09 20:30:48.783220 | controller | podannotations: {} 2026-06-09 20:30:48.783252 | controller | logjuicer: 2026-06-09 20:30:48.783265 | controller | size: 2026-06-09 20:30:48.783273 | controller | format: "" 2026-06-09 20:30:48.783281 | controller | classname: "" 2026-06-09 20:30:48.783289 | controller | mariadb: 2026-06-09 20:30:48.783296 | controller | dbstorage: 2026-06-09 20:30:48.783304 | controller | size: 2026-06-09 20:30:48.783312 | controller | format: "" 2026-06-09 20:30:48.783319 | controller | classname: "" 2026-06-09 20:30:48.783327 | controller | logstorage: 2026-06-09 20:30:48.783335 | controller | size: 2026-06-09 20:30:48.783343 | controller | format: "" 2026-06-09 20:30:48.783350 | controller | classname: "" 2026-06-09 20:30:48.783358 | controller | limits: null 2026-06-09 20:30:48.783366 | controller | gitserver: 2026-06-09 20:30:48.783373 | controller | storage: 2026-06-09 20:30:48.783381 | controller | size: 2026-06-09 20:30:48.783389 | controller | format: "" 2026-06-09 20:30:48.783396 | controller | classname: "" 2026-06-09 20:30:48.783404 | controller | codesearch: 2026-06-09 20:30:48.783412 | controller | storage: 2026-06-09 20:30:48.783419 | controller | size: 2026-06-09 20:30:48.783427 | controller | format: "" 2026-06-09 20:30:48.783435 | controller | classname: "" 2026-06-09 20:30:48.783442 | controller | limits: null 2026-06-09 20:30:48.783450 | controller | enabled: null 2026-06-09 20:30:48.783458 | controller | hostaliases: [] 2026-06-09 20:30:48.783465 | controller | gateway: null 2026-06-10 00:30:48.937941 | controller | ok: Runtime: 0:00:00.095279 2026-06-10 00:30:48.950650 | 2026-06-10 00:30:48.950828 | TASK [health-check/test-standalone-update : Verify updated spec in configmap] 2026-06-10 00:30:48.991266 | controller | ok: All assertions passed 2026-06-10 00:30:49.002120 | 2026-06-10 00:30:49.002283 | TASK [health-check/test-standalone-update : Get standalone owner configmap annotation after update] 2026-06-10 00:30:49.537272 | controller | ok: Runtime: 0:00:00.066324 2026-06-10 00:30:49.545176 | 2026-06-10 00:30:49.545319 | TASK [health-check/test-standalone-update : Verify that the last-reconcile annotation has changed] 2026-06-10 00:30:49.584750 | controller | ok: All assertions passed 2026-06-10 00:30:49.598452 | 2026-06-10 00:30:49.598756 | TASK [Restore original state] 2026-06-10 00:30:49.643226 | controller | ok 2026-06-10 00:30:49.674471 | 2026-06-10 00:30:49.674794 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:30:49.947634 | controller | ok 2026-06-10 00:30:49.957176 | 2026-06-10 00:30:49.957376 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:30:50.165168 | controller | changed 2026-06-10 00:30:50.174515 | 2026-06-10 00:30:50.174707 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:30:50.762257 | controller | 2026-06-09T20:30:50-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:30:50.762355 | controller | 2026-06-09 20:30:50.772503 | controller | 2026-06-09T20:30:50-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:30:50.772559 | controller | 2026-06-09 20:30:50.786802 | controller | 2026-06-09T20:30:50-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:30:50.795740 | controller | 2026-06-09T20:30:50-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-09 20:30:50.795787 | controller | 2026-06-09T20:30:50-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-09 20:30:50.809461 | controller | 2026-06-09T20:30:50-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-09 20:30:50.809525 | controller | 2026-06-09T20:30:50-04:00 INFO Updating object, name: git-server 2026-06-09 20:30:50.831353 | controller | 2026-06-09T20:30:50-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:30:51.391923 | controller | 2026-06-09T20:30:51-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-09 20:30:51.391995 | controller | 2026-06-09T20:30:51-04:00 INFO Updating object, name: logserver 2026-06-09 20:30:51.409410 | controller | 2026-06-09T20:30:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:30:51.409469 | controller | 2026-06-09T20:30:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:30:51.416326 | controller | 2026-06-09T20:30:51-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:30:51.437934 | controller | 2026-06-09T20:30:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:30:51.437994 | controller | 2026-06-09T20:30:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:30:51.796238 | controller | 2026-06-09T20:30:51-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:30:51.798724 | controller | 2026-06-09T20:30:51-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 20:30:51.798749 | controller | 2026-06-09T20:30:51-04:00 INFO 2026-06-09 20:30:51.798755 | controller | - Config: Waiting ... 2026-06-09 20:30:51.798759 | controller | - Gateway: OK 2026-06-09 20:30:51.798763 | controller | - GitServer: Waiting ... 2026-06-09 20:30:51.798767 | controller | - HoundSearch: OK 2026-06-09 20:30:51.798771 | controller | - LogJuicer: OK 2026-06-09 20:30:51.798774 | controller | - Logserver: Waiting ... 2026-06-09 20:30:51.798778 | controller | - MariaDB: OK 2026-06-09 20:30:51.798782 | controller | - Zookeeper: OK 2026-06-09 20:30:51.798785 | controller | 2026-06-09 20:30:51.798789 | controller | 2026-06-09T20:30:51-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:30:56.807904 | controller | 2026-06-09T20:30:56-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:30:56.831937 | controller | 2026-06-09T20:30:56-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:30:56.832006 | controller | 2026-06-09T20:30:56-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:31:01.841219 | controller | 2026-06-09T20:31:01-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:31:01.841293 | controller | 2026-06-09T20:31:01-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:31:06.850179 | controller | 2026-06-09T20:31:06-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:31:06.850275 | controller | 2026-06-09T20:31:06-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:31:11.857545 | controller | 2026-06-09T20:31:11-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:31:11.857625 | controller | 2026-06-09T20:31:11-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:31:16.865419 | controller | 2026-06-09T20:31:16-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:31:16.885300 | controller | 2026-06-09T20:31:16-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:31:16.917641 | controller | 2026-06-09T20:31:16-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:31:16.917738 | controller | 2026-06-09T20:31:16-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:31:21.927052 | controller | 2026-06-09T20:31:21-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:31:21.927416 | controller | 2026-06-09T20:31:21-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:31:26.934524 | controller | 2026-06-09T20:31:26-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:31:26.934617 | controller | 2026-06-09T20:31:26-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:31:31.946501 | controller | 2026-06-09T20:31:31-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:31:31.946580 | controller | 2026-06-09T20:31:31-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:31:36.960439 | controller | 2026-06-09T20:31:36-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:31:36.960572 | controller | 2026-06-09T20:31:36-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:31:41.975705 | controller | 2026-06-09T20:31:41-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:31:41.976392 | controller | 2026-06-09T20:31:41-04:00 INFO statefulset rollout not ok, name: logserver 2026-06-09 20:31:46.980388 | controller | 2026-06-09T20:31:46-04:00 INFO Statefulset "logserver" not ready after 30s 2026-06-09 20:31:46.986997 | controller | 2026-06-09T20:31:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:31:46.987064 | controller | 2026-06-09T20:31:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:31:46.994488 | controller | 2026-06-09T20:31:46-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:31:47.006479 | controller | 2026-06-09T20:31:47-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:31:47.006523 | controller | 2026-06-09T20:31:47-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:31:47.024860 | controller | 2026-06-09T20:31:47-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:31:47.028156 | controller | 2026-06-09T20:31:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:31:47.053311 | controller | 2026-06-09T20:31:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:31:47.053359 | controller | 2026-06-09T20:31:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:31:47.196865 | controller | 2026-06-09T20:31:47-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:31:47.210131 | controller | 2026-06-09T20:31:47-04:00 INFO Updating secret, name: zuul-config 2026-06-09 20:31:47.210169 | controller | 2026-06-09T20:31:47-04:00 INFO Updating object, name: zuul-config 2026-06-09 20:31:47.990125 | controller | 2026-06-09T20:31:47-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 20:31:47.990224 | controller | 2026-06-09T20:31:47-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 20:31:48.795629 | controller | 2026-06-09T20:31:48-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:31:49.589738 | controller | 2026-06-09T20:31:49-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 20:31:49.589881 | controller | 2026-06-09T20:31:49-04:00 INFO Updating object, name: zuul-web 2026-06-09 20:31:50.394750 | controller | 2026-06-09T20:31:50-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:31:50.397114 | controller | 2026-06-09T20:31:50-04:00 INFO 2026-06-09 20:31:50.397133 | controller | - Config: Waiting ... 2026-06-09 20:31:50.397140 | controller | - Gateway: OK 2026-06-09 20:31:50.397145 | controller | - GitServer: OK 2026-06-09 20:31:50.397150 | controller | - HoundSearch: OK 2026-06-09 20:31:50.397155 | controller | - LogJuicer: OK 2026-06-09 20:31:50.397160 | controller | - Logserver: Waiting ... 2026-06-09 20:31:50.397165 | controller | - MariaDB: OK 2026-06-09 20:31:50.397170 | controller | - NodePoolBuilder: OK 2026-06-09 20:31:50.397175 | controller | - NodePoolLauncher: OK 2026-06-09 20:31:50.397180 | controller | - Zookeeper: OK 2026-06-09 20:31:50.397185 | controller | - Zuul: Waiting ... 2026-06-09 20:31:50.397190 | controller | - |_ Config Secrets: OK 2026-06-09 20:31:50.397195 | controller | - |_ Executor: OK 2026-06-09 20:31:50.397200 | controller | - |_ Merger: OK 2026-06-09 20:31:50.397205 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:31:50.397210 | controller | - |_ Web: Waiting ... 2026-06-09 20:31:50.397215 | controller | 2026-06-09 20:31:50.397222 | controller | 2026-06-09T20:31:50-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:31:55.405476 | controller | 2026-06-09T20:31:55-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:31:55.429995 | controller | 2026-06-09T20:31:55-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:31:55.444441 | controller | 2026-06-09T20:31:55-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:31:56.015521 | controller | 2026-06-09T20:31:56-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:31:56.020545 | controller | 2026-06-09T20:31:56-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:31:56.167695 | controller | 2026-06-09T20:31:56-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:31:56.175863 | controller | 2026-06-09T20:31:56-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:31:56.176200 | controller | 2026-06-09T20:31:56-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:31:56.188119 | controller | 2026-06-09T20:31:56-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:31:56.202916 | controller | 2026-06-09T20:31:56-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:31:56.202957 | controller | 2026-06-09T20:31:56-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:31:56.415447 | controller | 2026-06-09T20:31:56-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:31:56.418221 | controller | 2026-06-09T20:31:56-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:31:56.813112 | controller | 2026-06-09T20:31:56-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:31:56.813152 | controller | 2026-06-09T20:31:56-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:31:57.015560 | controller | 2026-06-09T20:31:57-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:31:57.833555 | controller | 2026-06-09T20:31:57-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:31:57.833608 | controller | 2026-06-09T20:31:57-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:02.843844 | controller | 2026-06-09T20:32:02-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:02.843948 | controller | 2026-06-09T20:32:02-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:07.855952 | controller | 2026-06-09T20:32:07-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:07.856002 | controller | 2026-06-09T20:32:07-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:12.867576 | controller | 2026-06-09T20:32:12-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:12.867684 | controller | 2026-06-09T20:32:12-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:17.875635 | controller | 2026-06-09T20:32:17-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:17.875685 | controller | 2026-06-09T20:32:17-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:22.887948 | controller | 2026-06-09T20:32:22-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:22.888038 | controller | 2026-06-09T20:32:22-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:27.894708 | controller | 2026-06-09T20:32:27-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:32:27.923266 | controller | 2026-06-09T20:32:27-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:32:27.943018 | controller | 2026-06-09T20:32:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:32:27.943105 | controller | 2026-06-09T20:32:27-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:32:28.305968 | controller | 2026-06-09T20:32:28-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:32:28.308404 | controller | 2026-06-09T20:32:28-04:00 INFO 2026-06-09 20:32:28.308453 | controller | - Config: Waiting ... 2026-06-09 20:32:28.308469 | controller | - Gateway: OK 2026-06-09 20:32:28.308483 | controller | - GitServer: OK 2026-06-09 20:32:28.308497 | controller | - HoundSearch: OK 2026-06-09 20:32:28.308510 | controller | - LogJuicer: OK 2026-06-09 20:32:28.308522 | controller | - Logserver: OK 2026-06-09 20:32:28.308534 | controller | - MariaDB: OK 2026-06-09 20:32:28.308546 | controller | - NodePoolBuilder: OK 2026-06-09 20:32:28.308559 | controller | - NodePoolLauncher: OK 2026-06-09 20:32:28.308567 | controller | - Zookeeper: OK 2026-06-09 20:32:28.308576 | controller | - Zuul: Waiting ... 2026-06-09 20:32:28.308584 | controller | - |_ Config Secrets: OK 2026-06-09 20:32:28.308593 | controller | - |_ Executor: OK 2026-06-09 20:32:28.308601 | controller | - |_ Merger: OK 2026-06-09 20:32:28.308610 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:32:28.308619 | controller | - |_ Web: Waiting ... 2026-06-09 20:32:28.308628 | controller | 2026-06-09 20:32:28.308643 | controller | 2026-06-09T20:32:28-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:32:33.315624 | controller | 2026-06-09T20:32:33-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:32:33.340166 | controller | 2026-06-09T20:32:33-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:32:33.358357 | controller | 2026-06-09T20:32:33-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:32:33.923865 | controller | 2026-06-09T20:32:33-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:32:33.934595 | controller | 2026-06-09T20:32:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:32:33.934657 | controller | 2026-06-09T20:32:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:32:33.945034 | controller | 2026-06-09T20:32:33-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:32:33.961051 | controller | 2026-06-09T20:32:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:32:33.961116 | controller | 2026-06-09T20:32:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:32:34.331216 | controller | 2026-06-09T20:32:34-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:32:34.334655 | controller | 2026-06-09T20:32:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:32:34.733773 | controller | 2026-06-09T20:32:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:32:34.733837 | controller | 2026-06-09T20:32:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:32:34.926776 | controller | 2026-06-09T20:32:34-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:32:35.737445 | controller | 2026-06-09T20:32:35-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:35.737571 | controller | 2026-06-09T20:32:35-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:40.750571 | controller | 2026-06-09T20:32:40-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:40.750639 | controller | 2026-06-09T20:32:40-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:45.764179 | controller | 2026-06-09T20:32:45-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:45.764291 | controller | 2026-06-09T20:32:45-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:50.775344 | controller | 2026-06-09T20:32:50-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:50.775399 | controller | 2026-06-09T20:32:50-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:32:55.786199 | controller | 2026-06-09T20:32:55-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:32:55.795766 | controller | 2026-06-09T20:32:55-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:32:55.795822 | controller | 2026-06-09T20:32:55-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:32:55.816396 | controller | 2026-06-09T20:32:55-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:32:55.828920 | controller | 2026-06-09T20:32:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:32:55.828958 | controller | 2026-06-09T20:32:55-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:32:56.206401 | controller | 2026-06-09T20:32:56-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:32:56.607262 | controller | 2026-06-09T20:32:56-04:00 INFO 2026-06-09 20:32:56.607330 | controller | - Config: OK 2026-06-09 20:32:56.607338 | controller | - Gateway: OK 2026-06-09 20:32:56.607344 | controller | - GitServer: OK 2026-06-09 20:32:56.607350 | controller | - HoundSearch: OK 2026-06-09 20:32:56.607355 | controller | - LogJuicer: OK 2026-06-09 20:32:56.607361 | controller | - Logserver: OK 2026-06-09 20:32:56.607369 | controller | - MariaDB: OK 2026-06-09 20:32:56.607376 | controller | - NodePoolBuilder: OK 2026-06-09 20:32:56.607384 | controller | - NodePoolLauncher: OK 2026-06-09 20:32:56.607392 | controller | - Zookeeper: OK 2026-06-09 20:32:56.607399 | controller | - Zuul: OK 2026-06-09 20:32:56.607406 | controller | 2026-06-09 20:32:56.607414 | controller | 2026-06-09T20:32:56-04:00 INFO Updating controller configmap ... 2026-06-09 20:32:56.999042 | controller | 2026-06-09T20:32:56-04:00 INFO Standalone reconcile done. 2026-06-10 00:32:57.400214 | controller | ok: Runtime: 0:02:06.531569 2026-06-10 00:32:57.412027 | 2026-06-10 00:32:57.412172 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:32:57.957570 | controller | ok: Runtime: 0:00:00.004710 2026-06-10 00:32:57.975655 | 2026-06-10 00:32:57.975818 | TASK [health-check/test-standalone-update : Remove temporary directory] 2026-06-10 00:32:58.255071 | controller | changed 2026-06-10 00:32:58.274642 | 2026-06-10 00:32:58.274798 | TASK [health-check/check-zuul-weeder : Get zuul-weeder status] 2026-06-10 00:32:59.060339 | controller | ok: OK (unknown bytes) 2026-06-10 00:32:59.115784 | 2026-06-10 00:32:59.115918 | TASK [scale down zuul components] 2026-06-10 00:32:59.145436 | controller | ok 2026-06-10 00:32:59.161644 | 2026-06-10 00:32:59.161756 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 00:32:59.190452 | controller | ok: 2026-06-10 00:32:59.190783 | controller | { 2026-06-10 00:32:59.190824 | controller | "kind": "sts", 2026-06-10 00:32:59.190851 | controller | "name": "zuul-executor", 2026-06-10 00:32:59.190874 | controller | "replicas": 0 2026-06-10 00:32:59.190897 | controller | } 2026-06-10 00:32:59.190964 | controller | changed: All items complete 2026-06-10 00:32:59.190991 | 2026-06-10 00:32:59.195348 | controller | ok: 2026-06-10 00:32:59.195409 | controller | { 2026-06-10 00:32:59.195440 | controller | "kind": "sts", 2026-06-10 00:32:59.195464 | controller | "name": "zuul-merger", 2026-06-10 00:32:59.195487 | controller | "replicas": 0 2026-06-10 00:32:59.195509 | controller | } 2026-06-10 00:32:59.199536 | controller | ok: 2026-06-10 00:32:59.199628 | controller | { 2026-06-10 00:32:59.199681 | controller | "kind": "sts", 2026-06-10 00:32:59.199727 | controller | "name": "zuul-scheduler", 2026-06-10 00:32:59.199770 | controller | "replicas": 0 2026-06-10 00:32:59.199811 | controller | } 2026-06-10 00:32:59.204022 | controller | ok: 2026-06-10 00:32:59.204082 | controller | { 2026-06-10 00:32:59.204115 | controller | "kind": "deployment", 2026-06-10 00:32:59.204140 | controller | "name": "zuul-web", 2026-06-10 00:32:59.204166 | controller | "replicas": 0 2026-06-10 00:32:59.204189 | controller | } 2026-06-10 00:32:59.284894 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:32:59.287830 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:32:59.290490 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:32:59.293234 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:32:59.298297 | 2026-06-10 00:32:59.298358 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 00:32:59.828288 | controller | ok: Runtime: 0:00:00.077841 2026-06-10 00:32:59.841067 | 2026-06-10 00:32:59.841274 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-09 20:33:00.241625 | controller | Defaulted container "zuul-executor" out of: zuul-executor, zuul-executor-nodeexporter 2026-06-10 00:33:00.384924 | controller | ok: Runtime: 0:00:00.135596 2026-06-10 00:33:00.393109 | 2026-06-10 00:33:00.393345 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-09 20:33:00.757797 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 00:33:00.938157 | controller | ok: Runtime: 0:00:00.102950 2026-06-10 00:33:00.950950 | 2026-06-10 00:33:00.951098 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 00:33:11.687984 | controller | ok: Runtime: 0:00:00.086636 2026-06-10 00:33:11.698142 | 2026-06-10 00:33:11.698276 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 00:33:12.239223 | controller | ok: Runtime: 0:00:00.067358 2026-06-10 00:33:12.247292 | 2026-06-10 00:33:12.247404 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-09 20:33:12.539090 | controller | Defaulted container "zuul-merger" out of: zuul-merger, zuul-merger-nodeexporter 2026-06-10 00:33:12.785740 | controller | ok: Runtime: 0:00:00.078915 2026-06-10 00:33:12.800559 | 2026-06-10 00:33:12.800774 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-09 20:33:13.184714 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 00:33:13.345418 | controller | ok: Runtime: 0:00:00.096345 2026-06-10 00:33:13.357322 | 2026-06-10 00:33:13.357493 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 00:33:24.025407 | controller | ok: Runtime: 0:00:00.084465 2026-06-10 00:33:24.037506 | 2026-06-10 00:33:24.037695 | TASK [health-check/scale-resources : Get initial replica count for zuul-scheduler] 2026-06-10 00:33:24.581055 | controller | ok: Runtime: 0:00:00.095356 2026-06-10 00:33:24.592697 | 2026-06-10 00:33:24.592876 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-09 20:33:24.979340 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 00:33:25.140694 | controller | ok: Runtime: 0:00:00.126986 2026-06-10 00:33:25.152733 | 2026-06-10 00:33:25.152935 | TASK [health-check/scale-resources : Scale on sts zuul-scheduler] 2026-06-09 20:33:25.585854 | controller | statefulset.apps/zuul-scheduler scaled 2026-06-10 00:33:25.744943 | controller | ok: Runtime: 0:00:00.108582 2026-06-10 00:33:25.759265 | 2026-06-10 00:33:25.759533 | TASK [health-check/scale-resources : Get replica count for zuul-scheduler] 2026-06-10 00:33:36.437649 | controller | ok: Runtime: 0:00:00.085414 2026-06-10 00:33:36.450897 | 2026-06-10 00:33:36.451128 | TASK [health-check/scale-resources : Get initial replica count for zuul-web] 2026-06-10 00:33:36.999657 | controller | ok: Runtime: 0:00:00.065356 2026-06-10 00:33:37.015783 | 2026-06-10 00:33:37.015977 | TASK [health-check/scale-resources : Collect deployment log before executing the scale command] 2026-06-10 00:33:37.560042 | controller | ok: Runtime: 0:00:00.084552 2026-06-10 00:33:37.571854 | 2026-06-10 00:33:37.572018 | TASK [health-check/scale-resources : Scale on deployment zuul-web] 2026-06-09 20:33:37.855811 | controller | deployment.apps/zuul-web scaled 2026-06-10 00:33:38.123799 | controller | ok: Runtime: 0:00:00.057511 2026-06-10 00:33:38.137402 | 2026-06-10 00:33:38.137612 | TASK [health-check/scale-resources : Get replica count for zuul-web] 2026-06-10 00:33:38.685762 | controller | ok: Runtime: 0:00:00.094283 2026-06-10 00:33:38.703404 | 2026-06-10 00:33:38.703622 | TASK [Enable log forwarding] 2026-06-10 00:33:38.738528 | controller | ok 2026-06-10 00:33:38.770662 | 2026-06-10 00:33:38.770777 | TASK [update-custom-resource : read current crd] 2026-06-10 00:33:39.222915 | controller | ok 2026-06-10 00:33:39.232752 | 2026-06-10 00:33:39.232928 | TASK [update-custom-resource : merge the new cr] 2026-06-10 00:33:39.928522 | controller | changed 2026-06-10 00:33:39.938795 | 2026-06-10 00:33:39.939019 | TASK [run the operator in standalone mode] 2026-06-10 00:33:39.972204 | controller | ok 2026-06-10 00:33:39.995891 | 2026-06-10 00:33:39.996074 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:33:40.023199 | controller | skipping: Conditional result was False 2026-06-10 00:33:40.032902 | 2026-06-10 00:33:40.033038 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:33:40.059016 | controller | skipping: Conditional result was False 2026-06-10 00:33:40.067003 | 2026-06-10 00:33:40.067121 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:33:40.527138 | controller | 2026-06-09T20:33:40-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:33:40.527222 | controller | 2026-06-09 20:33:40.535275 | controller | 2026-06-09T20:33:40-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:33:40.535316 | controller | 2026-06-09 20:33:40.547448 | controller | 2026-06-09T20:33:40-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:33:40.564876 | controller | 2026-06-09T20:33:40-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:33:40.571568 | controller | 2026-06-09T20:33:40-04:00 INFO Creating config map name: fluentbit-mariadb-cfg-config-map 2026-06-09 20:33:40.571616 | controller | 2026-06-09T20:33:40-04:00 INFO Creating object, name: fluentbit-mariadb-cfg-config-map 2026-06-09 20:33:40.575412 | controller | 2026-06-09T20:33:40-04:00 INFO Statefulset "mariadb" configuration changed, applying... 2026-06-09 20:33:40.575443 | controller | 2026-06-09T20:33:40-04:00 INFO Updating object, name: mariadb 2026-06-09 20:33:40.956308 | controller | 2026-06-09T20:33:40-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:33:40.961319 | controller | 2026-06-09T20:33:40-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:33:41.045450 | controller | 2026-06-09T20:33:41-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:33:41.050092 | controller | 2026-06-09T20:33:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:33:41.050114 | controller | 2026-06-09T20:33:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:33:41.055325 | controller | 2026-06-09T20:33:41-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:33:41.064483 | controller | 2026-06-09T20:33:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:33:41.064497 | controller | 2026-06-09T20:33:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:33:41.358566 | controller | 2026-06-09T20:33:41-04:00 INFO Creating config map name: fluentbit-zk-cfg-config-map 2026-06-09 20:33:41.358668 | controller | 2026-06-09T20:33:41-04:00 INFO Creating object, name: fluentbit-zk-cfg-config-map 2026-06-09 20:33:41.365461 | controller | 2026-06-09T20:33:41-04:00 INFO Statefulset "zookeeper" configuration changed, applying... 2026-06-09 20:33:41.365507 | controller | 2026-06-09T20:33:41-04:00 INFO Updating object, name: zookeeper 2026-06-09 20:33:41.372731 | controller | 2026-06-09T20:33:41-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-09 20:33:41.372774 | controller | 2026-06-09T20:33:41-04:00 INFO 2026-06-09 20:33:41.372786 | controller | - Config: Waiting ... 2026-06-09 20:33:41.372794 | controller | - Gateway: OK 2026-06-09 20:33:41.372803 | controller | - GitServer: OK 2026-06-09 20:33:41.372811 | controller | - HoundSearch: OK 2026-06-09 20:33:41.372819 | controller | - LogJuicer: OK 2026-06-09 20:33:41.372828 | controller | - Logserver: OK 2026-06-09 20:33:41.372836 | controller | - MariaDB: Waiting ... 2026-06-09 20:33:41.372844 | controller | - Zookeeper: Waiting ... 2026-06-09 20:33:41.372852 | controller | 2026-06-09 20:33:41.372876 | controller | 2026-06-09T20:33:41-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:33:46.380106 | controller | 2026-06-09T20:33:46-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:33:46.405879 | controller | 2026-06-09T20:33:46-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:33:46.416287 | controller | 2026-06-09T20:33:46-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:33:46.416359 | controller | 2026-06-09T20:33:46-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 20:33:51.422633 | controller | 2026-06-09T20:33:51-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:33:51.466419 | controller | 2026-06-09T20:33:51-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:33:51.478059 | controller | 2026-06-09T20:33:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:33:51.478154 | controller | 2026-06-09T20:33:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:33:51.489783 | controller | 2026-06-09T20:33:51-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:33:51.504718 | controller | 2026-06-09T20:33:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:33:51.504790 | controller | 2026-06-09T20:33:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:33:51.534997 | controller | 2026-06-09T20:33:51-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:33:51.535133 | controller | 2026-06-09T20:33:51-04:00 INFO statefulset rollout not ok, name: zookeeper 2026-06-09 20:33:56.546987 | controller | 2026-06-09T20:33:56-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:33:56.551037 | controller | 2026-06-09T20:33:56-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:33:56.564746 | controller | 2026-06-09T20:33:56-04:00 INFO Updating configmap, name: nodepool-launcher-extra-config-config-map 2026-06-09 20:33:56.564819 | controller | 2026-06-09T20:33:56-04:00 INFO Updating object, name: nodepool-launcher-extra-config-config-map 2026-06-09 20:33:56.581193 | controller | 2026-06-09T20:33:56-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-09 20:33:56.581314 | controller | 2026-06-09T20:33:56-04:00 INFO Updating object, name: nodepool-launcher 2026-06-09 20:33:56.603468 | controller | 2026-06-09T20:33:56-04:00 INFO Updating configmap, name: nodepool-builder-extra-config-config-map 2026-06-09 20:33:56.603540 | controller | 2026-06-09T20:33:56-04:00 INFO Updating object, name: nodepool-builder-extra-config-config-map 2026-06-09 20:33:56.618591 | controller | 2026-06-09T20:33:56-04:00 INFO Creating config map name: fluentbit-dib-cfg-config-map 2026-06-09 20:33:56.618705 | controller | 2026-06-09T20:33:56-04:00 INFO Creating object, name: fluentbit-dib-cfg-config-map 2026-06-09 20:33:56.769885 | controller | 2026-06-09T20:33:56-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-09 20:33:56.770004 | controller | 2026-06-09T20:33:56-04:00 INFO Updating object, name: nodepool-builder 2026-06-09 20:33:57.559848 | controller | 2026-06-09T20:33:57-04:00 INFO Updating configmap, name: zuul-logging-config-map 2026-06-09 20:33:57.559943 | controller | 2026-06-09T20:33:57-04:00 INFO Updating object, name: zuul-logging-config-map 2026-06-09 20:33:57.763696 | controller | 2026-06-09T20:33:57-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 20:33:57.763792 | controller | 2026-06-09T20:33:57-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 20:33:58.562528 | controller | 2026-06-09T20:33:58-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-09 20:33:58.562588 | controller | 2026-06-09T20:33:58-04:00 INFO Updating object, name: zuul-executor 2026-06-09 20:33:59.364566 | controller | 2026-06-09T20:33:59-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 20:33:59.364664 | controller | 2026-06-09T20:33:59-04:00 INFO Updating object, name: zuul-web 2026-06-09 20:34:00.162686 | controller | 2026-06-09T20:34:00-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-09 20:34:00.162778 | controller | 2026-06-09T20:34:00-04:00 INFO Updating object, name: zuul-merger 2026-06-09 20:34:00.169841 | controller | 2026-06-09T20:34:00-04:00 INFO 2026-06-09 20:34:00.169886 | controller | - Config: Waiting ... 2026-06-09 20:34:00.169893 | controller | - Gateway: OK 2026-06-09 20:34:00.169899 | controller | - GitServer: OK 2026-06-09 20:34:00.169904 | controller | - HoundSearch: OK 2026-06-09 20:34:00.169910 | controller | - LogJuicer: OK 2026-06-09 20:34:00.169915 | controller | - Logserver: OK 2026-06-09 20:34:00.169920 | controller | - MariaDB: OK 2026-06-09 20:34:00.169925 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:34:00.169931 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:34:00.169936 | controller | - Zookeeper: OK 2026-06-09 20:34:00.169942 | controller | - Zuul: Waiting ... 2026-06-09 20:34:00.169947 | controller | - |_ Config Secrets: OK 2026-06-09 20:34:00.169953 | controller | - |_ Executor: Waiting ... 2026-06-09 20:34:00.169958 | controller | - |_ Merger: Waiting ... 2026-06-09 20:34:00.169964 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:34:00.169969 | controller | - |_ Web: Waiting ... 2026-06-09 20:34:00.169974 | controller | 2026-06-09 20:34:00.170025 | controller | 2026-06-09T20:34:00-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:34:05.177938 | controller | 2026-06-09T20:34:05-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:34:05.198550 | controller | 2026-06-09T20:34:05-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:34:05.211643 | controller | 2026-06-09T20:34:05-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:34:05.783701 | controller | 2026-06-09T20:34:05-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:34:05.793223 | controller | 2026-06-09T20:34:05-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:34:05.793377 | controller | 2026-06-09T20:34:05-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:34:05.801644 | controller | 2026-06-09T20:34:05-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:34:05.813202 | controller | 2026-06-09T20:34:05-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:34:05.813274 | controller | 2026-06-09T20:34:05-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:34:06.192854 | controller | 2026-06-09T20:34:06-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:34:06.195633 | controller | 2026-06-09T20:34:06-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:34:06.597217 | controller | 2026-06-09T20:34:06-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:34:06.597335 | controller | 2026-06-09T20:34:06-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:34:06.795740 | controller | 2026-06-09T20:34:06-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:34:06.795822 | controller | 2026-06-09T20:34:06-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:34:07.601176 | controller | 2026-06-09T20:34:07-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:34:07.605732 | controller | 2026-06-09T20:34:07-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:34:07.605770 | controller | 2026-06-09T20:34:07-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:34:08.386773 | controller | 2026-06-09T20:34:08-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:34:09.183314 | controller | 2026-06-09T20:34:09-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:34:09.984514 | controller | 2026-06-09T20:34:09-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:34:10.387319 | controller | 2026-06-09T20:34:10-04:00 INFO 2026-06-09 20:34:10.387427 | controller | - Config: OK 2026-06-09 20:34:10.387439 | controller | - Gateway: OK 2026-06-09 20:34:10.387450 | controller | - GitServer: OK 2026-06-09 20:34:10.387459 | controller | - HoundSearch: OK 2026-06-09 20:34:10.387467 | controller | - LogJuicer: OK 2026-06-09 20:34:10.387476 | controller | - Logserver: OK 2026-06-09 20:34:10.387485 | controller | - MariaDB: OK 2026-06-09 20:34:10.387494 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:34:10.387503 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:34:10.387511 | controller | - Zookeeper: OK 2026-06-09 20:34:10.387520 | controller | - Zuul: OK 2026-06-09 20:34:10.387529 | controller | 2026-06-09 20:34:10.387539 | controller | 2026-06-09T20:34:10-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:34:15.395491 | controller | 2026-06-09T20:34:15-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:34:15.418313 | controller | 2026-06-09T20:34:15-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:34:15.434201 | controller | 2026-06-09T20:34:15-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:34:16.005766 | controller | 2026-06-09T20:34:16-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:34:16.019695 | controller | 2026-06-09T20:34:16-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:34:16.019751 | controller | 2026-06-09T20:34:16-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:34:16.030358 | controller | 2026-06-09T20:34:16-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:34:16.045393 | controller | 2026-06-09T20:34:16-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:34:16.045461 | controller | 2026-06-09T20:34:16-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:34:16.412275 | controller | 2026-06-09T20:34:16-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:34:16.416201 | controller | 2026-06-09T20:34:16-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:34:16.823106 | controller | 2026-06-09T20:34:16-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:34:16.823175 | controller | 2026-06-09T20:34:16-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:34:17.019940 | controller | 2026-06-09T20:34:17-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:34:17.020061 | controller | 2026-06-09T20:34:17-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:34:17.824222 | controller | 2026-06-09T20:34:17-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:34:17.827950 | controller | 2026-06-09T20:34:17-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:34:17.827972 | controller | 2026-06-09T20:34:17-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:34:18.611951 | controller | 2026-06-09T20:34:18-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:34:19.400181 | controller | 2026-06-09T20:34:19-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:34:20.206183 | controller | 2026-06-09T20:34:20-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:34:20.604874 | controller | 2026-06-09T20:34:20-04:00 INFO 2026-06-09 20:34:20.604926 | controller | - Config: OK 2026-06-09 20:34:20.604931 | controller | - Gateway: OK 2026-06-09 20:34:20.604935 | controller | - GitServer: OK 2026-06-09 20:34:20.604938 | controller | - HoundSearch: OK 2026-06-09 20:34:20.604942 | controller | - LogJuicer: OK 2026-06-09 20:34:20.604945 | controller | - Logserver: OK 2026-06-09 20:34:20.604949 | controller | - MariaDB: OK 2026-06-09 20:34:20.604952 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:34:20.604956 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:34:20.604960 | controller | - Zookeeper: OK 2026-06-09 20:34:20.604963 | controller | - Zuul: OK 2026-06-09 20:34:20.604967 | controller | 2026-06-09 20:34:20.604998 | controller | 2026-06-09T20:34:20-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-09 20:34:25.613791 | controller | 2026-06-09T20:34:25-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:34:25.640181 | controller | 2026-06-09T20:34:25-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:34:25.659897 | controller | 2026-06-09T20:34:25-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:34:26.216583 | controller | 2026-06-09T20:34:26-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:34:26.225354 | controller | 2026-06-09T20:34:26-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:34:26.225412 | controller | 2026-06-09T20:34:26-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:34:26.233514 | controller | 2026-06-09T20:34:26-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:34:26.242859 | controller | 2026-06-09T20:34:26-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:34:26.242923 | controller | 2026-06-09T20:34:26-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:34:26.624703 | controller | 2026-06-09T20:34:26-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:34:26.626622 | controller | 2026-06-09T20:34:26-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:34:27.031953 | controller | 2026-06-09T20:34:27-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:34:27.032039 | controller | 2026-06-09T20:34:27-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:34:27.235794 | controller | 2026-06-09T20:34:27-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:34:27.235923 | controller | 2026-06-09T20:34:27-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:34:28.034819 | controller | 2026-06-09T20:34:28-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:34:28.039945 | controller | 2026-06-09T20:34:28-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:34:28.039985 | controller | 2026-06-09T20:34:28-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:34:28.823397 | controller | 2026-06-09T20:34:28-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:34:29.618510 | controller | 2026-06-09T20:34:29-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:34:30.423157 | controller | 2026-06-09T20:34:30-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:34:30.826415 | controller | 2026-06-09T20:34:30-04:00 INFO 2026-06-09 20:34:30.826486 | controller | - Config: OK 2026-06-09 20:34:30.826497 | controller | - Gateway: OK 2026-06-09 20:34:30.826505 | controller | - GitServer: OK 2026-06-09 20:34:30.826513 | controller | - HoundSearch: OK 2026-06-09 20:34:30.826521 | controller | - LogJuicer: OK 2026-06-09 20:34:30.826529 | controller | - Logserver: OK 2026-06-09 20:34:30.826537 | controller | - MariaDB: OK 2026-06-09 20:34:30.826545 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:34:30.826553 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:34:30.826561 | controller | - Zookeeper: OK 2026-06-09 20:34:30.826569 | controller | - Zuul: OK 2026-06-09 20:34:30.826596 | controller | 2026-06-09 20:34:30.826605 | controller | 2026-06-09T20:34:30-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-09 20:34:35.834541 | controller | 2026-06-09T20:34:35-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:34:35.865669 | controller | 2026-06-09T20:34:35-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:34:35.891310 | controller | 2026-06-09T20:34:35-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:34:36.442694 | controller | 2026-06-09T20:34:36-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:34:36.454855 | controller | 2026-06-09T20:34:36-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:34:36.454931 | controller | 2026-06-09T20:34:36-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:34:36.464397 | controller | 2026-06-09T20:34:36-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:34:36.478729 | controller | 2026-06-09T20:34:36-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:34:36.478806 | controller | 2026-06-09T20:34:36-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:34:36.846108 | controller | 2026-06-09T20:34:36-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:34:36.849206 | controller | 2026-06-09T20:34:36-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:34:37.246002 | controller | 2026-06-09T20:34:37-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:34:37.246104 | controller | 2026-06-09T20:34:37-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:34:37.447416 | controller | 2026-06-09T20:34:37-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:34:37.447498 | controller | 2026-06-09T20:34:37-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:34:38.253547 | controller | 2026-06-09T20:34:38-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:34:38.257990 | controller | 2026-06-09T20:34:38-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:34:38.258038 | controller | 2026-06-09T20:34:38-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:34:39.048459 | controller | 2026-06-09T20:34:39-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:34:39.838319 | controller | 2026-06-09T20:34:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:34:40.647694 | controller | 2026-06-09T20:34:40-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:34:41.049645 | controller | 2026-06-09T20:34:41-04:00 INFO 2026-06-09 20:34:41.049757 | controller | - Config: OK 2026-06-09 20:34:41.049769 | controller | - Gateway: OK 2026-06-09 20:34:41.049777 | controller | - GitServer: OK 2026-06-09 20:34:41.049785 | controller | - HoundSearch: OK 2026-06-09 20:34:41.049793 | controller | - LogJuicer: OK 2026-06-09 20:34:41.049801 | controller | - Logserver: OK 2026-06-09 20:34:41.049809 | controller | - MariaDB: OK 2026-06-09 20:34:41.049817 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:34:41.049826 | controller | - NodePoolLauncher: OK 2026-06-09 20:34:41.049834 | controller | - Zookeeper: OK 2026-06-09 20:34:41.049842 | controller | - Zuul: OK 2026-06-09 20:34:41.049851 | controller | 2026-06-09 20:34:41.049868 | controller | 2026-06-09T20:34:41-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-09 20:34:46.056521 | controller | 2026-06-09T20:34:46-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:34:46.084512 | controller | 2026-06-09T20:34:46-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:34:46.111737 | controller | 2026-06-09T20:34:46-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:34:46.666932 | controller | 2026-06-09T20:34:46-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:34:46.679365 | controller | 2026-06-09T20:34:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:34:46.679444 | controller | 2026-06-09T20:34:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:34:46.689591 | controller | 2026-06-09T20:34:46-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:34:46.701729 | controller | 2026-06-09T20:34:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:34:46.701831 | controller | 2026-06-09T20:34:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:34:47.077814 | controller | 2026-06-09T20:34:47-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:34:47.081039 | controller | 2026-06-09T20:34:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:34:47.478683 | controller | 2026-06-09T20:34:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:34:47.478802 | controller | 2026-06-09T20:34:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:34:47.676679 | controller | 2026-06-09T20:34:47-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:34:48.480305 | controller | 2026-06-09T20:34:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:34:48.485703 | controller | 2026-06-09T20:34:48-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:34:48.485773 | controller | 2026-06-09T20:34:48-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:34:49.266692 | controller | 2026-06-09T20:34:49-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:34:50.061561 | controller | 2026-06-09T20:34:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:34:50.866109 | controller | 2026-06-09T20:34:50-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:34:51.266112 | controller | 2026-06-09T20:34:51-04:00 INFO 2026-06-09 20:34:51.266168 | controller | - Config: OK 2026-06-09 20:34:51.266176 | controller | - Gateway: OK 2026-06-09 20:34:51.266181 | controller | - GitServer: OK 2026-06-09 20:34:51.266186 | controller | - HoundSearch: OK 2026-06-09 20:34:51.266192 | controller | - LogJuicer: OK 2026-06-09 20:34:51.266197 | controller | - Logserver: OK 2026-06-09 20:34:51.266203 | controller | - MariaDB: OK 2026-06-09 20:34:51.266208 | controller | - NodePoolBuilder: OK 2026-06-09 20:34:51.266213 | controller | - NodePoolLauncher: OK 2026-06-09 20:34:51.266218 | controller | - Zookeeper: OK 2026-06-09 20:34:51.266223 | controller | - Zuul: OK 2026-06-09 20:34:51.266244 | controller | 2026-06-09 20:34:51.266255 | controller | 2026-06-09T20:34:51-04:00 INFO Updating controller configmap ... 2026-06-09 20:34:51.658807 | controller | 2026-06-09T20:34:51-04:00 INFO Standalone reconcile done. 2026-06-10 00:34:51.715256 | controller | ok: Runtime: 0:01:11.395472 2026-06-10 00:34:51.727291 | 2026-06-10 00:34:51.727443 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:34:52.257440 | controller | skipping: Conditional result was False 2026-06-10 00:34:52.288762 | 2026-06-10 00:34:52.289031 | TASK [scale zuul components back up to defaults] 2026-06-10 00:34:52.325995 | controller | ok 2026-06-10 00:34:52.355217 | 2026-06-10 00:34:52.355608 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 00:34:52.380709 | controller | ok: 2026-06-10 00:34:52.380995 | controller | { 2026-06-10 00:34:52.381054 | controller | "kind": "sts", 2026-06-10 00:34:52.381098 | controller | "name": "zuul-executor", 2026-06-10 00:34:52.381138 | controller | "replicas": 1 2026-06-10 00:34:52.381189 | controller | } 2026-06-10 00:34:52.387977 | controller | ok: 2026-06-10 00:34:52.388175 | controller | { 2026-06-10 00:34:52.388269 | controller | "kind": "sts", 2026-06-10 00:34:52.388342 | controller | "name": "zuul-merger", 2026-06-10 00:34:52.388397 | controller | "replicas": 1 2026-06-10 00:34:52.388444 | controller | } 2026-06-10 00:34:52.395448 | controller | ok: 2026-06-10 00:34:52.395547 | controller | { 2026-06-10 00:34:52.395636 | controller | "kind": "sts", 2026-06-10 00:34:52.395680 | controller | "name": "zuul-scheduler", 2026-06-10 00:34:52.395719 | controller | "replicas": 1 2026-06-10 00:34:52.395756 | controller | } 2026-06-10 00:34:52.402823 | controller | ok: 2026-06-10 00:34:52.402919 | controller | { 2026-06-10 00:34:52.402969 | controller | "kind": "deployment", 2026-06-10 00:34:52.403009 | controller | "name": "zuul-web", 2026-06-10 00:34:52.403046 | controller | "replicas": 1 2026-06-10 00:34:52.403082 | controller | } 2026-06-10 00:34:52.429675 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:34:52.434653 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:34:52.439657 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:34:52.444937 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:34:52.453492 | 2026-06-10 00:34:52.453679 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 00:34:53.052264 | controller | ok: Runtime: 0:00:00.082645 2026-06-10 00:34:53.060347 | 2026-06-10 00:34:53.060478 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 00:34:53.581683 | controller | skipping: Conditional result was False 2026-06-10 00:34:53.595813 | 2026-06-10 00:34:53.596146 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-09 20:34:53.999127 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 00:34:54.143284 | controller | ok: Runtime: 0:00:00.114928 2026-06-10 00:34:54.155074 | 2026-06-10 00:34:54.155240 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 00:35:15.237312 | controller | ok: Runtime: 0:00:00.079107 2026-06-10 00:35:15.252957 | 2026-06-10 00:35:15.253195 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 00:35:15.793035 | controller | ok: Runtime: 0:00:00.066338 2026-06-10 00:35:15.798875 | 2026-06-10 00:35:15.798962 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 00:35:16.320820 | controller | skipping: Conditional result was False 2026-06-10 00:35:16.335437 | 2026-06-10 00:35:16.335685 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-09 20:35:16.704303 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 00:35:16.883003 | controller | ok: Runtime: 0:00:00.099785 2026-06-10 00:35:16.894379 | 2026-06-10 00:35:16.894542 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 00:35:37.902631 | controller | ok: Runtime: 0:00:00.062283 2026-06-10 00:35:37.916860 | 2026-06-10 00:35:37.917014 | TASK [health-check/scale-resources : Get initial replica count for zuul-scheduler] 2026-06-10 00:35:38.460114 | controller | ok: Runtime: 0:00:00.072267 2026-06-10 00:35:38.471681 | 2026-06-10 00:35:38.471848 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 00:35:39.010892 | controller | skipping: Conditional result was False 2026-06-10 00:35:39.025000 | 2026-06-10 00:35:39.025175 | TASK [health-check/scale-resources : Scale on sts zuul-scheduler] 2026-06-09 20:35:39.325822 | controller | statefulset.apps/zuul-scheduler scaled 2026-06-10 00:35:39.574770 | controller | ok: Runtime: 0:00:00.081312 2026-06-10 00:35:39.586116 | 2026-06-10 00:35:39.586268 | TASK [health-check/scale-resources : Get replica count for zuul-scheduler] 2026-06-10 00:36:42.029401 | controller | ok: Runtime: 0:00:00.096874 2026-06-10 00:36:42.035063 | 2026-06-10 00:36:42.035155 | TASK [health-check/scale-resources : Get initial replica count for zuul-web] 2026-06-10 00:36:42.564971 | controller | ok: Runtime: 0:00:00.104309 2026-06-10 00:36:42.577033 | 2026-06-10 00:36:42.577186 | TASK [health-check/scale-resources : Collect deployment log before executing the scale command] 2026-06-10 00:36:43.116907 | controller | skipping: Conditional result was False 2026-06-10 00:36:43.131006 | 2026-06-10 00:36:43.131175 | TASK [health-check/scale-resources : Scale on deployment zuul-web] 2026-06-09 20:36:43.501897 | controller | deployment.apps/zuul-web scaled 2026-06-10 00:36:43.680534 | controller | ok: Runtime: 0:00:00.096446 2026-06-10 00:36:43.695351 | 2026-06-10 00:36:43.695908 | TASK [health-check/scale-resources : Get replica count for zuul-web] 2026-06-10 00:37:46.109786 | controller | ok: Runtime: 0:00:00.106386 2026-06-10 00:37:46.134820 | 2026-06-10 00:37:46.134974 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 00:37:46.166114 | controller | ok: 2026-06-10 00:37:46.166406 | controller | { 2026-06-10 00:37:46.166435 | controller | "kind": "sts", 2026-06-10 00:37:46.166456 | controller | "name": "zuul-executor", 2026-06-10 00:37:46.166473 | controller | "replicas": 2 2026-06-10 00:37:46.166490 | controller | } 2026-06-10 00:37:46.166524 | controller | ok: All items complete 2026-06-10 00:37:46.166545 | 2026-06-10 00:37:46.170463 | controller | ok: 2026-06-10 00:37:46.170524 | controller | { 2026-06-10 00:37:46.170547 | controller | "kind": "sts", 2026-06-10 00:37:46.170566 | controller | "name": "zuul-merger", 2026-06-10 00:37:46.170610 | controller | "replicas": 2 2026-06-10 00:37:46.170629 | controller | } 2026-06-10 00:37:46.194854 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:37:46.201258 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 00:37:46.212495 | 2026-06-10 00:37:46.212609 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 00:37:46.748262 | controller | ok: Runtime: 0:00:00.101903 2026-06-10 00:37:46.759726 | 2026-06-10 00:37:46.759894 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-09 20:37:47.120689 | controller | Defaulted container "zuul-executor" out of: zuul-executor, zuul-executor-nodeexporter 2026-06-10 00:37:47.308022 | controller | ok: Runtime: 0:00:00.099355 2026-06-10 00:37:47.319063 | 2026-06-10 00:37:47.319197 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-09 20:37:47.635930 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 00:37:47.867933 | controller | ok: Runtime: 0:00:00.079529 2026-06-10 00:37:47.880342 | 2026-06-10 00:37:47.880568 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 00:38:08.931849 | controller | ok: Runtime: 0:00:00.092913 2026-06-10 00:38:08.942528 | 2026-06-10 00:38:08.942705 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 00:38:09.485000 | controller | ok: Runtime: 0:00:00.082523 2026-06-10 00:38:09.496629 | 2026-06-10 00:38:09.496770 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-09 20:38:09.914788 | controller | Defaulted container "zuul-merger" out of: zuul-merger, zuul-merger-nodeexporter 2026-06-10 00:38:10.092254 | controller | ok: Runtime: 0:00:00.085101 2026-06-10 00:38:10.104234 | 2026-06-10 00:38:10.104406 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-09 20:38:10.479485 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 00:38:10.648017 | controller | ok: Runtime: 0:00:00.103877 2026-06-10 00:38:10.659218 | 2026-06-10 00:38:10.659368 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 00:38:31.749060 | controller | ok: Runtime: 0:00:00.095837 2026-06-10 00:38:31.767686 | 2026-06-10 00:38:31.767784 | LOOP [health-check/zuul-components : Check zuul components sequentially] 2026-06-10 00:38:31.792018 | controller | ok: 2026-06-10 00:38:31.792415 | controller | { 2026-06-10 00:38:31.792471 | controller | "component": "web", 2026-06-10 00:38:31.792529 | controller | "kind": "deployment" 2026-06-10 00:38:31.792608 | controller | } 2026-06-10 00:38:31.799293 | controller | ok: 2026-06-10 00:38:31.799387 | controller | { 2026-06-10 00:38:31.799434 | controller | "component": "scheduler", 2026-06-10 00:38:31.799472 | controller | "kind": "statefulset" 2026-06-10 00:38:31.799508 | controller | } 2026-06-10 00:38:31.806167 | controller | ok: 2026-06-10 00:38:31.806265 | controller | { 2026-06-10 00:38:31.806326 | controller | "component": "merger", 2026-06-10 00:38:31.806386 | controller | "kind": "statefulset" 2026-06-10 00:38:31.806455 | controller | } 2026-06-10 00:38:31.813302 | controller | ok: 2026-06-10 00:38:31.813365 | controller | { 2026-06-10 00:38:31.813395 | controller | "component": "executor", 2026-06-10 00:38:31.813418 | controller | "kind": "statefulset" 2026-06-10 00:38:31.813440 | controller | } 2026-06-10 00:38:31.826318 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 00:38:31.830101 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 00:38:31.833654 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 00:38:31.837017 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 00:38:31.846028 | 2026-06-10 00:38:31.846120 | TASK [health-check/zuul-components : Get expected replica count for zuul-web] 2026-06-10 00:38:32.379046 | controller | ok: Runtime: 0:00:00.104425 2026-06-10 00:38:32.386346 | 2026-06-10 00:38:32.386459 | TASK [health-check/zuul-components : Wait until all of zuul-web replicas are ready] 2026-06-10 00:38:32.920212 | controller | ok: Runtime: 0:00:00.114443 2026-06-10 00:38:32.932245 | 2026-06-10 00:38:32.932381 | TASK [health-check/zuul-components : Validate zuul-web Component(s)] 2026-06-10 00:38:33.590652 | controller | ok: OK (799 bytes) 2026-06-10 00:38:33.603755 | 2026-06-10 00:38:33.603902 | TASK [health-check/zuul-components : Get expected replica count for zuul-scheduler] 2026-06-10 00:38:34.146661 | controller | ok: Runtime: 0:00:00.099605 2026-06-10 00:38:34.159490 | 2026-06-10 00:38:34.159700 | TASK [health-check/zuul-components : Wait until all of zuul-scheduler replicas are ready] 2026-06-10 00:38:34.705636 | controller | ok: Runtime: 0:00:00.112298 2026-06-10 00:38:34.717728 | 2026-06-10 00:38:34.717939 | TASK [health-check/zuul-components : Validate zuul-scheduler Component(s)] 2026-06-10 00:38:35.350393 | controller | ok: OK (799 bytes) 2026-06-10 00:38:35.362475 | 2026-06-10 00:38:35.362649 | TASK [health-check/zuul-components : Get expected replica count for zuul-merger] 2026-06-10 00:38:35.907819 | controller | ok: Runtime: 0:00:00.098633 2026-06-10 00:38:35.919122 | 2026-06-10 00:38:35.919266 | TASK [health-check/zuul-components : Wait until all of zuul-merger replicas are ready] 2026-06-10 00:38:36.464402 | controller | ok: Runtime: 0:00:00.090068 2026-06-10 00:38:36.476110 | 2026-06-10 00:38:36.476254 | TASK [health-check/zuul-components : Validate zuul-merger Component(s)] 2026-06-10 00:38:37.180247 | controller | ok: OK (799 bytes) 2026-06-10 00:38:37.193318 | 2026-06-10 00:38:37.193516 | TASK [health-check/zuul-components : Get expected replica count for zuul-executor] 2026-06-10 00:38:37.738464 | controller | ok: Runtime: 0:00:00.096234 2026-06-10 00:38:37.749647 | 2026-06-10 00:38:37.749888 | TASK [health-check/zuul-components : Wait until all of zuul-executor replicas are ready] 2026-06-10 00:38:38.296253 | controller | ok: Runtime: 0:00:00.069789 2026-06-10 00:38:38.307956 | 2026-06-10 00:38:38.308110 | TASK [health-check/zuul-components : Validate zuul-executor Component(s)] 2026-06-10 00:38:38.920259 | controller | ok: OK (799 bytes) 2026-06-10 00:38:38.938003 | 2026-06-10 00:38:38.938186 | TASK [health-check/zuul-connections : Setting Dummy Connections] 2026-06-10 00:38:38.987414 | controller | ok 2026-06-10 00:38:39.001570 | 2026-06-10 00:38:39.001816 | TASK [health-check/zuul-connections : Create SMTP Connection Secret] 2026-06-10 00:38:40.124789 | controller | changed 2026-06-10 00:38:40.131311 | 2026-06-10 00:38:40.131396 | TASK [health-check/zuul-connections : Create ElasticSearch Connection Secret] 2026-06-10 00:38:40.988356 | controller | changed 2026-06-10 00:38:41.005339 | 2026-06-10 00:38:41.005552 | TASK [health-check/zuul-connections : Create Gerrit Connection Secret] 2026-06-10 00:38:41.833274 | controller | changed 2026-06-10 00:38:41.846397 | 2026-06-10 00:38:41.846571 | TASK [health-check/zuul-connections : Create GitHub Connection Secret] 2026-06-10 00:38:42.695991 | controller | changed 2026-06-10 00:38:42.704995 | 2026-06-10 00:38:42.705087 | TASK [health-check/zuul-connections : Create GitLab Connection Secret] 2026-06-10 00:38:43.504510 | controller | changed 2026-06-10 00:38:43.520716 | 2026-06-10 00:38:43.520857 | TASK [health-check/zuul-connections : Create Pagure Connection Secret] 2026-06-10 00:38:44.329891 | controller | changed 2026-06-10 00:38:44.344447 | 2026-06-10 00:38:44.344629 | TASK [health-check/zuul-connections : set_fact] 2026-06-10 00:38:44.382650 | controller | ok 2026-06-10 00:38:44.391955 | 2026-06-10 00:38:44.392079 | TASK [Add dummy Zuul connections] 2026-06-10 00:38:44.421515 | controller | ok 2026-06-10 00:38:44.448023 | 2026-06-10 00:38:44.448111 | TASK [update-custom-resource : read current crd] 2026-06-10 00:38:44.703769 | controller | ok 2026-06-10 00:38:44.711720 | 2026-06-10 00:38:44.711810 | TASK [update-custom-resource : merge the new cr] 2026-06-10 00:38:45.518951 | controller | changed 2026-06-10 00:38:45.525143 | 2026-06-10 00:38:45.525229 | TASK [run the operator in standalone mode] 2026-06-10 00:38:45.547886 | controller | ok 2026-06-10 00:38:45.572992 | 2026-06-10 00:38:45.573212 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:38:45.600458 | controller | skipping: Conditional result was False 2026-06-10 00:38:45.613342 | 2026-06-10 00:38:45.613634 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:38:45.642842 | controller | skipping: Conditional result was False 2026-06-10 00:38:45.656220 | 2026-06-10 00:38:45.656369 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:38:46.314897 | controller | 2026-06-09T20:38:46-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:38:46.314983 | controller | 2026-06-09 20:38:46.333904 | controller | 2026-06-09T20:38:46-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:38:46.333945 | controller | 2026-06-09 20:38:46.349074 | controller | 2026-06-09T20:38:46-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:38:46.370150 | controller | 2026-06-09T20:38:46-04:00 INFO Updating configmap, name: git-server-pi-config-map 2026-06-09 20:38:46.370188 | controller | 2026-06-09T20:38:46-04:00 INFO Updating object, name: git-server-pi-config-map 2026-06-09 20:38:46.380065 | controller | 2026-06-09T20:38:46-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-09 20:38:46.380099 | controller | 2026-06-09T20:38:46-04:00 INFO Updating object, name: git-server 2026-06-09 20:38:46.763650 | controller | 2026-06-09T20:38:46-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:38:47.759815 | controller | 2026-06-09T20:38:47-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:38:47.767207 | controller | 2026-06-09T20:38:47-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:38:47.897877 | controller | 2026-06-09T20:38:47-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:38:47.908554 | controller | 2026-06-09T20:38:47-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:38:47.908648 | controller | 2026-06-09T20:38:47-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:38:47.921285 | controller | 2026-06-09T20:38:47-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:38:47.934745 | controller | 2026-06-09T20:38:47-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:38:47.934789 | controller | 2026-06-09T20:38:47-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:38:48.171344 | controller | 2026-06-09T20:38:48-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:38:48.174433 | controller | 2026-06-09T20:38:48-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 20:38:48.174476 | controller | 2026-06-09T20:38:48-04:00 INFO 2026-06-09 20:38:48.174487 | controller | - Config: Waiting ... 2026-06-09 20:38:48.174497 | controller | - Gateway: OK 2026-06-09 20:38:48.174505 | controller | - GitServer: Waiting ... 2026-06-09 20:38:48.174515 | controller | - HoundSearch: OK 2026-06-09 20:38:48.174523 | controller | - LogJuicer: OK 2026-06-09 20:38:48.174531 | controller | - Logserver: OK 2026-06-09 20:38:48.174539 | controller | - MariaDB: OK 2026-06-09 20:38:48.174547 | controller | - Zookeeper: OK 2026-06-09 20:38:48.174556 | controller | 2026-06-09 20:38:48.174564 | controller | 2026-06-09T20:38:48-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:38:53.186559 | controller | 2026-06-09T20:38:53-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:38:53.225916 | controller | 2026-06-09T20:38:53-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:38:53.225982 | controller | 2026-06-09T20:38:53-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:38:58.237056 | controller | 2026-06-09T20:38:58-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:38:58.237158 | controller | 2026-06-09T20:38:58-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:39:03.248770 | controller | 2026-06-09T20:39:03-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:39:03.248829 | controller | 2026-06-09T20:39:03-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:39:08.260792 | controller | 2026-06-09T20:39:08-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:39:08.261005 | controller | 2026-06-09T20:39:08-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:39:13.272258 | controller | 2026-06-09T20:39:13-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:39:13.284767 | controller | 2026-06-09T20:39:13-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:39:13.300375 | controller | 2026-06-09T20:39:13-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:39:13.305250 | controller | 2026-06-09T20:39:13-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:39:13.305269 | controller | 2026-06-09T20:39:13-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:39:13.309376 | controller | 2026-06-09T20:39:13-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:39:13.316700 | controller | 2026-06-09T20:39:13-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:39:13.316737 | controller | 2026-06-09T20:39:13-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:39:13.329685 | controller | 2026-06-09T20:39:13-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:39:13.331925 | controller | 2026-06-09T20:39:13-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:39:13.697295 | controller | 2026-06-09T20:39:13-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:39:13.697378 | controller | 2026-06-09T20:39:13-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:39:14.290765 | controller | 2026-06-09T20:39:14-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:39:14.294221 | controller | 2026-06-09T20:39:14-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:39:14.298667 | controller | 2026-06-09T20:39:14-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:39:16.678914 | controller | 2026-06-09T20:39:16-04:00 INFO Updating secret, name: zuul-config 2026-06-09 20:39:16.679012 | controller | 2026-06-09T20:39:16-04:00 INFO Updating object, name: zuul-config 2026-06-09 20:39:16.904481 | controller | 2026-06-09T20:39:16-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 20:39:16.904536 | controller | 2026-06-09T20:39:16-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 20:39:16.942347 | controller | 2026-06-09T20:39:16-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-09 20:39:16.942429 | controller | 2026-06-09T20:39:16-04:00 INFO Updating object, name: zuul-executor 2026-06-09 20:39:17.292400 | controller | 2026-06-09T20:39:17-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 20:39:17.292454 | controller | 2026-06-09T20:39:17-04:00 INFO Updating object, name: zuul-web 2026-06-09 20:39:18.092392 | controller | 2026-06-09T20:39:18-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-09 20:39:18.092510 | controller | 2026-06-09T20:39:18-04:00 INFO Updating object, name: zuul-merger 2026-06-09 20:39:18.107004 | controller | 2026-06-09T20:39:18-04:00 INFO 2026-06-09 20:39:18.107046 | controller | - Config: Waiting ... 2026-06-09 20:39:18.107053 | controller | - Gateway: OK 2026-06-09 20:39:18.107059 | controller | - GitServer: OK 2026-06-09 20:39:18.107064 | controller | - HoundSearch: OK 2026-06-09 20:39:18.107069 | controller | - LogJuicer: OK 2026-06-09 20:39:18.107074 | controller | - Logserver: OK 2026-06-09 20:39:18.107079 | controller | - MariaDB: OK 2026-06-09 20:39:18.107084 | controller | - NodePoolBuilder: OK 2026-06-09 20:39:18.107090 | controller | - NodePoolLauncher: OK 2026-06-09 20:39:18.107095 | controller | - Zookeeper: OK 2026-06-09 20:39:18.107100 | controller | - Zuul: Waiting ... 2026-06-09 20:39:18.107105 | controller | - |_ Config Secrets: OK 2026-06-09 20:39:18.107110 | controller | - |_ Executor: Waiting ... 2026-06-09 20:39:18.107116 | controller | - |_ Merger: Waiting ... 2026-06-09 20:39:18.107121 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:39:18.107126 | controller | - |_ Web: Waiting ... 2026-06-09 20:39:18.107131 | controller | 2026-06-09 20:39:18.107167 | controller | 2026-06-09T20:39:18-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:39:23.115495 | controller | 2026-06-09T20:39:23-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:39:23.140405 | controller | 2026-06-09T20:39:23-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:39:23.525389 | controller | 2026-06-09T20:39:23-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:39:24.523702 | controller | 2026-06-09T20:39:24-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:39:24.533285 | controller | 2026-06-09T20:39:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:39:24.533326 | controller | 2026-06-09T20:39:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:39:24.541820 | controller | 2026-06-09T20:39:24-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:39:24.551457 | controller | 2026-06-09T20:39:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:39:24.551511 | controller | 2026-06-09T20:39:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:39:24.928891 | controller | 2026-06-09T20:39:24-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:39:24.932409 | controller | 2026-06-09T20:39:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:39:25.339277 | controller | 2026-06-09T20:39:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:39:25.339376 | controller | 2026-06-09T20:39:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:39:25.527597 | controller | 2026-06-09T20:39:25-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:39:25.530453 | controller | 2026-06-09T20:39:25-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:39:25.913627 | controller | 2026-06-09T20:39:25-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:39:28.332183 | controller | 2026-06-09T20:39:28-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:39:28.332289 | controller | 2026-06-09T20:39:28-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:39:33.339965 | controller | 2026-06-09T20:39:33-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:39:33.340032 | controller | 2026-06-09T20:39:33-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:39:38.352355 | controller | 2026-06-09T20:39:38-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:39:38.352459 | controller | 2026-06-09T20:39:38-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:39:43.363499 | controller | 2026-06-09T20:39:43-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:39:43.363602 | controller | 2026-06-09T20:39:43-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:39:48.373515 | controller | 2026-06-09T20:39:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:39:48.373604 | controller | 2026-06-09T20:39:48-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:39:53.382693 | controller | 2026-06-09T20:39:53-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:39:53.382753 | controller | 2026-06-09T20:39:53-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:39:58.390494 | controller | 2026-06-09T20:39:58-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:39:58.414012 | controller | 2026-06-09T20:39:58-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:39:58.427614 | controller | 2026-06-09T20:39:58-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:39:58.427668 | controller | 2026-06-09T20:39:58-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:39:58.803341 | controller | 2026-06-09T20:39:58-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:39:58.806338 | controller | 2026-06-09T20:39:58-04:00 INFO 2026-06-09 20:39:58.806350 | controller | - Config: Waiting ... 2026-06-09 20:39:58.806355 | controller | - Gateway: OK 2026-06-09 20:39:58.806361 | controller | - GitServer: OK 2026-06-09 20:39:58.806365 | controller | - HoundSearch: OK 2026-06-09 20:39:58.806369 | controller | - LogJuicer: OK 2026-06-09 20:39:58.806372 | controller | - Logserver: OK 2026-06-09 20:39:58.806376 | controller | - MariaDB: OK 2026-06-09 20:39:58.806379 | controller | - NodePoolBuilder: OK 2026-06-09 20:39:58.806383 | controller | - NodePoolLauncher: OK 2026-06-09 20:39:58.806386 | controller | - Zookeeper: OK 2026-06-09 20:39:58.806390 | controller | - Zuul: Waiting ... 2026-06-09 20:39:58.806393 | controller | - |_ Config Secrets: OK 2026-06-09 20:39:58.806397 | controller | - |_ Executor: OK 2026-06-09 20:39:58.806401 | controller | - |_ Merger: OK 2026-06-09 20:39:58.806404 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:39:58.806408 | controller | - |_ Web: Waiting ... 2026-06-09 20:39:58.806412 | controller | 2026-06-09 20:39:58.806415 | controller | 2026-06-09T20:39:58-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:40:03.815914 | controller | 2026-06-09T20:40:03-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:40:03.852142 | controller | 2026-06-09T20:40:03-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:40:04.224921 | controller | 2026-06-09T20:40:04-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:40:05.222142 | controller | 2026-06-09T20:40:05-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:40:05.229620 | controller | 2026-06-09T20:40:05-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:40:05.229681 | controller | 2026-06-09T20:40:05-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:40:05.236306 | controller | 2026-06-09T20:40:05-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:40:05.245694 | controller | 2026-06-09T20:40:05-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:40:05.245772 | controller | 2026-06-09T20:40:05-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:40:05.629782 | controller | 2026-06-09T20:40:05-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:40:05.631906 | controller | 2026-06-09T20:40:05-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:40:06.024384 | controller | 2026-06-09T20:40:06-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:40:06.024449 | controller | 2026-06-09T20:40:06-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:40:06.232873 | controller | 2026-06-09T20:40:06-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:40:06.235595 | controller | 2026-06-09T20:40:06-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:40:06.613972 | controller | 2026-06-09T20:40:06-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:40:09.035873 | controller | 2026-06-09T20:40:09-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:40:09.035926 | controller | 2026-06-09T20:40:09-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:40:14.046404 | controller | 2026-06-09T20:40:14-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:40:14.046441 | controller | 2026-06-09T20:40:14-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:40:19.056940 | controller | 2026-06-09T20:40:19-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:40:19.057010 | controller | 2026-06-09T20:40:19-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:40:24.068861 | controller | 2026-06-09T20:40:24-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:40:24.080436 | controller | 2026-06-09T20:40:24-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-09 20:40:24.080534 | controller | 2026-06-09T20:40:24-04:00 INFO Updating object, name: zuul-weeder 2026-06-09 20:40:24.122858 | controller | 2026-06-09T20:40:24-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:40:24.148614 | controller | 2026-06-09T20:40:24-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:40:24.148705 | controller | 2026-06-09T20:40:24-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:40:24.505034 | controller | 2026-06-09T20:40:24-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:40:24.508655 | controller | 2026-06-09T20:40:24-04:00 INFO 2026-06-09 20:40:24.508706 | controller | - Config: Waiting ... 2026-06-09 20:40:24.508718 | controller | - Gateway: OK 2026-06-09 20:40:24.508728 | controller | - GitServer: OK 2026-06-09 20:40:24.508736 | controller | - HoundSearch: OK 2026-06-09 20:40:24.508745 | controller | - LogJuicer: OK 2026-06-09 20:40:24.508754 | controller | - Logserver: OK 2026-06-09 20:40:24.508762 | controller | - MariaDB: OK 2026-06-09 20:40:24.508771 | controller | - NodePoolBuilder: OK 2026-06-09 20:40:24.508780 | controller | - NodePoolLauncher: OK 2026-06-09 20:40:24.508788 | controller | - Zookeeper: OK 2026-06-09 20:40:24.508797 | controller | - Zuul: Waiting ... 2026-06-09 20:40:24.508805 | controller | - |_ Config Secrets: OK 2026-06-09 20:40:24.508814 | controller | - |_ Executor: OK 2026-06-09 20:40:24.508823 | controller | - |_ Merger: OK 2026-06-09 20:40:24.508832 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:40:24.508841 | controller | - |_ Web: OK 2026-06-09 20:40:24.508850 | controller | 2026-06-09 20:40:24.508865 | controller | 2026-06-09T20:40:24-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-09 20:40:29.519270 | controller | 2026-06-09T20:40:29-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:40:29.556781 | controller | 2026-06-09T20:40:29-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:40:29.933370 | controller | 2026-06-09T20:40:29-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:40:30.926982 | controller | 2026-06-09T20:40:30-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:40:30.940273 | controller | 2026-06-09T20:40:30-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:40:30.940354 | controller | 2026-06-09T20:40:30-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:40:30.949982 | controller | 2026-06-09T20:40:30-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:40:30.963975 | controller | 2026-06-09T20:40:30-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:40:30.964035 | controller | 2026-06-09T20:40:30-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:40:31.341754 | controller | 2026-06-09T20:40:31-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:40:31.344890 | controller | 2026-06-09T20:40:31-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:40:31.738815 | controller | 2026-06-09T20:40:31-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:40:31.738865 | controller | 2026-06-09T20:40:31-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:40:31.940147 | controller | 2026-06-09T20:40:31-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:40:31.943859 | controller | 2026-06-09T20:40:31-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:40:32.316918 | controller | 2026-06-09T20:40:32-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:40:34.744046 | controller | 2026-06-09T20:40:34-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:40:34.753301 | controller | 2026-06-09T20:40:34-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:40:34.753389 | controller | 2026-06-09T20:40:34-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:40:34.773155 | controller | 2026-06-09T20:40:34-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:40:35.128930 | controller | 2026-06-09T20:40:35-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:40:35.129018 | controller | 2026-06-09T20:40:35-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:40:35.935930 | controller | 2026-06-09T20:40:35-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:40:36.343154 | controller | 2026-06-09T20:40:36-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-09 20:40:36.526324 | controller | 2026-06-09T20:40:36-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-09 20:40:43.080581 | controller | 2026-06-09T20:40:43-04:00 INFO Deleting object, name: zuul-web 2026-06-09 20:40:43.087719 | controller | 2026-06-09T20:40:43-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-09 20:40:43.092510 | controller | 2026-06-09T20:40:43-04:00 INFO 2026-06-09 20:40:43.092573 | controller | - Config: Waiting ... 2026-06-09 20:40:43.092585 | controller | - Gateway: OK 2026-06-09 20:40:43.092605 | controller | - GitServer: OK 2026-06-09 20:40:43.092615 | controller | - HoundSearch: OK 2026-06-09 20:40:43.092624 | controller | - LogJuicer: OK 2026-06-09 20:40:43.092633 | controller | - Logserver: OK 2026-06-09 20:40:43.092642 | controller | - MariaDB: OK 2026-06-09 20:40:43.092653 | controller | - NodePoolBuilder: OK 2026-06-09 20:40:43.092662 | controller | - NodePoolLauncher: OK 2026-06-09 20:40:43.092672 | controller | - Zookeeper: OK 2026-06-09 20:40:43.092682 | controller | - Zuul: OK 2026-06-09 20:40:43.092692 | controller | 2026-06-09 20:40:43.092707 | controller | 2026-06-09T20:40:43-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-09 20:40:48.102671 | controller | 2026-06-09T20:40:48-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:40:48.145133 | controller | 2026-06-09T20:40:48-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:40:48.513755 | controller | 2026-06-09T20:40:48-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:40:49.513365 | controller | 2026-06-09T20:40:49-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:40:49.525121 | controller | 2026-06-09T20:40:49-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:40:49.525174 | controller | 2026-06-09T20:40:49-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:40:49.534705 | controller | 2026-06-09T20:40:49-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:40:49.547169 | controller | 2026-06-09T20:40:49-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:40:49.547280 | controller | 2026-06-09T20:40:49-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:40:49.918666 | controller | 2026-06-09T20:40:49-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:40:49.921747 | controller | 2026-06-09T20:40:49-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:40:50.316002 | controller | 2026-06-09T20:40:50-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:40:50.316068 | controller | 2026-06-09T20:40:50-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:40:50.514474 | controller | 2026-06-09T20:40:50-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:40:50.517670 | controller | 2026-06-09T20:40:50-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:40:50.899792 | controller | 2026-06-09T20:40:50-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:40:53.327487 | controller | 2026-06-09T20:40:53-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:40:53.334532 | controller | 2026-06-09T20:40:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:40:53.334578 | controller | 2026-06-09T20:40:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:40:53.353368 | controller | 2026-06-09T20:40:53-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:40:53.713648 | controller | 2026-06-09T20:40:53-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:40:54.520897 | controller | 2026-06-09T20:40:54-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:40:54.524610 | controller | 2026-06-09T20:40:54-04:00 INFO 2026-06-09 20:40:54.524649 | controller | - Config: Waiting ... 2026-06-09 20:40:54.524661 | controller | - Gateway: OK 2026-06-09 20:40:54.524671 | controller | - GitServer: OK 2026-06-09 20:40:54.524680 | controller | - HoundSearch: OK 2026-06-09 20:40:54.524689 | controller | - LogJuicer: OK 2026-06-09 20:40:54.524698 | controller | - Logserver: OK 2026-06-09 20:40:54.524707 | controller | - MariaDB: OK 2026-06-09 20:40:54.524715 | controller | - NodePoolBuilder: OK 2026-06-09 20:40:54.524724 | controller | - NodePoolLauncher: OK 2026-06-09 20:40:54.524733 | controller | - Zookeeper: OK 2026-06-09 20:40:54.524742 | controller | - Zuul: Waiting ... 2026-06-09 20:40:54.524750 | controller | - |_ Config Secrets: OK 2026-06-09 20:40:54.524759 | controller | - |_ Executor: OK 2026-06-09 20:40:54.524768 | controller | - |_ Merger: OK 2026-06-09 20:40:54.524777 | controller | - |_ Scheduler: OK 2026-06-09 20:40:54.524785 | controller | - |_ Web: Waiting ... 2026-06-09 20:40:54.524794 | controller | 2026-06-09 20:40:54.524809 | controller | 2026-06-09T20:40:54-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-09 20:40:59.537416 | controller | 2026-06-09T20:40:59-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:40:59.578102 | controller | 2026-06-09T20:40:59-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:40:59.947990 | controller | 2026-06-09T20:40:59-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:41:00.947522 | controller | 2026-06-09T20:41:00-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:41:00.967634 | controller | 2026-06-09T20:41:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:41:00.967705 | controller | 2026-06-09T20:41:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:41:00.981160 | controller | 2026-06-09T20:41:00-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:41:00.997608 | controller | 2026-06-09T20:41:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:41:00.997698 | controller | 2026-06-09T20:41:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:41:01.347342 | controller | 2026-06-09T20:41:01-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:41:01.351271 | controller | 2026-06-09T20:41:01-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:41:01.756452 | controller | 2026-06-09T20:41:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:41:01.756553 | controller | 2026-06-09T20:41:01-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:41:01.956296 | controller | 2026-06-09T20:41:01-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:41:01.960992 | controller | 2026-06-09T20:41:01-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:02.333097 | controller | 2026-06-09T20:41:02-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:04.761805 | controller | 2026-06-09T20:41:04-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:41:04.775666 | controller | 2026-06-09T20:41:04-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:41:04.775743 | controller | 2026-06-09T20:41:04-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:41:04.799171 | controller | 2026-06-09T20:41:04-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:41:05.146605 | controller | 2026-06-09T20:41:05-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:41:05.146697 | controller | 2026-06-09T20:41:05-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:41:05.951836 | controller | 2026-06-09T20:41:05-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:41:05.955980 | controller | 2026-06-09T20:41:05-04:00 INFO 2026-06-09 20:41:05.956017 | controller | - Config: Waiting ... 2026-06-09 20:41:05.956028 | controller | - Gateway: OK 2026-06-09 20:41:05.956037 | controller | - GitServer: OK 2026-06-09 20:41:05.956045 | controller | - HoundSearch: OK 2026-06-09 20:41:05.956054 | controller | - LogJuicer: OK 2026-06-09 20:41:05.956062 | controller | - Logserver: OK 2026-06-09 20:41:05.956070 | controller | - MariaDB: OK 2026-06-09 20:41:05.956078 | controller | - NodePoolBuilder: OK 2026-06-09 20:41:05.956086 | controller | - NodePoolLauncher: OK 2026-06-09 20:41:05.956094 | controller | - Zookeeper: OK 2026-06-09 20:41:05.956102 | controller | - Zuul: Waiting ... 2026-06-09 20:41:05.956110 | controller | - |_ Config Secrets: OK 2026-06-09 20:41:05.956118 | controller | - |_ Executor: OK 2026-06-09 20:41:05.956126 | controller | - |_ Merger: OK 2026-06-09 20:41:05.956134 | controller | - |_ Scheduler: OK 2026-06-09 20:41:05.956142 | controller | - |_ Web: Waiting ... 2026-06-09 20:41:05.956150 | controller | 2026-06-09 20:41:05.956159 | controller | 2026-06-09T20:41:05-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-09 20:41:10.965957 | controller | 2026-06-09T20:41:10-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:41:11.008511 | controller | 2026-06-09T20:41:11-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:41:11.376615 | controller | 2026-06-09T20:41:11-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:41:12.379300 | controller | 2026-06-09T20:41:12-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:41:12.390905 | controller | 2026-06-09T20:41:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:41:12.390991 | controller | 2026-06-09T20:41:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:41:12.400474 | controller | 2026-06-09T20:41:12-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:41:12.413718 | controller | 2026-06-09T20:41:12-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:41:12.413775 | controller | 2026-06-09T20:41:12-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:41:12.781192 | controller | 2026-06-09T20:41:12-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:41:12.784752 | controller | 2026-06-09T20:41:12-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:41:13.180123 | controller | 2026-06-09T20:41:13-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:41:13.180207 | controller | 2026-06-09T20:41:13-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:41:13.380929 | controller | 2026-06-09T20:41:13-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:41:13.384065 | controller | 2026-06-09T20:41:13-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:13.763912 | controller | 2026-06-09T20:41:13-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:16.188353 | controller | 2026-06-09T20:41:16-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:41:16.195342 | controller | 2026-06-09T20:41:16-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:41:16.195373 | controller | 2026-06-09T20:41:16-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:41:16.212103 | controller | 2026-06-09T20:41:16-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:41:16.575984 | controller | 2026-06-09T20:41:16-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:41:16.576099 | controller | 2026-06-09T20:41:16-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:41:17.383490 | controller | 2026-06-09T20:41:17-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:41:17.387706 | controller | 2026-06-09T20:41:17-04:00 INFO 2026-06-09 20:41:17.387737 | controller | - Config: Waiting ... 2026-06-09 20:41:17.387748 | controller | - Gateway: OK 2026-06-09 20:41:17.387757 | controller | - GitServer: OK 2026-06-09 20:41:17.387765 | controller | - HoundSearch: OK 2026-06-09 20:41:17.387773 | controller | - LogJuicer: OK 2026-06-09 20:41:17.387781 | controller | - Logserver: OK 2026-06-09 20:41:17.387789 | controller | - MariaDB: OK 2026-06-09 20:41:17.387796 | controller | - NodePoolBuilder: OK 2026-06-09 20:41:17.387804 | controller | - NodePoolLauncher: OK 2026-06-09 20:41:17.387812 | controller | - Zookeeper: OK 2026-06-09 20:41:17.387820 | controller | - Zuul: Waiting ... 2026-06-09 20:41:17.387827 | controller | - |_ Config Secrets: OK 2026-06-09 20:41:17.387835 | controller | - |_ Executor: OK 2026-06-09 20:41:17.387843 | controller | - |_ Merger: OK 2026-06-09 20:41:17.387851 | controller | - |_ Scheduler: OK 2026-06-09 20:41:17.387860 | controller | - |_ Web: Waiting ... 2026-06-09 20:41:17.387869 | controller | 2026-06-09 20:41:17.387883 | controller | 2026-06-09T20:41:17-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-09 20:41:22.396081 | controller | 2026-06-09T20:41:22-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:41:22.422823 | controller | 2026-06-09T20:41:22-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:41:22.808292 | controller | 2026-06-09T20:41:22-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:41:23.803203 | controller | 2026-06-09T20:41:23-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:41:23.815603 | controller | 2026-06-09T20:41:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:41:23.815668 | controller | 2026-06-09T20:41:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:41:23.823840 | controller | 2026-06-09T20:41:23-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:41:23.833590 | controller | 2026-06-09T20:41:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:41:23.833619 | controller | 2026-06-09T20:41:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:41:24.209503 | controller | 2026-06-09T20:41:24-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:41:24.212785 | controller | 2026-06-09T20:41:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:41:24.611777 | controller | 2026-06-09T20:41:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:41:24.611854 | controller | 2026-06-09T20:41:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:41:24.814604 | controller | 2026-06-09T20:41:24-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:41:24.818000 | controller | 2026-06-09T20:41:24-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:25.193160 | controller | 2026-06-09T20:41:25-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:27.616922 | controller | 2026-06-09T20:41:27-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:41:27.625660 | controller | 2026-06-09T20:41:27-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:41:27.625714 | controller | 2026-06-09T20:41:27-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:41:27.646449 | controller | 2026-06-09T20:41:27-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:41:28.006351 | controller | 2026-06-09T20:41:28-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:41:28.006416 | controller | 2026-06-09T20:41:28-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:41:28.810644 | controller | 2026-06-09T20:41:28-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:41:28.813841 | controller | 2026-06-09T20:41:28-04:00 INFO 2026-06-09 20:41:28.813879 | controller | - Config: Waiting ... 2026-06-09 20:41:28.813890 | controller | - Gateway: OK 2026-06-09 20:41:28.813898 | controller | - GitServer: OK 2026-06-09 20:41:28.813923 | controller | - HoundSearch: OK 2026-06-09 20:41:28.813931 | controller | - LogJuicer: OK 2026-06-09 20:41:28.813939 | controller | - Logserver: OK 2026-06-09 20:41:28.813947 | controller | - MariaDB: OK 2026-06-09 20:41:28.813955 | controller | - NodePoolBuilder: OK 2026-06-09 20:41:28.813963 | controller | - NodePoolLauncher: OK 2026-06-09 20:41:28.813971 | controller | - Zookeeper: OK 2026-06-09 20:41:28.813979 | controller | - Zuul: Waiting ... 2026-06-09 20:41:28.813986 | controller | - |_ Config Secrets: OK 2026-06-09 20:41:28.813994 | controller | - |_ Executor: OK 2026-06-09 20:41:28.814002 | controller | - |_ Merger: OK 2026-06-09 20:41:28.814010 | controller | - |_ Scheduler: OK 2026-06-09 20:41:28.814018 | controller | - |_ Web: Waiting ... 2026-06-09 20:41:28.814027 | controller | 2026-06-09 20:41:28.814040 | controller | 2026-06-09T20:41:28-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-09 20:41:33.821171 | controller | 2026-06-09T20:41:33-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:41:33.857132 | controller | 2026-06-09T20:41:33-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:41:34.230024 | controller | 2026-06-09T20:41:34-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:41:35.231957 | controller | 2026-06-09T20:41:35-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:41:35.243621 | controller | 2026-06-09T20:41:35-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:41:35.243673 | controller | 2026-06-09T20:41:35-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:41:35.254321 | controller | 2026-06-09T20:41:35-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:41:35.266363 | controller | 2026-06-09T20:41:35-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:41:35.266528 | controller | 2026-06-09T20:41:35-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:41:35.641152 | controller | 2026-06-09T20:41:35-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:41:35.645072 | controller | 2026-06-09T20:41:35-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:41:36.040294 | controller | 2026-06-09T20:41:36-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:41:36.040390 | controller | 2026-06-09T20:41:36-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:41:36.240522 | controller | 2026-06-09T20:41:36-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:41:36.244634 | controller | 2026-06-09T20:41:36-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:36.620117 | controller | 2026-06-09T20:41:36-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:39.047427 | controller | 2026-06-09T20:41:39-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:41:39.057721 | controller | 2026-06-09T20:41:39-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:41:39.057815 | controller | 2026-06-09T20:41:39-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:41:39.076986 | controller | 2026-06-09T20:41:39-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:41:39.433522 | controller | 2026-06-09T20:41:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:41:39.433597 | controller | 2026-06-09T20:41:39-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:41:40.238112 | controller | 2026-06-09T20:41:40-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:41:40.242354 | controller | 2026-06-09T20:41:40-04:00 INFO 2026-06-09 20:41:40.242435 | controller | - Config: Waiting ... 2026-06-09 20:41:40.242447 | controller | - Gateway: OK 2026-06-09 20:41:40.242456 | controller | - GitServer: OK 2026-06-09 20:41:40.242464 | controller | - HoundSearch: OK 2026-06-09 20:41:40.242472 | controller | - LogJuicer: OK 2026-06-09 20:41:40.242480 | controller | - Logserver: OK 2026-06-09 20:41:40.242488 | controller | - MariaDB: OK 2026-06-09 20:41:40.242496 | controller | - NodePoolBuilder: OK 2026-06-09 20:41:40.242504 | controller | - NodePoolLauncher: OK 2026-06-09 20:41:40.242512 | controller | - Zookeeper: OK 2026-06-09 20:41:40.242519 | controller | - Zuul: Waiting ... 2026-06-09 20:41:40.242527 | controller | - |_ Config Secrets: OK 2026-06-09 20:41:40.242535 | controller | - |_ Executor: OK 2026-06-09 20:41:40.242543 | controller | - |_ Merger: OK 2026-06-09 20:41:40.242551 | controller | - |_ Scheduler: OK 2026-06-09 20:41:40.242575 | controller | - |_ Web: Waiting ... 2026-06-09 20:41:40.242583 | controller | 2026-06-09 20:41:40.242592 | controller | 2026-06-09T20:41:40-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-09 20:41:45.246462 | controller | 2026-06-09T20:41:45-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:41:45.271939 | controller | 2026-06-09T20:41:45-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:41:45.653886 | controller | 2026-06-09T20:41:45-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:41:46.656693 | controller | 2026-06-09T20:41:46-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:41:46.668811 | controller | 2026-06-09T20:41:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:41:46.668886 | controller | 2026-06-09T20:41:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:41:46.678965 | controller | 2026-06-09T20:41:46-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:41:46.691143 | controller | 2026-06-09T20:41:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:41:46.691214 | controller | 2026-06-09T20:41:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:41:47.064866 | controller | 2026-06-09T20:41:47-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:41:47.067932 | controller | 2026-06-09T20:41:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:41:47.472302 | controller | 2026-06-09T20:41:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:41:47.472405 | controller | 2026-06-09T20:41:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:41:47.666270 | controller | 2026-06-09T20:41:47-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:41:47.669708 | controller | 2026-06-09T20:41:47-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:48.047302 | controller | 2026-06-09T20:41:48-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:50.470648 | controller | 2026-06-09T20:41:50-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:41:50.478901 | controller | 2026-06-09T20:41:50-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:41:50.478980 | controller | 2026-06-09T20:41:50-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:41:50.497083 | controller | 2026-06-09T20:41:50-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:41:50.858909 | controller | 2026-06-09T20:41:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:41:50.858970 | controller | 2026-06-09T20:41:50-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:41:51.661684 | controller | 2026-06-09T20:41:51-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:41:51.663874 | controller | 2026-06-09T20:41:51-04:00 INFO 2026-06-09 20:41:51.663891 | controller | - Config: Waiting ... 2026-06-09 20:41:51.663896 | controller | - Gateway: OK 2026-06-09 20:41:51.663900 | controller | - GitServer: OK 2026-06-09 20:41:51.663904 | controller | - HoundSearch: OK 2026-06-09 20:41:51.663908 | controller | - LogJuicer: OK 2026-06-09 20:41:51.663912 | controller | - Logserver: OK 2026-06-09 20:41:51.663915 | controller | - MariaDB: OK 2026-06-09 20:41:51.663919 | controller | - NodePoolBuilder: OK 2026-06-09 20:41:51.663922 | controller | - NodePoolLauncher: OK 2026-06-09 20:41:51.663926 | controller | - Zookeeper: OK 2026-06-09 20:41:51.663929 | controller | - Zuul: Waiting ... 2026-06-09 20:41:51.663933 | controller | - |_ Config Secrets: OK 2026-06-09 20:41:51.663937 | controller | - |_ Executor: OK 2026-06-09 20:41:51.663940 | controller | - |_ Merger: OK 2026-06-09 20:41:51.663944 | controller | - |_ Scheduler: OK 2026-06-09 20:41:51.663947 | controller | - |_ Web: Waiting ... 2026-06-09 20:41:51.663951 | controller | 2026-06-09 20:41:51.663978 | controller | 2026-06-09T20:41:51-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-09 20:41:56.674837 | controller | 2026-06-09T20:41:56-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:41:56.716756 | controller | 2026-06-09T20:41:56-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:41:57.083617 | controller | 2026-06-09T20:41:57-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:41:58.079870 | controller | 2026-06-09T20:41:58-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:41:58.087717 | controller | 2026-06-09T20:41:58-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:41:58.087747 | controller | 2026-06-09T20:41:58-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:41:58.094410 | controller | 2026-06-09T20:41:58-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:41:58.103731 | controller | 2026-06-09T20:41:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:41:58.103773 | controller | 2026-06-09T20:41:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:41:58.491117 | controller | 2026-06-09T20:41:58-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:41:58.493558 | controller | 2026-06-09T20:41:58-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:41:58.889580 | controller | 2026-06-09T20:41:58-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:41:58.889675 | controller | 2026-06-09T20:41:58-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:41:59.089631 | controller | 2026-06-09T20:41:59-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:41:59.094001 | controller | 2026-06-09T20:41:59-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:41:59.472051 | controller | 2026-06-09T20:41:59-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-09 20:42:01.894930 | controller | 2026-06-09T20:42:01-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:42:01.905394 | controller | 2026-06-09T20:42:01-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:42:01.905485 | controller | 2026-06-09T20:42:01-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:42:01.931200 | controller | 2026-06-09T20:42:01-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:42:02.283537 | controller | 2026-06-09T20:42:02-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:42:02.283640 | controller | 2026-06-09T20:42:02-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:42:03.091309 | controller | 2026-06-09T20:42:03-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:42:03.498563 | controller | 2026-06-09T20:42:03-04:00 INFO 2026-06-09 20:42:03.498641 | controller | - Config: OK 2026-06-09 20:42:03.498648 | controller | - Gateway: OK 2026-06-09 20:42:03.498654 | controller | - GitServer: OK 2026-06-09 20:42:03.498659 | controller | - HoundSearch: OK 2026-06-09 20:42:03.498664 | controller | - LogJuicer: OK 2026-06-09 20:42:03.498693 | controller | - Logserver: OK 2026-06-09 20:42:03.498698 | controller | - MariaDB: OK 2026-06-09 20:42:03.498703 | controller | - NodePoolBuilder: OK 2026-06-09 20:42:03.498708 | controller | - NodePoolLauncher: OK 2026-06-09 20:42:03.498713 | controller | - Zookeeper: OK 2026-06-09 20:42:03.498718 | controller | - Zuul: OK 2026-06-09 20:42:03.498724 | controller | 2026-06-09 20:42:03.498734 | controller | 2026-06-09T20:42:03-04:00 INFO Updating controller configmap ... 2026-06-09 20:42:03.880376 | controller | 2026-06-09T20:42:03-04:00 INFO Standalone reconcile done. 2026-06-10 00:42:03.952930 | controller | ok: Runtime: 0:03:17.925487 2026-06-10 00:42:03.963470 | 2026-06-10 00:42:03.963633 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:42:04.492122 | controller | skipping: Conditional result was False 2026-06-10 00:42:04.519145 | 2026-06-10 00:42:04.519330 | TASK [health-check/zuul-connections : Wait for the new Zuul connections to appear in the Zuul API] 2026-06-10 00:42:05.098310 | controller | ok: OK (1399 bytes) 2026-06-10 00:42:05.109413 | 2026-06-10 00:42:05.109564 | TASK [health-check/zuul-connections : Ensure the new Zuul connections (non SourceInterface) exist in the scheduler's zuul.conf] 2026-06-09 20:42:05.465810 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-09 20:42:05.576212 | controller | [connection dummy-elasticsearch-conn] 2026-06-09 20:42:05.673822 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-09 20:42:05.773133 | controller | [connection dummy-smtp-conn] 2026-06-10 00:42:06.153623 | controller | ok: Runtime: 0:00:00.417546 2026-06-10 00:42:06.159263 | 2026-06-10 00:42:06.159365 | TASK [health-check/zuul-connections : Ensure ElasticSearch URI is configured with basic auth] 2026-06-09 20:42:06.521339 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-09 20:42:06.674017 | controller | uri = http://es-username:es-password@test:9200 2026-06-10 00:42:06.722796 | controller | ok: Runtime: 0:00:00.264430 2026-06-10 00:42:06.731500 | 2026-06-10 00:42:06.731694 | TASK [health-check/zuul-connections : Ensure SMTP password is present] 2026-06-09 20:42:07.151783 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-09 20:42:07.260783 | controller | password = smtp-password 2026-06-10 00:42:07.339884 | controller | ok: Runtime: 0:00:00.211296 2026-06-10 00:42:07.345815 | 2026-06-10 00:42:07.345907 | TASK [health-check/zuul-connections : Ensure the new Zuul dummy gerrit secret exist in the scheduler's zuul.conf] 2026-06-09 20:42:07.775223 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-09 20:42:07.902126 | controller | sshkey = /var/lib/zuul-dummygerritsecret/..data/priv 2026-06-09 20:42:07.902267 | controller | sshkey = /var/lib/zuul-dummygerritsecret/..data/priv 2026-06-09 20:42:08.013336 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 00:42:08.386805 | controller | ok: Runtime: 0:00:00.474500 2026-06-10 00:42:08.396050 | 2026-06-10 00:42:08.396183 | TASK [Delete the dummy Zuul connections] 2026-06-10 00:42:08.431832 | controller | ok 2026-06-10 00:42:08.467399 | 2026-06-10 00:42:08.467538 | TASK [update-custom-resource : read current crd] 2026-06-10 00:42:08.756256 | controller | ok 2026-06-10 00:42:08.765735 | 2026-06-10 00:42:08.765919 | TASK [update-custom-resource : merge the new cr] 2026-06-10 00:42:09.571377 | controller | changed 2026-06-10 00:42:09.582015 | 2026-06-10 00:42:09.582163 | TASK [run the operator in standalone mode] 2026-06-10 00:42:09.621880 | controller | ok 2026-06-10 00:42:09.652340 | 2026-06-10 00:42:09.652474 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:42:09.680866 | controller | skipping: Conditional result was False 2026-06-10 00:42:09.689070 | 2026-06-10 00:42:09.689165 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:42:09.715473 | controller | skipping: Conditional result was False 2026-06-10 00:42:09.723692 | 2026-06-10 00:42:09.723792 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:42:10.254601 | controller | 2026-06-09T20:42:10-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:42:10.254657 | controller | 2026-06-09 20:42:10.262556 | controller | 2026-06-09T20:42:10-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:42:10.262572 | controller | 2026-06-09 20:42:10.277506 | controller | 2026-06-09T20:42:10-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:42:10.286975 | controller | 2026-06-09T20:42:10-04:00 INFO Updating configmap, name: git-server-pi-config-map 2026-06-09 20:42:10.287004 | controller | 2026-06-09T20:42:10-04:00 INFO Updating object, name: git-server-pi-config-map 2026-06-09 20:42:10.294390 | controller | 2026-06-09T20:42:10-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-09 20:42:10.294421 | controller | 2026-06-09T20:42:10-04:00 INFO Updating object, name: git-server 2026-06-09 20:42:10.318356 | controller | 2026-06-09T20:42:10-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:42:10.886993 | controller | 2026-06-09T20:42:10-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:42:10.894139 | controller | 2026-06-09T20:42:10-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:42:11.007889 | controller | 2026-06-09T20:42:11-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:42:11.012849 | controller | 2026-06-09T20:42:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:42:11.012897 | controller | 2026-06-09T20:42:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:42:11.018556 | controller | 2026-06-09T20:42:11-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:42:11.027725 | controller | 2026-06-09T20:42:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:42:11.027777 | controller | 2026-06-09T20:42:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:42:11.286496 | controller | 2026-06-09T20:42:11-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:42:11.288743 | controller | 2026-06-09T20:42:11-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 20:42:11.288786 | controller | 2026-06-09T20:42:11-04:00 INFO 2026-06-09 20:42:11.288798 | controller | - Config: Waiting ... 2026-06-09 20:42:11.288806 | controller | - Gateway: OK 2026-06-09 20:42:11.288813 | controller | - GitServer: Waiting ... 2026-06-09 20:42:11.288820 | controller | - HoundSearch: OK 2026-06-09 20:42:11.288827 | controller | - LogJuicer: OK 2026-06-09 20:42:11.288835 | controller | - Logserver: OK 2026-06-09 20:42:11.288842 | controller | - MariaDB: OK 2026-06-09 20:42:11.288850 | controller | - Zookeeper: OK 2026-06-09 20:42:11.288857 | controller | 2026-06-09 20:42:11.288945 | controller | 2026-06-09T20:42:11-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:42:16.294208 | controller | 2026-06-09T20:42:16-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:42:16.322976 | controller | 2026-06-09T20:42:16-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:42:16.323057 | controller | 2026-06-09T20:42:16-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:42:21.331775 | controller | 2026-06-09T20:42:21-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:42:21.331819 | controller | 2026-06-09T20:42:21-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:42:26.341993 | controller | 2026-06-09T20:42:26-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:42:26.342071 | controller | 2026-06-09T20:42:26-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:42:31.353061 | controller | 2026-06-09T20:42:31-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:42:31.353117 | controller | 2026-06-09T20:42:31-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 20:42:36.363026 | controller | 2026-06-09T20:42:36-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:42:36.388342 | controller | 2026-06-09T20:42:36-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:42:36.422152 | controller | 2026-06-09T20:42:36-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:42:36.433103 | controller | 2026-06-09T20:42:36-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:42:36.433154 | controller | 2026-06-09T20:42:36-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:42:36.442320 | controller | 2026-06-09T20:42:36-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:42:36.477102 | controller | 2026-06-09T20:42:36-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:42:36.477156 | controller | 2026-06-09T20:42:36-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:42:36.499622 | controller | 2026-06-09T20:42:36-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:42:36.503048 | controller | 2026-06-09T20:42:36-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:42:36.786320 | controller | 2026-06-09T20:42:36-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:42:36.786362 | controller | 2026-06-09T20:42:36-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:42:37.383651 | controller | 2026-06-09T20:42:37-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:42:37.770986 | controller | 2026-06-09T20:42:37-04:00 INFO Updating secret, name: zuul-config 2026-06-09 20:42:37.771023 | controller | 2026-06-09T20:42:37-04:00 INFO Updating object, name: zuul-config 2026-06-09 20:42:38.183643 | controller | 2026-06-09T20:42:38-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 20:42:38.183687 | controller | 2026-06-09T20:42:38-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 20:42:38.981967 | controller | 2026-06-09T20:42:38-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-09 20:42:38.982010 | controller | 2026-06-09T20:42:38-04:00 INFO Updating object, name: zuul-executor 2026-06-09 20:42:39.782588 | controller | 2026-06-09T20:42:39-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 20:42:39.782625 | controller | 2026-06-09T20:42:39-04:00 INFO Updating object, name: zuul-web 2026-06-09 20:42:40.581978 | controller | 2026-06-09T20:42:40-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-09 20:42:40.582046 | controller | 2026-06-09T20:42:40-04:00 INFO Updating object, name: zuul-merger 2026-06-09 20:42:40.588397 | controller | 2026-06-09T20:42:40-04:00 INFO 2026-06-09 20:42:40.588442 | controller | - Config: Waiting ... 2026-06-09 20:42:40.588458 | controller | - Gateway: OK 2026-06-09 20:42:40.588470 | controller | - GitServer: OK 2026-06-09 20:42:40.588481 | controller | - HoundSearch: OK 2026-06-09 20:42:40.588492 | controller | - LogJuicer: OK 2026-06-09 20:42:40.588503 | controller | - Logserver: OK 2026-06-09 20:42:40.588514 | controller | - MariaDB: OK 2026-06-09 20:42:40.588523 | controller | - NodePoolBuilder: OK 2026-06-09 20:42:40.588531 | controller | - NodePoolLauncher: OK 2026-06-09 20:42:40.588538 | controller | - Zookeeper: OK 2026-06-09 20:42:40.588546 | controller | - Zuul: Waiting ... 2026-06-09 20:42:40.588553 | controller | - |_ Config Secrets: OK 2026-06-09 20:42:40.588561 | controller | - |_ Executor: Waiting ... 2026-06-09 20:42:40.588591 | controller | - |_ Merger: Waiting ... 2026-06-09 20:42:40.588599 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:42:40.588607 | controller | - |_ Web: Waiting ... 2026-06-09 20:42:40.588614 | controller | 2026-06-09 20:42:40.588629 | controller | 2026-06-09T20:42:40-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:42:45.598006 | controller | 2026-06-09T20:42:45-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:42:45.626426 | controller | 2026-06-09T20:42:45-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:42:45.647477 | controller | 2026-06-09T20:42:45-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:42:46.204789 | controller | 2026-06-09T20:42:46-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:42:46.213559 | controller | 2026-06-09T20:42:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:42:46.213606 | controller | 2026-06-09T20:42:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:42:46.221122 | controller | 2026-06-09T20:42:46-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:42:46.230126 | controller | 2026-06-09T20:42:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:42:46.230200 | controller | 2026-06-09T20:42:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:42:46.612838 | controller | 2026-06-09T20:42:46-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:42:46.615547 | controller | 2026-06-09T20:42:46-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:42:47.016845 | controller | 2026-06-09T20:42:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:42:47.016906 | controller | 2026-06-09T20:42:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:42:47.217222 | controller | 2026-06-09T20:42:47-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:42:48.020922 | controller | 2026-06-09T20:42:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:42:48.021028 | controller | 2026-06-09T20:42:48-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:42:53.028320 | controller | 2026-06-09T20:42:53-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:42:53.028390 | controller | 2026-06-09T20:42:53-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:42:58.039181 | controller | 2026-06-09T20:42:58-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:42:58.039276 | controller | 2026-06-09T20:42:58-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:43:03.048375 | controller | 2026-06-09T20:43:03-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:03.048427 | controller | 2026-06-09T20:43:03-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:43:08.056330 | controller | 2026-06-09T20:43:08-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:08.056393 | controller | 2026-06-09T20:43:08-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:43:13.064326 | controller | 2026-06-09T20:43:13-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:13.064381 | controller | 2026-06-09T20:43:13-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:43:18.071081 | controller | 2026-06-09T20:43:18-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:43:18.095508 | controller | 2026-06-09T20:43:18-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:43:18.118538 | controller | 2026-06-09T20:43:18-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:43:18.118588 | controller | 2026-06-09T20:43:18-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:43:18.484633 | controller | 2026-06-09T20:43:18-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:43:18.487873 | controller | 2026-06-09T20:43:18-04:00 INFO 2026-06-09 20:43:18.487905 | controller | - Config: Waiting ... 2026-06-09 20:43:18.487915 | controller | - Gateway: OK 2026-06-09 20:43:18.487924 | controller | - GitServer: OK 2026-06-09 20:43:18.487932 | controller | - HoundSearch: OK 2026-06-09 20:43:18.487940 | controller | - LogJuicer: OK 2026-06-09 20:43:18.487947 | controller | - Logserver: OK 2026-06-09 20:43:18.487955 | controller | - MariaDB: OK 2026-06-09 20:43:18.487963 | controller | - NodePoolBuilder: OK 2026-06-09 20:43:18.487971 | controller | - NodePoolLauncher: OK 2026-06-09 20:43:18.487978 | controller | - Zookeeper: OK 2026-06-09 20:43:18.487986 | controller | - Zuul: Waiting ... 2026-06-09 20:43:18.487994 | controller | - |_ Config Secrets: OK 2026-06-09 20:43:18.488002 | controller | - |_ Executor: OK 2026-06-09 20:43:18.488009 | controller | - |_ Merger: OK 2026-06-09 20:43:18.488017 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:43:18.488025 | controller | - |_ Web: Waiting ... 2026-06-09 20:43:18.488033 | controller | 2026-06-09 20:43:18.488041 | controller | 2026-06-09T20:43:18-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:43:23.498702 | controller | 2026-06-09T20:43:23-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:43:23.524616 | controller | 2026-06-09T20:43:23-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:43:23.547459 | controller | 2026-06-09T20:43:23-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:43:24.105691 | controller | 2026-06-09T20:43:24-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:43:24.118430 | controller | 2026-06-09T20:43:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:43:24.118497 | controller | 2026-06-09T20:43:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:43:24.128497 | controller | 2026-06-09T20:43:24-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:43:24.141506 | controller | 2026-06-09T20:43:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:43:24.141547 | controller | 2026-06-09T20:43:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:43:24.513573 | controller | 2026-06-09T20:43:24-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:43:24.516501 | controller | 2026-06-09T20:43:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:43:24.912037 | controller | 2026-06-09T20:43:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:43:24.912113 | controller | 2026-06-09T20:43:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:43:25.107656 | controller | 2026-06-09T20:43:25-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:43:25.920888 | controller | 2026-06-09T20:43:25-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:25.920992 | controller | 2026-06-09T20:43:25-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:43:30.932427 | controller | 2026-06-09T20:43:30-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:30.932517 | controller | 2026-06-09T20:43:30-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:43:35.940860 | controller | 2026-06-09T20:43:35-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:35.940998 | controller | 2026-06-09T20:43:35-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:43:40.952676 | controller | 2026-06-09T20:43:40-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:40.952755 | controller | 2026-06-09T20:43:40-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:43:45.961451 | controller | 2026-06-09T20:43:45-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:45.970144 | controller | 2026-06-09T20:43:45-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-09 20:43:45.970174 | controller | 2026-06-09T20:43:45-04:00 INFO Updating object, name: zuul-weeder 2026-06-09 20:43:46.002128 | controller | 2026-06-09T20:43:46-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:43:46.042974 | controller | 2026-06-09T20:43:46-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:43:46.043063 | controller | 2026-06-09T20:43:46-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:43:46.388880 | controller | 2026-06-09T20:43:46-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:43:46.392852 | controller | 2026-06-09T20:43:46-04:00 INFO 2026-06-09 20:43:46.392890 | controller | - Config: Waiting ... 2026-06-09 20:43:46.392901 | controller | - Gateway: OK 2026-06-09 20:43:46.392911 | controller | - GitServer: OK 2026-06-09 20:43:46.392919 | controller | - HoundSearch: OK 2026-06-09 20:43:46.392927 | controller | - LogJuicer: OK 2026-06-09 20:43:46.392935 | controller | - Logserver: OK 2026-06-09 20:43:46.392943 | controller | - MariaDB: OK 2026-06-09 20:43:46.392950 | controller | - NodePoolBuilder: OK 2026-06-09 20:43:46.392958 | controller | - NodePoolLauncher: OK 2026-06-09 20:43:46.392966 | controller | - Zookeeper: OK 2026-06-09 20:43:46.392973 | controller | - Zuul: Waiting ... 2026-06-09 20:43:46.392981 | controller | - |_ Config Secrets: OK 2026-06-09 20:43:46.392989 | controller | - |_ Executor: OK 2026-06-09 20:43:46.392997 | controller | - |_ Merger: OK 2026-06-09 20:43:46.393005 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:43:46.393013 | controller | - |_ Web: OK 2026-06-09 20:43:46.393021 | controller | 2026-06-09 20:43:46.393029 | controller | 2026-06-09T20:43:46-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-09 20:43:51.401924 | controller | 2026-06-09T20:43:51-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:43:51.429718 | controller | 2026-06-09T20:43:51-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:43:51.448440 | controller | 2026-06-09T20:43:51-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:43:52.010076 | controller | 2026-06-09T20:43:52-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:43:52.021550 | controller | 2026-06-09T20:43:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:43:52.021623 | controller | 2026-06-09T20:43:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:43:52.031441 | controller | 2026-06-09T20:43:52-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:43:52.044271 | controller | 2026-06-09T20:43:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:43:52.044296 | controller | 2026-06-09T20:43:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:43:52.416913 | controller | 2026-06-09T20:43:52-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:43:52.419675 | controller | 2026-06-09T20:43:52-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:43:52.827345 | controller | 2026-06-09T20:43:52-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:43:52.827442 | controller | 2026-06-09T20:43:52-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:43:53.027296 | controller | 2026-06-09T20:43:53-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:43:53.819414 | controller | 2026-06-09T20:43:53-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:43:53.825250 | controller | 2026-06-09T20:43:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:43:53.825278 | controller | 2026-06-09T20:43:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:43:54.614990 | controller | 2026-06-09T20:43:54-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:43:55.406561 | controller | 2026-06-09T20:43:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:43:55.406651 | controller | 2026-06-09T20:43:55-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:43:56.213688 | controller | 2026-06-09T20:43:56-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:43:56.615455 | controller | 2026-06-09T20:43:56-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-09 20:43:56.802693 | controller | 2026-06-09T20:43:56-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-09 20:44:03.276270 | controller | 2026-06-09T20:44:03-04:00 INFO Deleting object, name: zuul-web 2026-06-09 20:44:03.281261 | controller | 2026-06-09T20:44:03-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-09 20:44:03.285589 | controller | 2026-06-09T20:44:03-04:00 INFO 2026-06-09 20:44:03.285622 | controller | - Config: Waiting ... 2026-06-09 20:44:03.285635 | controller | - Gateway: OK 2026-06-09 20:44:03.285648 | controller | - GitServer: OK 2026-06-09 20:44:03.285659 | controller | - HoundSearch: OK 2026-06-09 20:44:03.285671 | controller | - LogJuicer: OK 2026-06-09 20:44:03.285682 | controller | - Logserver: OK 2026-06-09 20:44:03.285693 | controller | - MariaDB: OK 2026-06-09 20:44:03.285704 | controller | - NodePoolBuilder: OK 2026-06-09 20:44:03.285715 | controller | - NodePoolLauncher: OK 2026-06-09 20:44:03.285725 | controller | - Zookeeper: OK 2026-06-09 20:44:03.285736 | controller | - Zuul: OK 2026-06-09 20:44:03.285768 | controller | 2026-06-09 20:44:03.285851 | controller | 2026-06-09T20:44:03-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-09 20:44:08.292286 | controller | 2026-06-09T20:44:08-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:44:08.320796 | controller | 2026-06-09T20:44:08-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:44:08.343788 | controller | 2026-06-09T20:44:08-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:44:08.896001 | controller | 2026-06-09T20:44:08-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:44:08.902396 | controller | 2026-06-09T20:44:08-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:44:08.902446 | controller | 2026-06-09T20:44:08-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:44:08.907708 | controller | 2026-06-09T20:44:08-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:44:08.915095 | controller | 2026-06-09T20:44:08-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:44:08.915132 | controller | 2026-06-09T20:44:08-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:44:09.306537 | controller | 2026-06-09T20:44:09-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:44:09.309429 | controller | 2026-06-09T20:44:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:44:09.711836 | controller | 2026-06-09T20:44:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:44:09.711925 | controller | 2026-06-09T20:44:09-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:44:09.908037 | controller | 2026-06-09T20:44:09-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:44:10.712715 | controller | 2026-06-09T20:44:10-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:44:10.721989 | controller | 2026-06-09T20:44:10-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:44:10.722039 | controller | 2026-06-09T20:44:10-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:44:11.504192 | controller | 2026-06-09T20:44:11-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:44:12.293227 | controller | 2026-06-09T20:44:12-04:00 INFO Creating object, name: zuul-web 2026-06-09 20:44:13.101041 | controller | 2026-06-09T20:44:13-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:44:13.105326 | controller | 2026-06-09T20:44:13-04:00 INFO 2026-06-09 20:44:13.105357 | controller | - Config: Waiting ... 2026-06-09 20:44:13.105370 | controller | - Gateway: OK 2026-06-09 20:44:13.105379 | controller | - GitServer: OK 2026-06-09 20:44:13.105388 | controller | - HoundSearch: OK 2026-06-09 20:44:13.105397 | controller | - LogJuicer: OK 2026-06-09 20:44:13.105405 | controller | - Logserver: OK 2026-06-09 20:44:13.105414 | controller | - MariaDB: OK 2026-06-09 20:44:13.105422 | controller | - NodePoolBuilder: OK 2026-06-09 20:44:13.105431 | controller | - NodePoolLauncher: OK 2026-06-09 20:44:13.105439 | controller | - Zookeeper: OK 2026-06-09 20:44:13.105448 | controller | - Zuul: Waiting ... 2026-06-09 20:44:13.105456 | controller | - |_ Config Secrets: OK 2026-06-09 20:44:13.105465 | controller | - |_ Executor: OK 2026-06-09 20:44:13.105473 | controller | - |_ Merger: OK 2026-06-09 20:44:13.105482 | controller | - |_ Scheduler: OK 2026-06-09 20:44:13.105490 | controller | - |_ Web: Waiting ... 2026-06-09 20:44:13.105499 | controller | 2026-06-09 20:44:13.105569 | controller | 2026-06-09T20:44:13-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-09 20:44:18.111124 | controller | 2026-06-09T20:44:18-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:44:18.136044 | controller | 2026-06-09T20:44:18-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:44:18.154184 | controller | 2026-06-09T20:44:18-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:44:18.721022 | controller | 2026-06-09T20:44:18-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:44:18.729272 | controller | 2026-06-09T20:44:18-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:44:18.729323 | controller | 2026-06-09T20:44:18-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:44:18.736861 | controller | 2026-06-09T20:44:18-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:44:18.746436 | controller | 2026-06-09T20:44:18-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:44:18.746480 | controller | 2026-06-09T20:44:18-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:44:19.128445 | controller | 2026-06-09T20:44:19-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:44:19.132942 | controller | 2026-06-09T20:44:19-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:44:19.526702 | controller | 2026-06-09T20:44:19-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:44:19.526780 | controller | 2026-06-09T20:44:19-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:44:19.722358 | controller | 2026-06-09T20:44:19-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:44:20.531483 | controller | 2026-06-09T20:44:20-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:44:20.538316 | controller | 2026-06-09T20:44:20-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:44:20.538352 | controller | 2026-06-09T20:44:20-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:44:21.323074 | controller | 2026-06-09T20:44:21-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:44:22.114615 | controller | 2026-06-09T20:44:22-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:44:22.114682 | controller | 2026-06-09T20:44:22-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:44:22.922379 | controller | 2026-06-09T20:44:22-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:44:22.926937 | controller | 2026-06-09T20:44:22-04:00 INFO 2026-06-09 20:44:22.926988 | controller | - Config: Waiting ... 2026-06-09 20:44:22.927003 | controller | - Gateway: OK 2026-06-09 20:44:22.927012 | controller | - GitServer: OK 2026-06-09 20:44:22.927020 | controller | - HoundSearch: OK 2026-06-09 20:44:22.927028 | controller | - LogJuicer: OK 2026-06-09 20:44:22.927036 | controller | - Logserver: OK 2026-06-09 20:44:22.927043 | controller | - MariaDB: OK 2026-06-09 20:44:22.927051 | controller | - NodePoolBuilder: OK 2026-06-09 20:44:22.927059 | controller | - NodePoolLauncher: OK 2026-06-09 20:44:22.927067 | controller | - Zookeeper: OK 2026-06-09 20:44:22.927074 | controller | - Zuul: Waiting ... 2026-06-09 20:44:22.927082 | controller | - |_ Config Secrets: OK 2026-06-09 20:44:22.927107 | controller | - |_ Executor: OK 2026-06-09 20:44:22.927115 | controller | - |_ Merger: OK 2026-06-09 20:44:22.927123 | controller | - |_ Scheduler: OK 2026-06-09 20:44:22.927131 | controller | - |_ Web: Waiting ... 2026-06-09 20:44:22.927139 | controller | 2026-06-09 20:44:22.927147 | controller | 2026-06-09T20:44:22-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-09 20:44:27.937045 | controller | 2026-06-09T20:44:27-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:44:27.965393 | controller | 2026-06-09T20:44:27-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:44:27.988538 | controller | 2026-06-09T20:44:27-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:44:28.544182 | controller | 2026-06-09T20:44:28-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:44:28.554450 | controller | 2026-06-09T20:44:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:44:28.554489 | controller | 2026-06-09T20:44:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:44:28.563330 | controller | 2026-06-09T20:44:28-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:44:28.575092 | controller | 2026-06-09T20:44:28-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:44:28.575169 | controller | 2026-06-09T20:44:28-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:44:28.953093 | controller | 2026-06-09T20:44:28-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:44:28.956943 | controller | 2026-06-09T20:44:28-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:44:29.354354 | controller | 2026-06-09T20:44:29-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:44:29.354407 | controller | 2026-06-09T20:44:29-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:44:29.552975 | controller | 2026-06-09T20:44:29-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:44:30.352920 | controller | 2026-06-09T20:44:30-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:44:30.361638 | controller | 2026-06-09T20:44:30-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:44:30.361736 | controller | 2026-06-09T20:44:30-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:44:31.150382 | controller | 2026-06-09T20:44:31-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:44:31.941760 | controller | 2026-06-09T20:44:31-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:44:31.941818 | controller | 2026-06-09T20:44:31-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:44:32.745120 | controller | 2026-06-09T20:44:32-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:44:32.747781 | controller | 2026-06-09T20:44:32-04:00 INFO 2026-06-09 20:44:32.747827 | controller | - Config: Waiting ... 2026-06-09 20:44:32.747842 | controller | - Gateway: OK 2026-06-09 20:44:32.747853 | controller | - GitServer: OK 2026-06-09 20:44:32.747863 | controller | - HoundSearch: OK 2026-06-09 20:44:32.747872 | controller | - LogJuicer: OK 2026-06-09 20:44:32.747879 | controller | - Logserver: OK 2026-06-09 20:44:32.747887 | controller | - MariaDB: OK 2026-06-09 20:44:32.747895 | controller | - NodePoolBuilder: OK 2026-06-09 20:44:32.747902 | controller | - NodePoolLauncher: OK 2026-06-09 20:44:32.747910 | controller | - Zookeeper: OK 2026-06-09 20:44:32.747918 | controller | - Zuul: Waiting ... 2026-06-09 20:44:32.747925 | controller | - |_ Config Secrets: OK 2026-06-09 20:44:32.747933 | controller | - |_ Executor: OK 2026-06-09 20:44:32.747941 | controller | - |_ Merger: OK 2026-06-09 20:44:32.747948 | controller | - |_ Scheduler: OK 2026-06-09 20:44:32.747956 | controller | - |_ Web: Waiting ... 2026-06-09 20:44:32.747964 | controller | 2026-06-09 20:44:32.747978 | controller | 2026-06-09T20:44:32-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-09 20:44:37.756121 | controller | 2026-06-09T20:44:37-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:44:37.774525 | controller | 2026-06-09T20:44:37-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:44:37.793999 | controller | 2026-06-09T20:44:37-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:44:38.367016 | controller | 2026-06-09T20:44:38-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:44:38.379490 | controller | 2026-06-09T20:44:38-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:44:38.379564 | controller | 2026-06-09T20:44:38-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:44:38.389999 | controller | 2026-06-09T20:44:38-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:44:38.403793 | controller | 2026-06-09T20:44:38-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:44:38.403847 | controller | 2026-06-09T20:44:38-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:44:38.776967 | controller | 2026-06-09T20:44:38-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:44:38.779828 | controller | 2026-06-09T20:44:38-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:44:39.178282 | controller | 2026-06-09T20:44:39-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:44:39.178363 | controller | 2026-06-09T20:44:39-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:44:39.370752 | controller | 2026-06-09T20:44:39-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:44:40.173536 | controller | 2026-06-09T20:44:40-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:44:40.182820 | controller | 2026-06-09T20:44:40-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:44:40.182869 | controller | 2026-06-09T20:44:40-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:44:40.968691 | controller | 2026-06-09T20:44:40-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:44:41.762042 | controller | 2026-06-09T20:44:41-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:44:41.762138 | controller | 2026-06-09T20:44:41-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:44:42.568904 | controller | 2026-06-09T20:44:42-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:44:42.574185 | controller | 2026-06-09T20:44:42-04:00 INFO 2026-06-09 20:44:42.574281 | controller | - Config: Waiting ... 2026-06-09 20:44:42.574298 | controller | - Gateway: OK 2026-06-09 20:44:42.574308 | controller | - GitServer: OK 2026-06-09 20:44:42.574316 | controller | - HoundSearch: OK 2026-06-09 20:44:42.574324 | controller | - LogJuicer: OK 2026-06-09 20:44:42.574331 | controller | - Logserver: OK 2026-06-09 20:44:42.574339 | controller | - MariaDB: OK 2026-06-09 20:44:42.574346 | controller | - NodePoolBuilder: OK 2026-06-09 20:44:42.574354 | controller | - NodePoolLauncher: OK 2026-06-09 20:44:42.574362 | controller | - Zookeeper: OK 2026-06-09 20:44:42.574369 | controller | - Zuul: Waiting ... 2026-06-09 20:44:42.574377 | controller | - |_ Config Secrets: OK 2026-06-09 20:44:42.574384 | controller | - |_ Executor: OK 2026-06-09 20:44:42.574392 | controller | - |_ Merger: OK 2026-06-09 20:44:42.574399 | controller | - |_ Scheduler: OK 2026-06-09 20:44:42.574407 | controller | - |_ Web: Waiting ... 2026-06-09 20:44:42.574415 | controller | 2026-06-09 20:44:42.574423 | controller | 2026-06-09T20:44:42-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-09 20:44:47.582531 | controller | 2026-06-09T20:44:47-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:44:47.611671 | controller | 2026-06-09T20:44:47-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:44:47.630946 | controller | 2026-06-09T20:44:47-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:44:48.192099 | controller | 2026-06-09T20:44:48-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:44:48.203957 | controller | 2026-06-09T20:44:48-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:44:48.204022 | controller | 2026-06-09T20:44:48-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:44:48.213599 | controller | 2026-06-09T20:44:48-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:44:48.224438 | controller | 2026-06-09T20:44:48-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:44:48.224487 | controller | 2026-06-09T20:44:48-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:44:48.599133 | controller | 2026-06-09T20:44:48-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:44:48.602325 | controller | 2026-06-09T20:44:48-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:44:49.006508 | controller | 2026-06-09T20:44:49-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:44:49.006562 | controller | 2026-06-09T20:44:49-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:44:49.198980 | controller | 2026-06-09T20:44:49-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:44:50.007607 | controller | 2026-06-09T20:44:50-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:44:50.017149 | controller | 2026-06-09T20:44:50-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:44:50.017269 | controller | 2026-06-09T20:44:50-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:44:50.792333 | controller | 2026-06-09T20:44:50-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:44:51.587175 | controller | 2026-06-09T20:44:51-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:44:51.587291 | controller | 2026-06-09T20:44:51-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:44:52.392487 | controller | 2026-06-09T20:44:52-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:44:52.397181 | controller | 2026-06-09T20:44:52-04:00 INFO 2026-06-09 20:44:52.397268 | controller | - Config: Waiting ... 2026-06-09 20:44:52.397284 | controller | - Gateway: OK 2026-06-09 20:44:52.397293 | controller | - GitServer: OK 2026-06-09 20:44:52.397301 | controller | - HoundSearch: OK 2026-06-09 20:44:52.397309 | controller | - LogJuicer: OK 2026-06-09 20:44:52.397316 | controller | - Logserver: OK 2026-06-09 20:44:52.397324 | controller | - MariaDB: OK 2026-06-09 20:44:52.397332 | controller | - NodePoolBuilder: OK 2026-06-09 20:44:52.397339 | controller | - NodePoolLauncher: OK 2026-06-09 20:44:52.397347 | controller | - Zookeeper: OK 2026-06-09 20:44:52.397355 | controller | - Zuul: Waiting ... 2026-06-09 20:44:52.397363 | controller | - |_ Config Secrets: OK 2026-06-09 20:44:52.397371 | controller | - |_ Executor: OK 2026-06-09 20:44:52.397378 | controller | - |_ Merger: OK 2026-06-09 20:44:52.397386 | controller | - |_ Scheduler: OK 2026-06-09 20:44:52.397394 | controller | - |_ Web: Waiting ... 2026-06-09 20:44:52.397402 | controller | 2026-06-09 20:44:52.397418 | controller | 2026-06-09T20:44:52-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-09 20:44:57.406053 | controller | 2026-06-09T20:44:57-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:44:57.435013 | controller | 2026-06-09T20:44:57-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:44:57.455874 | controller | 2026-06-09T20:44:57-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:44:58.017040 | controller | 2026-06-09T20:44:58-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:44:58.031099 | controller | 2026-06-09T20:44:58-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:44:58.031158 | controller | 2026-06-09T20:44:58-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:44:58.041928 | controller | 2026-06-09T20:44:58-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:44:58.054590 | controller | 2026-06-09T20:44:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:44:58.054638 | controller | 2026-06-09T20:44:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:44:58.421013 | controller | 2026-06-09T20:44:58-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:44:58.423917 | controller | 2026-06-09T20:44:58-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:44:58.822314 | controller | 2026-06-09T20:44:58-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:44:58.822408 | controller | 2026-06-09T20:44:58-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:44:59.021780 | controller | 2026-06-09T20:44:59-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:44:59.822854 | controller | 2026-06-09T20:44:59-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:44:59.830275 | controller | 2026-06-09T20:44:59-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:44:59.830321 | controller | 2026-06-09T20:44:59-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:45:00.614782 | controller | 2026-06-09T20:45:00-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:45:01.407936 | controller | 2026-06-09T20:45:01-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:45:01.407991 | controller | 2026-06-09T20:45:01-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:45:02.214550 | controller | 2026-06-09T20:45:02-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:45:02.217155 | controller | 2026-06-09T20:45:02-04:00 INFO 2026-06-09 20:45:02.217181 | controller | - Config: Waiting ... 2026-06-09 20:45:02.217185 | controller | - Gateway: OK 2026-06-09 20:45:02.217189 | controller | - GitServer: OK 2026-06-09 20:45:02.217193 | controller | - HoundSearch: OK 2026-06-09 20:45:02.217208 | controller | - LogJuicer: OK 2026-06-09 20:45:02.217212 | controller | - Logserver: OK 2026-06-09 20:45:02.217215 | controller | - MariaDB: OK 2026-06-09 20:45:02.217219 | controller | - NodePoolBuilder: OK 2026-06-09 20:45:02.217222 | controller | - NodePoolLauncher: OK 2026-06-09 20:45:02.217226 | controller | - Zookeeper: OK 2026-06-09 20:45:02.217252 | controller | - Zuul: Waiting ... 2026-06-09 20:45:02.217261 | controller | - |_ Config Secrets: OK 2026-06-09 20:45:02.217267 | controller | - |_ Executor: OK 2026-06-09 20:45:02.217271 | controller | - |_ Merger: OK 2026-06-09 20:45:02.217274 | controller | - |_ Scheduler: OK 2026-06-09 20:45:02.217278 | controller | - |_ Web: Waiting ... 2026-06-09 20:45:02.217282 | controller | 2026-06-09 20:45:02.217320 | controller | 2026-06-09T20:45:02-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-09 20:45:07.227226 | controller | 2026-06-09T20:45:07-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:45:07.253826 | controller | 2026-06-09T20:45:07-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:45:07.276346 | controller | 2026-06-09T20:45:07-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:45:07.833639 | controller | 2026-06-09T20:45:07-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:45:07.842650 | controller | 2026-06-09T20:45:07-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:45:07.842693 | controller | 2026-06-09T20:45:07-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:45:07.851178 | controller | 2026-06-09T20:45:07-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:45:07.860387 | controller | 2026-06-09T20:45:07-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:45:07.860425 | controller | 2026-06-09T20:45:07-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:45:08.242476 | controller | 2026-06-09T20:45:08-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:45:08.245898 | controller | 2026-06-09T20:45:08-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:45:08.649301 | controller | 2026-06-09T20:45:08-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:45:08.649396 | controller | 2026-06-09T20:45:08-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:45:08.840064 | controller | 2026-06-09T20:45:08-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:45:09.646944 | controller | 2026-06-09T20:45:09-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:45:09.655727 | controller | 2026-06-09T20:45:09-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:45:09.655766 | controller | 2026-06-09T20:45:09-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:45:10.438281 | controller | 2026-06-09T20:45:10-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:45:11.233336 | controller | 2026-06-09T20:45:11-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:45:11.233390 | controller | 2026-06-09T20:45:11-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:45:12.037587 | controller | 2026-06-09T20:45:12-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:45:12.041296 | controller | 2026-06-09T20:45:12-04:00 INFO 2026-06-09 20:45:12.041323 | controller | - Config: Waiting ... 2026-06-09 20:45:12.041333 | controller | - Gateway: OK 2026-06-09 20:45:12.041342 | controller | - GitServer: OK 2026-06-09 20:45:12.041350 | controller | - HoundSearch: OK 2026-06-09 20:45:12.041358 | controller | - LogJuicer: OK 2026-06-09 20:45:12.041366 | controller | - Logserver: OK 2026-06-09 20:45:12.041373 | controller | - MariaDB: OK 2026-06-09 20:45:12.041381 | controller | - NodePoolBuilder: OK 2026-06-09 20:45:12.041389 | controller | - NodePoolLauncher: OK 2026-06-09 20:45:12.041397 | controller | - Zookeeper: OK 2026-06-09 20:45:12.041404 | controller | - Zuul: Waiting ... 2026-06-09 20:45:12.041412 | controller | - |_ Config Secrets: OK 2026-06-09 20:45:12.041420 | controller | - |_ Executor: OK 2026-06-09 20:45:12.041428 | controller | - |_ Merger: OK 2026-06-09 20:45:12.041436 | controller | - |_ Scheduler: OK 2026-06-09 20:45:12.041443 | controller | - |_ Web: Waiting ... 2026-06-09 20:45:12.041452 | controller | 2026-06-09 20:45:12.041460 | controller | 2026-06-09T20:45:12-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-09 20:45:17.050570 | controller | 2026-06-09T20:45:17-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:45:17.080706 | controller | 2026-06-09T20:45:17-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:45:17.102868 | controller | 2026-06-09T20:45:17-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:45:17.660863 | controller | 2026-06-09T20:45:17-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:45:17.673345 | controller | 2026-06-09T20:45:17-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:45:17.673394 | controller | 2026-06-09T20:45:17-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:45:17.685336 | controller | 2026-06-09T20:45:17-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:45:17.697686 | controller | 2026-06-09T20:45:17-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:45:17.697730 | controller | 2026-06-09T20:45:17-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:45:18.059131 | controller | 2026-06-09T20:45:18-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:45:18.061495 | controller | 2026-06-09T20:45:18-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:45:18.463000 | controller | 2026-06-09T20:45:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:45:18.463076 | controller | 2026-06-09T20:45:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:45:18.657341 | controller | 2026-06-09T20:45:18-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:45:19.467833 | controller | 2026-06-09T20:45:19-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:45:19.475031 | controller | 2026-06-09T20:45:19-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:45:19.475083 | controller | 2026-06-09T20:45:19-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:45:20.260064 | controller | 2026-06-09T20:45:20-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:45:21.051881 | controller | 2026-06-09T20:45:21-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:45:21.051922 | controller | 2026-06-09T20:45:21-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:45:21.858590 | controller | 2026-06-09T20:45:21-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:45:22.264522 | controller | 2026-06-09T20:45:22-04:00 INFO 2026-06-09 20:45:22.264588 | controller | - Config: OK 2026-06-09 20:45:22.264599 | controller | - Gateway: OK 2026-06-09 20:45:22.264608 | controller | - GitServer: OK 2026-06-09 20:45:22.264615 | controller | - HoundSearch: OK 2026-06-09 20:45:22.264623 | controller | - LogJuicer: OK 2026-06-09 20:45:22.264631 | controller | - Logserver: OK 2026-06-09 20:45:22.264639 | controller | - MariaDB: OK 2026-06-09 20:45:22.264647 | controller | - NodePoolBuilder: OK 2026-06-09 20:45:22.264655 | controller | - NodePoolLauncher: OK 2026-06-09 20:45:22.264662 | controller | - Zookeeper: OK 2026-06-09 20:45:22.264670 | controller | - Zuul: OK 2026-06-09 20:45:22.264678 | controller | 2026-06-09 20:45:22.264686 | controller | 2026-06-09T20:45:22-04:00 INFO Updating controller configmap ... 2026-06-09 20:45:22.654223 | controller | 2026-06-09T20:45:22-04:00 INFO Standalone reconcile done. 2026-06-10 00:45:23.001508 | controller | ok: Runtime: 0:03:12.663975 2026-06-10 00:45:23.017945 | 2026-06-10 00:45:23.018076 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:45:23.555356 | controller | skipping: Conditional result was False 2026-06-10 00:45:23.583096 | 2026-06-10 00:45:23.583331 | TASK [health-check/zuul-connections : Wait for the dummy Zuul connections to be removed from the API] 2026-06-10 00:45:24.151258 | controller | ok: OK (415 bytes) 2026-06-10 00:45:24.162174 | 2026-06-10 00:45:24.162312 | TASK [health-check/zuul-connections : Ensure the new Zuul connections (non SourceInterface) no longer exist in the scheduler's zuul.conf] 2026-06-09 20:45:24.554562 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-09 20:45:24.691732 | controller | command terminated with exit code 1 2026-06-09 20:45:24.790183 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-09 20:45:24.975645 | controller | command terminated with exit code 1 2026-06-10 00:45:25.207384 | controller | ok: Runtime: 0:00:00.527543 2026-06-10 00:45:25.217657 | 2026-06-10 00:45:25.217858 | TASK [health-check/zuul-connections : Delete GitHub Connection Secret] 2026-06-10 00:45:26.022908 | controller | changed 2026-06-10 00:45:26.032619 | 2026-06-10 00:45:26.032757 | TASK [health-check/zuul-connections : Delete GitLab Connection Secret] 2026-06-10 00:45:26.877366 | controller | changed 2026-06-10 00:45:26.887234 | 2026-06-10 00:45:26.887367 | TASK [health-check/zuul-connections : Delete Pagure Connection Secret] 2026-06-10 00:45:27.697734 | controller | changed 2026-06-10 00:45:27.721871 | 2026-06-10 00:45:27.722093 | TASK [health-check/zuul-authenticators : Initialize new authenticator config] 2026-06-10 00:45:27.757633 | controller | ok 2026-06-10 00:45:27.763504 | 2026-06-10 00:45:27.763615 | TASK [Add the authenticator to spec] 2026-06-10 00:45:27.794853 | controller | ok 2026-06-10 00:45:27.814811 | 2026-06-10 00:45:27.814911 | TASK [update-custom-resource : read current crd] 2026-06-10 00:45:28.080604 | controller | ok 2026-06-10 00:45:28.089483 | 2026-06-10 00:45:28.089647 | TASK [update-custom-resource : merge the new cr] 2026-06-10 00:45:28.915480 | controller | changed 2026-06-10 00:45:28.925113 | 2026-06-10 00:45:28.925238 | TASK [run the operator in standalone mode] 2026-06-10 00:45:28.959707 | controller | ok 2026-06-10 00:45:28.993405 | 2026-06-10 00:45:28.993497 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:45:29.018404 | controller | skipping: Conditional result was False 2026-06-10 00:45:29.026147 | 2026-06-10 00:45:29.026239 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:45:29.051167 | controller | skipping: Conditional result was False 2026-06-10 00:45:29.059200 | 2026-06-10 00:45:29.059288 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:45:29.615248 | controller | 2026-06-09T20:45:29-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:45:29.615318 | controller | 2026-06-09 20:45:29.624419 | controller | 2026-06-09T20:45:29-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:45:29.624452 | controller | 2026-06-09 20:45:29.634349 | controller | 2026-06-09T20:45:29-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:45:29.652402 | controller | 2026-06-09T20:45:29-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:45:29.666011 | controller | 2026-06-09T20:45:29-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:45:30.240913 | controller | 2026-06-09T20:45:30-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:45:30.245024 | controller | 2026-06-09T20:45:30-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:45:30.346074 | controller | 2026-06-09T20:45:30-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:45:30.352160 | controller | 2026-06-09T20:45:30-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:45:30.352196 | controller | 2026-06-09T20:45:30-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:45:30.359921 | controller | 2026-06-09T20:45:30-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:45:30.368102 | controller | 2026-06-09T20:45:30-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:45:30.368140 | controller | 2026-06-09T20:45:30-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:45:30.650479 | controller | 2026-06-09T20:45:30-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:45:30.653124 | controller | 2026-06-09T20:45:30-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:45:31.064028 | controller | 2026-06-09T20:45:31-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:45:31.064115 | controller | 2026-06-09T20:45:31-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:45:31.442564 | controller | 2026-06-09T20:45:31-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:45:32.035379 | controller | 2026-06-09T20:45:32-04:00 INFO Updating secret, name: zuul-config 2026-06-09 20:45:32.035456 | controller | 2026-06-09T20:45:32-04:00 INFO Updating object, name: zuul-config 2026-06-09 20:45:32.270316 | controller | 2026-06-09T20:45:32-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 20:45:32.270407 | controller | 2026-06-09T20:45:32-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 20:45:33.042324 | controller | 2026-06-09T20:45:33-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:45:33.835808 | controller | 2026-06-09T20:45:33-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 20:45:33.835934 | controller | 2026-06-09T20:45:33-04:00 INFO Updating object, name: zuul-web 2026-06-09 20:45:34.650473 | controller | 2026-06-09T20:45:34-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:45:34.656667 | controller | 2026-06-09T20:45:34-04:00 INFO 2026-06-09 20:45:34.656809 | controller | - Config: Waiting ... 2026-06-09 20:45:34.656821 | controller | - Gateway: OK 2026-06-09 20:45:34.656831 | controller | - GitServer: OK 2026-06-09 20:45:34.656840 | controller | - HoundSearch: OK 2026-06-09 20:45:34.656849 | controller | - LogJuicer: OK 2026-06-09 20:45:34.656858 | controller | - Logserver: OK 2026-06-09 20:45:34.656866 | controller | - MariaDB: OK 2026-06-09 20:45:34.656875 | controller | - NodePoolBuilder: OK 2026-06-09 20:45:34.656884 | controller | - NodePoolLauncher: OK 2026-06-09 20:45:34.656896 | controller | - Zookeeper: OK 2026-06-09 20:45:34.656907 | controller | - Zuul: Waiting ... 2026-06-09 20:45:34.656920 | controller | - |_ Config Secrets: OK 2026-06-09 20:45:34.656964 | controller | - |_ Executor: OK 2026-06-09 20:45:34.656978 | controller | - |_ Merger: OK 2026-06-09 20:45:34.656991 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:45:34.657003 | controller | - |_ Web: Waiting ... 2026-06-09 20:45:34.657015 | controller | 2026-06-09 20:45:34.657027 | controller | 2026-06-09T20:45:34-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:45:39.666879 | controller | 2026-06-09T20:45:39-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:45:39.700934 | controller | 2026-06-09T20:45:39-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:45:39.739314 | controller | 2026-06-09T20:45:39-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:45:40.272641 | controller | 2026-06-09T20:45:40-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:45:40.282424 | controller | 2026-06-09T20:45:40-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:45:40.282462 | controller | 2026-06-09T20:45:40-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:45:40.291847 | controller | 2026-06-09T20:45:40-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:45:40.385414 | controller | 2026-06-09T20:45:40-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:45:40.385471 | controller | 2026-06-09T20:45:40-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:45:40.687331 | controller | 2026-06-09T20:45:40-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:45:40.690294 | controller | 2026-06-09T20:45:40-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:45:41.081729 | controller | 2026-06-09T20:45:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:45:41.081823 | controller | 2026-06-09T20:45:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:45:41.288126 | controller | 2026-06-09T20:45:41-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:45:42.092304 | controller | 2026-06-09T20:45:42-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:45:42.092342 | controller | 2026-06-09T20:45:42-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:45:47.103416 | controller | 2026-06-09T20:45:47-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:45:47.103487 | controller | 2026-06-09T20:45:47-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:45:52.113013 | controller | 2026-06-09T20:45:52-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:45:52.113075 | controller | 2026-06-09T20:45:52-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:45:57.124428 | controller | 2026-06-09T20:45:57-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:45:57.124484 | controller | 2026-06-09T20:45:57-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:46:02.136855 | controller | 2026-06-09T20:46:02-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:46:02.136897 | controller | 2026-06-09T20:46:02-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:46:07.144917 | controller | 2026-06-09T20:46:07-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:46:07.144993 | controller | 2026-06-09T20:46:07-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:46:12.151271 | controller | 2026-06-09T20:46:12-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:46:12.178952 | controller | 2026-06-09T20:46:12-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:46:12.195521 | controller | 2026-06-09T20:46:12-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:46:12.195576 | controller | 2026-06-09T20:46:12-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:46:12.565334 | controller | 2026-06-09T20:46:12-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:46:12.568375 | controller | 2026-06-09T20:46:12-04:00 INFO 2026-06-09 20:46:12.568400 | controller | - Config: Waiting ... 2026-06-09 20:46:12.568407 | controller | - Gateway: OK 2026-06-09 20:46:12.568413 | controller | - GitServer: OK 2026-06-09 20:46:12.568418 | controller | - HoundSearch: OK 2026-06-09 20:46:12.568424 | controller | - LogJuicer: OK 2026-06-09 20:46:12.568429 | controller | - Logserver: OK 2026-06-09 20:46:12.568434 | controller | - MariaDB: OK 2026-06-09 20:46:12.568439 | controller | - NodePoolBuilder: OK 2026-06-09 20:46:12.568457 | controller | - NodePoolLauncher: OK 2026-06-09 20:46:12.568463 | controller | - Zookeeper: OK 2026-06-09 20:46:12.568468 | controller | - Zuul: Waiting ... 2026-06-09 20:46:12.568474 | controller | - |_ Config Secrets: OK 2026-06-09 20:46:12.568479 | controller | - |_ Executor: OK 2026-06-09 20:46:12.568484 | controller | - |_ Merger: OK 2026-06-09 20:46:12.568489 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:46:12.568495 | controller | - |_ Web: Waiting ... 2026-06-09 20:46:12.568500 | controller | 2026-06-09 20:46:12.568508 | controller | 2026-06-09T20:46:12-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:46:17.577975 | controller | 2026-06-09T20:46:17-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:46:17.603280 | controller | 2026-06-09T20:46:17-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:46:17.627503 | controller | 2026-06-09T20:46:17-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:46:18.185304 | controller | 2026-06-09T20:46:18-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:46:18.195708 | controller | 2026-06-09T20:46:18-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:46:18.195794 | controller | 2026-06-09T20:46:18-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:46:18.206657 | controller | 2026-06-09T20:46:18-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:46:18.218255 | controller | 2026-06-09T20:46:18-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:46:18.218304 | controller | 2026-06-09T20:46:18-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:46:18.593405 | controller | 2026-06-09T20:46:18-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:46:18.597993 | controller | 2026-06-09T20:46:18-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:46:18.990858 | controller | 2026-06-09T20:46:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:46:18.990945 | controller | 2026-06-09T20:46:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:46:19.196049 | controller | 2026-06-09T20:46:19-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:46:19.997141 | controller | 2026-06-09T20:46:19-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:46:19.997203 | controller | 2026-06-09T20:46:19-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:46:25.005939 | controller | 2026-06-09T20:46:25-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:46:25.006035 | controller | 2026-06-09T20:46:25-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:46:30.017816 | controller | 2026-06-09T20:46:30-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:46:30.017910 | controller | 2026-06-09T20:46:30-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:46:35.026765 | controller | 2026-06-09T20:46:35-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:46:35.026819 | controller | 2026-06-09T20:46:35-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:46:40.040730 | controller | 2026-06-09T20:46:40-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:46:40.049843 | controller | 2026-06-09T20:46:40-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:46:40.049898 | controller | 2026-06-09T20:46:40-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:46:40.065925 | controller | 2026-06-09T20:46:40-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:46:40.079639 | controller | 2026-06-09T20:46:40-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:46:40.079702 | controller | 2026-06-09T20:46:40-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:46:40.464091 | controller | 2026-06-09T20:46:40-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:46:40.867553 | controller | 2026-06-09T20:46:40-04:00 INFO 2026-06-09 20:46:40.867626 | controller | - Config: OK 2026-06-09 20:46:40.867639 | controller | - Gateway: OK 2026-06-09 20:46:40.867649 | controller | - GitServer: OK 2026-06-09 20:46:40.867659 | controller | - HoundSearch: OK 2026-06-09 20:46:40.867668 | controller | - LogJuicer: OK 2026-06-09 20:46:40.867677 | controller | - Logserver: OK 2026-06-09 20:46:40.867686 | controller | - MariaDB: OK 2026-06-09 20:46:40.867695 | controller | - NodePoolBuilder: OK 2026-06-09 20:46:40.867704 | controller | - NodePoolLauncher: OK 2026-06-09 20:46:40.867713 | controller | - Zookeeper: OK 2026-06-09 20:46:40.867722 | controller | - Zuul: OK 2026-06-09 20:46:40.867731 | controller | 2026-06-09 20:46:40.867741 | controller | 2026-06-09T20:46:40-04:00 INFO Updating controller configmap ... 2026-06-09 20:46:41.254597 | controller | 2026-06-09T20:46:41-04:00 INFO Standalone reconcile done. 2026-06-10 00:46:41.686461 | controller | ok: Runtime: 0:01:11.927361 2026-06-10 00:46:41.699549 | 2026-06-10 00:46:41.699793 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:46:42.240774 | controller | skipping: Conditional result was False 2026-06-10 00:46:42.266063 | 2026-06-10 00:46:42.266292 | TASK [health-check/zuul-authenticators : Wait for the new authenticator to appear in the Zuul API] 2026-06-10 00:46:42.794700 | controller | ok: OK (589 bytes) 2026-06-10 00:46:42.801174 | 2026-06-10 00:46:42.801264 | TASK [Remove the test authenticator from the spec] 2026-06-10 00:46:42.830976 | controller | ok 2026-06-10 00:46:42.853225 | 2026-06-10 00:46:42.853316 | TASK [update-custom-resource : read current crd] 2026-06-10 00:46:43.108514 | controller | ok 2026-06-10 00:46:43.117287 | 2026-06-10 00:46:43.117423 | TASK [update-custom-resource : merge the new cr] 2026-06-10 00:46:43.829333 | controller | changed 2026-06-10 00:46:43.836415 | 2026-06-10 00:46:43.836507 | TASK [run the operator in standalone mode] 2026-06-10 00:46:43.873011 | controller | ok 2026-06-10 00:46:43.906471 | 2026-06-10 00:46:43.906676 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:46:43.931998 | controller | skipping: Conditional result was False 2026-06-10 00:46:43.939752 | 2026-06-10 00:46:43.939844 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:46:43.966296 | controller | skipping: Conditional result was False 2026-06-10 00:46:43.974184 | 2026-06-10 00:46:43.974289 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:46:44.575533 | controller | 2026-06-09T20:46:44-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:46:44.575598 | controller | 2026-06-09 20:46:44.589051 | controller | 2026-06-09T20:46:44-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:46:44.589079 | controller | 2026-06-09 20:46:44.602059 | controller | 2026-06-09T20:46:44-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:46:44.628655 | controller | 2026-06-09T20:46:44-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:46:44.647057 | controller | 2026-06-09T20:46:44-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:46:45.213054 | controller | 2026-06-09T20:46:45-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:46:45.219873 | controller | 2026-06-09T20:46:45-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:46:45.368073 | controller | 2026-06-09T20:46:45-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:46:45.377877 | controller | 2026-06-09T20:46:45-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:46:45.377925 | controller | 2026-06-09T20:46:45-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:46:45.389605 | controller | 2026-06-09T20:46:45-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:46:45.402113 | controller | 2026-06-09T20:46:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:46:45.402164 | controller | 2026-06-09T20:46:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:46:45.620078 | controller | 2026-06-09T20:46:45-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:46:45.622891 | controller | 2026-06-09T20:46:45-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:46:46.024467 | controller | 2026-06-09T20:46:46-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:46:46.024543 | controller | 2026-06-09T20:46:46-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:46:46.402529 | controller | 2026-06-09T20:46:46-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:46:47.003165 | controller | 2026-06-09T20:46:47-04:00 INFO Updating secret, name: zuul-config 2026-06-09 20:46:47.003271 | controller | 2026-06-09T20:46:47-04:00 INFO Updating object, name: zuul-config 2026-06-09 20:46:47.226942 | controller | 2026-06-09T20:46:47-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 20:46:47.227023 | controller | 2026-06-09T20:46:47-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 20:46:48.008105 | controller | 2026-06-09T20:46:48-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:46:48.803363 | controller | 2026-06-09T20:46:48-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 20:46:48.803447 | controller | 2026-06-09T20:46:48-04:00 INFO Updating object, name: zuul-web 2026-06-09 20:46:49.607919 | controller | 2026-06-09T20:46:49-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:46:49.610719 | controller | 2026-06-09T20:46:49-04:00 INFO 2026-06-09 20:46:49.610753 | controller | - Config: Waiting ... 2026-06-09 20:46:49.610765 | controller | - Gateway: OK 2026-06-09 20:46:49.610774 | controller | - GitServer: OK 2026-06-09 20:46:49.610783 | controller | - HoundSearch: OK 2026-06-09 20:46:49.610792 | controller | - LogJuicer: OK 2026-06-09 20:46:49.610801 | controller | - Logserver: OK 2026-06-09 20:46:49.610809 | controller | - MariaDB: OK 2026-06-09 20:46:49.610818 | controller | - NodePoolBuilder: OK 2026-06-09 20:46:49.610826 | controller | - NodePoolLauncher: OK 2026-06-09 20:46:49.610834 | controller | - Zookeeper: OK 2026-06-09 20:46:49.610843 | controller | - Zuul: Waiting ... 2026-06-09 20:46:49.610851 | controller | - |_ Config Secrets: OK 2026-06-09 20:46:49.610860 | controller | - |_ Executor: OK 2026-06-09 20:46:49.610869 | controller | - |_ Merger: OK 2026-06-09 20:46:49.610877 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:46:49.610886 | controller | - |_ Web: Waiting ... 2026-06-09 20:46:49.610895 | controller | 2026-06-09 20:46:49.610904 | controller | 2026-06-09T20:46:49-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:46:54.619739 | controller | 2026-06-09T20:46:54-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:46:54.646042 | controller | 2026-06-09T20:46:54-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:46:54.669666 | controller | 2026-06-09T20:46:54-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:46:55.227603 | controller | 2026-06-09T20:46:55-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:46:55.242645 | controller | 2026-06-09T20:46:55-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:46:55.242685 | controller | 2026-06-09T20:46:55-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:46:55.251980 | controller | 2026-06-09T20:46:55-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:46:55.266244 | controller | 2026-06-09T20:46:55-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:46:55.266284 | controller | 2026-06-09T20:46:55-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:46:55.657403 | controller | 2026-06-09T20:46:55-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:46:55.659525 | controller | 2026-06-09T20:46:55-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:46:56.040539 | controller | 2026-06-09T20:46:56-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:46:56.040596 | controller | 2026-06-09T20:46:56-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:46:56.240340 | controller | 2026-06-09T20:46:56-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:46:57.044217 | controller | 2026-06-09T20:46:57-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:46:57.044348 | controller | 2026-06-09T20:46:57-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:02.055646 | controller | 2026-06-09T20:47:02-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:02.055731 | controller | 2026-06-09T20:47:02-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:07.065122 | controller | 2026-06-09T20:47:07-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:07.065213 | controller | 2026-06-09T20:47:07-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:12.077481 | controller | 2026-06-09T20:47:12-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:12.077562 | controller | 2026-06-09T20:47:12-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:17.087433 | controller | 2026-06-09T20:47:17-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:17.087504 | controller | 2026-06-09T20:47:17-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:22.097475 | controller | 2026-06-09T20:47:22-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:22.097551 | controller | 2026-06-09T20:47:22-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:27.104170 | controller | 2026-06-09T20:47:27-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 20:47:27.127329 | controller | 2026-06-09T20:47:27-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:47:27.146270 | controller | 2026-06-09T20:47:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:47:27.146333 | controller | 2026-06-09T20:47:27-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 20:47:27.516548 | controller | 2026-06-09T20:47:27-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:47:27.519910 | controller | 2026-06-09T20:47:27-04:00 INFO 2026-06-09 20:47:27.519929 | controller | - Config: Waiting ... 2026-06-09 20:47:27.519936 | controller | - Gateway: OK 2026-06-09 20:47:27.519941 | controller | - GitServer: OK 2026-06-09 20:47:27.519946 | controller | - HoundSearch: OK 2026-06-09 20:47:27.519951 | controller | - LogJuicer: OK 2026-06-09 20:47:27.519956 | controller | - Logserver: OK 2026-06-09 20:47:27.519961 | controller | - MariaDB: OK 2026-06-09 20:47:27.519966 | controller | - NodePoolBuilder: OK 2026-06-09 20:47:27.519979 | controller | - NodePoolLauncher: OK 2026-06-09 20:47:27.519985 | controller | - Zookeeper: OK 2026-06-09 20:47:27.519990 | controller | - Zuul: Waiting ... 2026-06-09 20:47:27.519995 | controller | - |_ Config Secrets: OK 2026-06-09 20:47:27.520000 | controller | - |_ Executor: OK 2026-06-09 20:47:27.520006 | controller | - |_ Merger: OK 2026-06-09 20:47:27.520013 | controller | - |_ Scheduler: Waiting ... 2026-06-09 20:47:27.520018 | controller | - |_ Web: Waiting ... 2026-06-09 20:47:27.520023 | controller | 2026-06-09 20:47:27.520028 | controller | 2026-06-09T20:47:27-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:47:32.528979 | controller | 2026-06-09T20:47:32-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:47:32.553571 | controller | 2026-06-09T20:47:32-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:47:32.577714 | controller | 2026-06-09T20:47:32-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:47:33.136309 | controller | 2026-06-09T20:47:33-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:47:33.149669 | controller | 2026-06-09T20:47:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:47:33.149740 | controller | 2026-06-09T20:47:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:47:33.158925 | controller | 2026-06-09T20:47:33-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:47:33.171415 | controller | 2026-06-09T20:47:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:47:33.171485 | controller | 2026-06-09T20:47:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:47:33.544133 | controller | 2026-06-09T20:47:33-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:47:33.547089 | controller | 2026-06-09T20:47:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:47:33.946295 | controller | 2026-06-09T20:47:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:47:33.946346 | controller | 2026-06-09T20:47:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:47:34.146929 | controller | 2026-06-09T20:47:34-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:47:34.953611 | controller | 2026-06-09T20:47:34-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:34.953678 | controller | 2026-06-09T20:47:34-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:39.965493 | controller | 2026-06-09T20:47:39-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:39.965546 | controller | 2026-06-09T20:47:39-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:44.973004 | controller | 2026-06-09T20:47:44-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:44.973098 | controller | 2026-06-09T20:47:44-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:49.985542 | controller | 2026-06-09T20:47:49-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:49.985627 | controller | 2026-06-09T20:47:49-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 20:47:54.994849 | controller | 2026-06-09T20:47:54-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:47:55.005056 | controller | 2026-06-09T20:47:55-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:47:55.005122 | controller | 2026-06-09T20:47:55-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:47:55.022983 | controller | 2026-06-09T20:47:55-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:47:55.039696 | controller | 2026-06-09T20:47:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:47:55.039745 | controller | 2026-06-09T20:47:55-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:47:55.418447 | controller | 2026-06-09T20:47:55-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:47:55.820297 | controller | 2026-06-09T20:47:55-04:00 INFO 2026-06-09 20:47:55.820335 | controller | - Config: OK 2026-06-09 20:47:55.820340 | controller | - Gateway: OK 2026-06-09 20:47:55.820345 | controller | - GitServer: OK 2026-06-09 20:47:55.820348 | controller | - HoundSearch: OK 2026-06-09 20:47:55.820352 | controller | - LogJuicer: OK 2026-06-09 20:47:55.820356 | controller | - Logserver: OK 2026-06-09 20:47:55.820359 | controller | - MariaDB: OK 2026-06-09 20:47:55.820363 | controller | - NodePoolBuilder: OK 2026-06-09 20:47:55.820379 | controller | - NodePoolLauncher: OK 2026-06-09 20:47:55.820383 | controller | - Zookeeper: OK 2026-06-09 20:47:55.820387 | controller | - Zuul: OK 2026-06-09 20:47:55.820413 | controller | 2026-06-09 20:47:55.820417 | controller | 2026-06-09T20:47:55-04:00 INFO Updating controller configmap ... 2026-06-09 20:47:56.211156 | controller | 2026-06-09T20:47:56-04:00 INFO Standalone reconcile done. 2026-06-10 00:47:56.602569 | controller | ok: Runtime: 0:01:11.973578 2026-06-10 00:47:56.613687 | 2026-06-10 00:47:56.613832 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:47:57.154275 | controller | skipping: Conditional result was False 2026-06-10 00:47:57.192042 | 2026-06-10 00:47:57.192275 | TASK [health-check/zuul-authenticators : Wait for the test authenticator to be delisted] 2026-06-10 00:47:57.719502 | controller | ok: OK (388 bytes) 2026-06-10 00:47:57.729351 | 2026-06-10 00:47:57.729416 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 00:47:57.757167 | controller | ok 2026-06-10 00:47:57.774803 | 2026-06-10 00:47:57.774882 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the internal tenant] 2026-06-10 00:47:58.313002 | controller | ok: OK (2 bytes) 2026-06-10 00:47:58.328274 | 2026-06-10 00:47:58.328649 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the internal tenant] 2026-06-10 00:47:58.359891 | controller | skipping: Conditional result was False 2026-06-10 00:47:58.385040 | 2026-06-10 00:47:58.385267 | TASK [health-check/config-update-zuul : Set random value] 2026-06-10 00:47:58.470271 | controller | ok 2026-06-10 00:47:58.480872 | 2026-06-10 00:47:58.481015 | TASK [health-check/config-update-zuul : Create vars to generate and submit resources addition] 2026-06-10 00:47:58.517140 | controller | ok 2026-06-10 00:47:58.526449 | 2026-06-10 00:47:58.526617 | TASK [ansible.builtin.include_role : create-gerrit-repository] 2026-06-10 00:47:58.551106 | controller | ok 2026-06-10 00:47:58.583020 | 2026-06-10 00:47:58.583131 | TASK [create-gerrit-repository : Create gerrit repository named ichiban-config-tC3E2wmd] 2026-06-09 20:48:00.260980 | controller | 2026-06-10 00:48:00,259 INFO managesf.model.yamlbkd.engine - Resource engine is using /tmp/direct-apply as workdir 2026-06-09 20:48:00.261138 | controller | 2026-06-10 00:48:00,260 INFO managesf.model.yamlbkd.engine - Resources engine: direct apply resources requested 2026-06-09 20:48:00.279072 | controller | 2026-06-10 00:48:00,278 INFO managesf.GerritClient - ichiban-config-tC3E2wmd-core: creating group (Team core for the ichiban-config-tC3E2wmd repo) 2026-06-09 20:48:00.374376 | controller | 2026-06-10 00:48:00,373 INFO managesf.GerritClient - ichiban-config-tC3E2wmd-core: deleting group member admin 2026-06-09 20:48:00.475694 | controller | 2026-06-10 00:48:00,474 INFO managesf.GerritClient - ichiban-config-tC3E2wmd-ptl: creating group (Team lead for the ichiban-config-tC3E2wmd repo) 2026-06-09 20:48:00.578472 | controller | 2026-06-10 00:48:00,577 INFO managesf.GerritClient - ichiban-config-tC3E2wmd-ptl: deleting group member admin 2026-06-09 20:48:00.672260 | controller | 2026-06-10 00:48:00,671 INFO managesf.GerritClient - ichiban-config-tC3E2wmd-ptl: adding group member admin@sfop.me 2026-06-09 20:48:00.775924 | controller | 2026-06-10 00:48:00,775 INFO managesf.GerritClient - ichiban-config-tC3E2wmd: creating project (ichiban-config-tC3E2wmd repository) 2026-06-09 20:48:00.870584 | controller | 2026-06-10 00:48:00,869 INFO managesf.services.gerrit.utils - [gerrit] Clone repository ichiban-config-tC3E2wmd 2026-06-09 20:48:01.157022 | controller | 2026-06-10 00:48:01,155 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'clone', 'ssh://admin@gerrit-sshd:29418/ichiban-config-tC3E2wmd', '/tmp/tmpmbc_ara4/clone-ichiban-config-tC3E2wmd'] output 2026-06-09 20:48:01.157102 | controller | 2026-06-10 00:48:01,156 INFO managesf.services.gerrit.utils - Cloning into '/tmp/tmpmbc_ara4/clone-ichiban-config-tC3E2wmd'... 2026-06-09 20:48:01.157114 | controller | Warning: Permanently added '[gerrit-sshd]:29418' (ED25519) to the list of known hosts. 2026-06-09 20:48:01.157124 | controller | Total 2 (delta 0), reused 0 (delta 0) 2026-06-09 20:48:01.157134 | controller | 2026-06-09 20:48:01.400547 | controller | 2026-06-10 00:48:01,399 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', '-a'] output 2026-06-09 20:48:01.400619 | controller | 2026-06-10 00:48:01,399 INFO managesf.services.gerrit.utils - 2026-06-09 20:48:01.492572 | controller | 2026-06-10 00:48:01,491 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on config for repository ichiban-config-tC3E2wmd 2026-06-09 20:48:01.682528 | controller | 2026-06-10 00:48:01,681 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', 'origin', 'refs/meta/config:refs/remotes/origin/meta/config'] output 2026-06-09 20:48:01.682580 | controller | 2026-06-10 00:48:01,681 INFO managesf.services.gerrit.utils - Total 4 (delta 0), reused 0 (delta 0) 2026-06-09 20:48:01.682588 | controller | From ssh://gerrit-sshd:29418/ichiban-config-tC3E2wmd 2026-06-09 20:48:01.682595 | controller | * [new ref] refs/meta/config -> origin/meta/config 2026-06-09 20:48:01.682601 | controller | 2026-06-09 20:48:01.692332 | controller | 2026-06-10 00:48:01,691 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'meta/config'] output 2026-06-09 20:48:01.692353 | controller | 2026-06-10 00:48:01,691 INFO managesf.services.gerrit.utils - Switched to a new branch 'meta/config' 2026-06-09 20:48:01.692359 | controller | branch 'meta/config' set up to track 'origin/meta/config'. 2026-06-09 20:48:01.692365 | controller | 2026-06-09 20:48:01.692390 | controller | 2026-06-10 00:48:01,691 INFO managesf.services.gerrit.utils - [gerrit] Add file project.config to index 2026-06-09 20:48:01.696492 | controller | 2026-06-10 00:48:01,695 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'project.config'] output 2026-06-09 20:48:01.696512 | controller | 2026-06-10 00:48:01,696 INFO managesf.services.gerrit.utils - 2026-06-09 20:48:01.696519 | controller | 2026-06-10 00:48:01,696 INFO managesf.services.gerrit.utils - [gerrit] Add file groups to index 2026-06-09 20:48:01.702602 | controller | 2026-06-10 00:48:01,702 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'groups'] output 2026-06-09 20:48:01.702623 | controller | 2026-06-10 00:48:01,702 INFO managesf.services.gerrit.utils - 2026-06-09 20:48:01.709327 | controller | 2026-06-10 00:48:01,708 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-09 20:48:01.709375 | controller | 2026-06-10 00:48:01,708 INFO managesf.services.gerrit.utils - M groups 2026-06-09 20:48:01.709383 | controller | M project.config 2026-06-09 20:48:01.709388 | controller | 2026-06-09 20:48:01.720347 | controller | 2026-06-10 00:48:01,719 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mProvides ACL and Groups'] output 2026-06-09 20:48:01.720389 | controller | 2026-06-10 00:48:01,719 INFO managesf.services.gerrit.utils - [meta/config cddd5ab] Provides ACL and Groups 2026-06-09 20:48:01.720400 | controller | 2 files changed, 31 insertions(+), 9 deletions(-) 2026-06-09 20:48:01.720408 | controller | 2026-06-09 20:48:02.068323 | controller | 2026-06-10 00:48:02,066 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'meta/config:meta/config'] output 2026-06-09 20:48:02.068378 | controller | 2026-06-10 00:48:02,066 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-09 20:48:02.068392 | controller | To ssh://gerrit-sshd:29418/ichiban-config-tC3E2wmd 2026-06-09 20:48:02.068397 | controller | f1133bf..cddd5ab meta/config -> refs/meta/config 2026-06-09 20:48:02.068403 | controller | 2026-06-09 20:48:02.068411 | controller | 2026-06-10 00:48:02,066 INFO managesf.services.gerrit.utils - [gerrit] Push on config for repository ichiban-config-tC3E2wmd 2026-06-09 20:48:02.068423 | controller | 2026-06-10 00:48:02,067 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on master for repository ichiban-config-tC3E2wmd 2026-06-09 20:48:02.083064 | controller | 2026-06-10 00:48:02,078 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'master'] output 2026-06-09 20:48:02.083117 | controller | 2026-06-10 00:48:02,079 INFO managesf.services.gerrit.utils - Switched to branch 'master' 2026-06-09 20:48:02.083123 | controller | Your branch is up to date with 'origin/master'. 2026-06-09 20:48:02.083130 | controller | 2026-06-09 20:48:02.092525 | controller | 2026-06-10 00:48:02,089 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'reset', '--hard', 'origin/master'] output 2026-06-09 20:48:02.092579 | controller | 2026-06-10 00:48:02,089 INFO managesf.services.gerrit.utils - HEAD is now at f9861aa Initial empty repository 2026-06-09 20:48:02.092586 | controller | 2026-06-09 20:48:02.092592 | controller | 2026-06-10 00:48:02,089 INFO managesf.services.gerrit.utils - [gerrit] Add file .gitreview to index 2026-06-09 20:48:02.101524 | controller | 2026-06-10 00:48:02,100 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', '.gitreview'] output 2026-06-09 20:48:02.101625 | controller | 2026-06-10 00:48:02,101 INFO managesf.services.gerrit.utils - 2026-06-09 20:48:02.114289 | controller | 2026-06-10 00:48:02,111 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-09 20:48:02.114334 | controller | 2026-06-10 00:48:02,111 INFO managesf.services.gerrit.utils - A .gitreview 2026-06-09 20:48:02.114341 | controller | 2026-06-09 20:48:02.173415 | controller | 2026-06-10 00:48:02,172 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mManageSF commit'] output 2026-06-09 20:48:02.173470 | controller | 2026-06-10 00:48:02,172 INFO managesf.services.gerrit.utils - [master 6ed1a62] ManageSF commit 2026-06-09 20:48:02.173476 | controller | 1 file changed, 5 insertions(+) 2026-06-09 20:48:02.173480 | controller | create mode 100644 .gitreview 2026-06-09 20:48:02.173485 | controller | 2026-06-09 20:48:02.415402 | controller | 2026-06-10 00:48:02,414 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'master'] output 2026-06-09 20:48:02.415463 | controller | 2026-06-10 00:48:02,414 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-09 20:48:02.415470 | controller | To ssh://gerrit-sshd:29418/ichiban-config-tC3E2wmd 2026-06-09 20:48:02.415475 | controller | f9861aa..6ed1a62 master -> master 2026-06-09 20:48:02.415482 | controller | 2026-06-09 20:48:02.415487 | controller | 2026-06-10 00:48:02,414 INFO managesf.services.gerrit.utils - [gerrit] Push on master for repository ichiban-config-tC3E2wmd 2026-06-09 20:48:02.415493 | controller | 2026-06-10 00:48:02,414 INFO managesf.services.gerrit.utils - [gerrit] List remote branches 2026-06-09 20:48:02.423006 | controller | 2026-06-10 00:48:02,421 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'branch', '-rv', '--abbrev=40'] output 2026-06-09 20:48:02.423082 | controller | 2026-06-10 00:48:02,422 INFO managesf.services.gerrit.utils - origin/HEAD -> origin/master 2026-06-09 20:48:02.423100 | controller | origin/master 6ed1a62847371fb2f7d26c6f663884b2cf7a66a5 ManageSF commit 2026-06-09 20:48:02.423113 | controller | origin/meta/config f1133bf1acd2bb4710ca5387032b68bc2cf6b0c8 Created project 2026-06-09 20:48:02.423126 | controller | 2026-06-09 20:48:02.423150 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-tC3E2wmd-acl] is going to be created. 2026-06-09 20:48:02.423426 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-tC3E2wmd-core] is going to be created. 2026-06-09 20:48:02.423466 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-tC3E2wmd-ptl] is going to be created. 2026-06-09 20:48:02.423479 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-tC3E2wmd] is going to be created. 2026-06-09 20:48:02.423491 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-tC3E2wmd-core] will be created. 2026-06-09 20:48:02.423510 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-tC3E2wmd-core] has been created. 2026-06-09 20:48:02.423542 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-tC3E2wmd-ptl] will be created. 2026-06-09 20:48:02.423555 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-tC3E2wmd-ptl] has been created. 2026-06-09 20:48:02.423567 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-tC3E2wmd-acl] will be created. 2026-06-09 20:48:02.423634 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-tC3E2wmd-acl] has been created. 2026-06-09 20:48:02.423648 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-tC3E2wmd] will be created. 2026-06-09 20:48:02.423656 | controller | 2026-06-10 00:48:02,422 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-tC3E2wmd] has been created. 2026-06-09 20:48:02.423698 | controller | 2026-06-09 20:48:02.423708 | controller | === Resources actions === 2026-06-09 20:48:02.423717 | controller | 2026-06-09 20:48:02.423725 | controller | Resource [type: acls, ID: ichiban-config-tC3E2wmd-acl] is going to be created. 2026-06-09 20:48:02.423733 | controller | Resource [type: groups, ID: ichiban-config-tC3E2wmd-core] is going to be created. 2026-06-09 20:48:02.423742 | controller | Resource [type: groups, ID: ichiban-config-tC3E2wmd-ptl] is going to be created. 2026-06-09 20:48:02.423750 | controller | Resource [type: repos, ID: ichiban-config-tC3E2wmd] is going to be created. 2026-06-09 20:48:02.423758 | controller | Resource [type: groups, ID: ichiban-config-tC3E2wmd-core] will be created. 2026-06-09 20:48:02.423766 | controller | Resource [type: groups, ID: ichiban-config-tC3E2wmd-core] has been created. 2026-06-09 20:48:02.423774 | controller | Resource [type: groups, ID: ichiban-config-tC3E2wmd-ptl] will be created. 2026-06-09 20:48:02.423781 | controller | Resource [type: groups, ID: ichiban-config-tC3E2wmd-ptl] has been created. 2026-06-09 20:48:02.423789 | controller | Resource [type: acls, ID: ichiban-config-tC3E2wmd-acl] will be created. 2026-06-09 20:48:02.423808 | controller | Resource [type: acls, ID: ichiban-config-tC3E2wmd-acl] has been created. 2026-06-09 20:48:02.423816 | controller | Resource [type: repos, ID: ichiban-config-tC3E2wmd] will be created. 2026-06-09 20:48:02.423824 | controller | Resource [type: repos, ID: ichiban-config-tC3E2wmd] has been created. 2026-06-10 00:48:02.621716 | controller | ok: Runtime: 0:00:03.711997 2026-06-10 00:48:02.632536 | 2026-06-10 00:48:02.632927 | TASK [create-gerrit-repository : Ensure repository exists in Gerrit] 2026-06-10 00:48:03.135000 | controller | ok: OK (752 bytes) 2026-06-10 00:48:03.152940 | 2026-06-10 00:48:03.153080 | TASK [health-check/config-update-zuul : Set bad Zuul tenant config] 2026-06-10 00:48:03.567796 | controller | changed: Block inserted 2026-06-10 00:48:03.580808 | 2026-06-10 00:48:03.580952 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 00:48:03.889866 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.007483 2026-06-10 00:48:03.890314 | controller | ok: All items complete 2026-06-10 00:48:03.890383 | 2026-06-10 00:48:04.139793 | controller | [master 6d9f13b] Add zuul/main.yaml 2026-06-10 00:48:04.139945 | controller | 1 file changed, 5 insertions(+) 2026-06-10 00:48:04.140009 | controller | ok: Item: git commit -m 'Add zuul/main.yaml' Runtime: 0:00:00.011128 2026-06-10 00:48:04.159033 | 2026-06-10 00:48:04.159289 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 00:48:04.196337 | controller | ok 2026-06-10 00:48:04.314279 | 2026-06-10 00:48:04.314433 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 00:48:04.343321 | controller | ok 2026-06-10 00:48:04.350214 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 00:48:04.355592 | 2026-06-10 00:48:04.355664 | TASK [health-check/repo-submit-change : Submit change] 2026-06-09 20:48:05.884997 | controller | Your change was committed before the commit hook was installed. 2026-06-09 20:48:05.885054 | controller | Amending the commit to add a gerrit change id. 2026-06-09 20:48:05.885062 | controller | remote: 2026-06-09 20:48:05.885069 | controller | remote: Processing changes: new: 1 (\) 2026-06-09 20:48:05.885078 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-09 20:48:05.885086 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-09 20:48:05.885094 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-09 20:48:05.885100 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-09 20:48:05.885107 | controller | remote: 2026-06-09 20:48:05.885113 | controller | remote: SUCCESS 2026-06-09 20:48:05.885119 | controller | remote: 2026-06-09 20:48:05.885127 | controller | remote: https://gerrit.sfop.me/c/config/+/1 Add zuul/main.yaml [NEW] 2026-06-09 20:48:05.885133 | controller | remote: 2026-06-09 20:48:05.885139 | controller | To https://gerrit.sfop.me/a/config 2026-06-09 20:48:05.885145 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 00:48:06.390916 | controller | ok: Runtime: 0:00:01.261599 2026-06-10 00:48:06.400224 | 2026-06-10 00:48:06.400359 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-09 20:48:06.718928 | controller | I289d8a6c0af38959060f9d37221f404d8f96ec5a 2026-06-10 00:48:06.949042 | controller | ok: Runtime: 0:00:00.011569 2026-06-10 00:48:06.959130 | 2026-06-10 00:48:06.959272 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 00:48:07.765373 | controller | ok: OK (47 bytes) 2026-06-10 00:48:07.779611 | 2026-06-10 00:48:07.779785 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 00:48:07.816174 | controller | ok 2026-06-10 00:48:07.830830 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 00:48:07.844439 | 2026-06-10 00:48:07.844525 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 00:48:08.405657 | controller | ok: OK (7142 bytes) 2026-06-10 00:48:08.415965 | 2026-06-10 00:48:08.416135 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 00:48:08.472253 | controller | ok 2026-06-10 00:48:08.490399 | 2026-06-10 00:48:08.490549 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 00:49:12.216526 | controller | ok: OK (1162 bytes) 2026-06-10 00:49:12.227631 | 2026-06-10 00:49:12.227806 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 00:49:12.254922 | controller | skipping: Conditional result was False 2026-06-10 00:49:12.267337 | 2026-06-10 00:49:12.267482 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 00:49:12.294037 | controller | skipping: Conditional result was False 2026-06-10 00:49:12.306404 | 2026-06-10 00:49:12.306553 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 00:49:12.343829 | controller | skipping: Conditional result was False 2026-06-10 00:49:12.366842 | 2026-06-10 00:49:12.367015 | TASK [health-check/config-update-zuul : Set Zuul tenant config] 2026-06-10 00:49:12.627427 | controller | changed: Block inserted 2026-06-10 00:49:12.637029 | 2026-06-10 00:49:12.637167 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 00:49:12.938430 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.003797 2026-06-10 00:49:13.220307 | controller | [master a33bd73] Add zuul/main.yaml 2026-06-10 00:49:13.220488 | controller | Date: Tue Jun 9 20:48:04 2026 -0400 2026-06-10 00:49:13.220542 | controller | 1 file changed, 8 insertions(+) 2026-06-10 00:49:13.220632 | controller | ok: Item: git commit --amend --no-edit Runtime: 0:00:00.048557 2026-06-10 00:49:13.234615 | 2026-06-10 00:49:13.234826 | TASK [Submit change and verify Zuul console output] 2026-06-10 00:49:13.275324 | controller | ok 2026-06-10 00:49:13.315914 | 2026-06-10 00:49:13.316028 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 00:49:13.337395 | controller | ok 2026-06-10 00:49:13.345895 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 00:49:13.354732 | 2026-06-10 00:49:13.354820 | TASK [health-check/repo-submit-change : Submit change] 2026-06-09 20:49:14.301929 | controller | remote: 2026-06-09 20:49:14.302019 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-09 20:49:14.302033 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-09 20:49:14.302043 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-09 20:49:14.302055 | controller | remote: Processing changes: refs: 1, updated: 1, done 2026-06-09 20:49:14.302067 | controller | remote: 2026-06-09 20:49:14.302076 | controller | remote: SUCCESS 2026-06-09 20:49:14.302086 | controller | remote: 2026-06-09 20:49:14.302095 | controller | remote: https://gerrit.sfop.me/c/config/+/1 Add zuul/main.yaml 2026-06-09 20:49:14.302105 | controller | remote: 2026-06-09 20:49:14.302114 | controller | To https://gerrit.sfop.me/a/config 2026-06-09 20:49:14.302125 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 00:49:14.389196 | controller | ok: Runtime: 0:00:00.722594 2026-06-10 00:49:14.399930 | 2026-06-10 00:49:14.400107 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-09 20:49:14.689281 | controller | I289d8a6c0af38959060f9d37221f404d8f96ec5a 2026-06-10 00:49:14.943393 | controller | ok: Runtime: 0:00:00.010632 2026-06-10 00:49:14.953642 | 2026-06-10 00:49:14.953805 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 00:49:15.577319 | controller | ok: OK (47 bytes) 2026-06-10 00:49:15.588845 | 2026-06-10 00:49:15.588989 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 00:49:15.615863 | controller | skipping: Conditional result was False 2026-06-10 00:49:15.629108 | 2026-06-10 00:49:15.629274 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 00:49:15.672529 | controller | ok 2026-06-10 00:49:15.683322 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-zuul-console-success.yaml 2026-06-10 00:49:15.697507 | 2026-06-10 00:49:15.697646 | TASK [health-check/repo-submit-change : Wait for config-update job] 2026-06-10 00:50:17.841104 | controller | ok: Runtime: 0:00:00.084163 2026-06-10 00:50:17.901631 | 2026-06-10 00:50:17.901801 | TASK [health-check/repo-submit-change : Connect and validate console stream] 2026-06-09 20:50:18.245877 | controller | 2026-06-10 00:50:07.994697 | Job console starting 2026-06-09 20:50:18.246009 | controller | 2026-06-10 00:50:07.995419 | Updating git repos 2026-06-09 20:50:18.246020 | controller | 2026-06-10 00:50:08.064955 | Cloning repos into workspace 2026-06-09 20:50:18.246029 | controller | 2026-06-10 00:50:09.111885 | Restoring repo states 2026-06-09 20:50:18.246038 | controller | 2026-06-10 00:50:13.664707 | Merging changes 2026-06-09 20:50:18.246046 | controller | 2026-06-10 00:50:13.664744 | Checking out repos 2026-06-09 20:50:18.246054 | controller | 2026-06-10 00:50:13.695469 | Preparing playbooks 2026-06-09 20:50:18.246060 | controller | 2026-06-10 00:50:14.646008 | Running Ansible setup 2026-06-09 20:50:18.246072 | controller | 2026-06-10 00:50:18.000872 | PRE-RUN START: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-09 20:50:18.246079 | controller | 2026-06-09 20:50:20.261975 | controller | 2026-06-10 00:50:19.879924 | 2026-06-09 20:50:20.262040 | controller | 2026-06-10 00:50:19.880150 | PLAY [localhost] 2026-06-09 20:50:20.262052 | controller | 2026-06-10 00:50:19.896166 | 2026-06-09 20:50:20.262062 | controller | 2026-06-10 00:50:19.896399 | TASK [Gathering Facts] 2026-06-09 20:50:20.262071 | controller | 2026-06-09 20:50:21.762298 | controller | 2026-06-10 00:50:21.675887 | localhost | ok 2026-06-09 20:50:21.762722 | controller | 2026-06-10 00:50:21.682378 | 2026-06-09 20:50:21.762735 | controller | 2026-06-10 00:50:21.682448 | TASK [Setup log path fact] 2026-06-09 20:50:21.762744 | controller | 2026-06-10 00:50:21.697564 | localhost | ok 2026-06-09 20:50:21.762753 | controller | 2026-06-09 20:50:22.262561 | controller | 2026-06-10 00:50:21.765954 | 2026-06-09 20:50:22.262628 | controller | 2026-06-10 00:50:21.766050 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-09 20:50:22.262637 | controller | 2026-06-10 00:50:21.781781 | localhost | ok 2026-06-09 20:50:22.262645 | controller | 2026-06-10 00:50:21.787054 | 2026-06-09 20:50:22.262662 | controller | 2026-06-10 00:50:21.787133 | TASK [emit-job-header : Print job information] 2026-06-09 20:50:22.262670 | controller | 2026-06-10 00:50:21.883074 | # Job Information 2026-06-09 20:50:22.262677 | controller | 2026-06-10 00:50:21.883214 | Ansible Version: 2.16.18 2026-06-09 20:50:22.262684 | controller | 2026-06-10 00:50:21.883259 | Job: config-update 2026-06-09 20:50:22.262691 | controller | 2026-06-10 00:50:21.883280 | Pipeline: post 2026-06-09 20:50:22.262698 | controller | 2026-06-10 00:50:21.883297 | Executor: zuul-executor-1.zuul-executor-headless.sf.svc.cluster.local 2026-06-09 20:50:22.262705 | controller | 2026-06-10 00:50:21.883315 | Triggered by: https://gerrit.sfop.me/plugins/gitiles/config/+/a33bd7322e7353b8f3689ca9f790d3a67f155250^!/ 2026-06-09 20:50:22.262713 | controller | 2026-06-10 00:50:21.883336 | Log URL (when completed): https://sfop.me/logs/e5e/internal/e5ea1728ae2b4786a5850c063f69b1d2/ 2026-06-09 20:50:22.262720 | controller | 2026-06-10 00:50:21.883353 | Event ID: ccfe454b11dd4008bfedd7e1a7b8ecf0 2026-06-09 20:50:22.262727 | controller | 2026-06-10 00:50:21.885770 | 2026-06-09 20:50:22.262734 | controller | 2026-06-10 00:50:21.885848 | LOOP [emit-job-header : Print node information] 2026-06-09 20:50:22.262740 | controller | 2026-06-10 00:50:21.910356 | 2026-06-09 20:50:22.262754 | controller | 2026-06-09 20:50:22.262761 | controller | 2026-06-10 00:50:21.910535 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-09 20:50:22.262768 | controller | 2026-06-09 20:50:22.762545 | controller | 2026-06-10 00:50:22.701173 | localhost -> localhost | changed 2026-06-09 20:50:22.762598 | controller | 2026-06-10 00:50:22.705586 | 2026-06-09 20:50:22.762604 | controller | 2026-06-10 00:50:22.705649 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-06-09 20:50:22.762608 | controller | 2026-06-10 00:50:23.435857 | controller | ok: Runtime: 0:00:05.025340 2026-06-10 00:50:23.448610 | 2026-06-10 00:50:23.448753 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 00:50:23.482994 | controller | ok 2026-06-10 00:50:23.495813 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 00:50:23.511163 | 2026-06-10 00:50:23.511298 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-09 20:50:23.782060 | controller | a33bd7322e7353b8f3689ca9f790d3a67f155250 2026-06-10 00:50:24.050368 | controller | ok: Runtime: 0:00:00.008954 2026-06-10 00:50:24.063821 | 2026-06-10 00:50:24.063984 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 00:51:49.425860 | controller | ok: OK (1142 bytes) 2026-06-10 00:51:49.438980 | 2026-06-10 00:51:49.439170 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 00:51:49.458191 | controller | skipping: Conditional result was False 2026-06-10 00:51:49.486127 | 2026-06-10 00:51:49.486341 | TASK [health-check/config-update-zuul : ansible.builtin.set_fact] 2026-06-10 00:51:49.522377 | controller | ok 2026-06-10 00:51:49.534984 | 2026-06-10 00:51:49.535242 | TASK [Check that Zuul has pushed the job console on the logserver] 2026-06-10 00:51:49.558335 | controller | ok 2026-06-10 00:51:49.593690 | 2026-06-10 00:51:49.593802 | TASK [health-check/ensure-job-result-artifacts : Ensure job console (job-output.txt.gz) exists in Logserver] 2026-06-10 00:51:50.083717 | controller | ok: OK (4513 bytes) 2026-06-10 00:51:50.093568 | 2026-06-10 00:51:50.093758 | TASK [health-check/ensure-job-result-artifacts : Ensure internal logserver rewrite from job-output.txt to job-output.txt.gz] 2026-06-10 00:51:50.596000 | controller | ok: OK (4513 bytes) 2026-06-10 00:51:50.613722 | 2026-06-10 00:51:50.613866 | TASK [health-check/config-update-zuul : Get Zuul projects list] 2026-06-10 00:51:51.126272 | controller | ok: OK (144 bytes) 2026-06-10 00:51:51.141755 | 2026-06-10 00:51:51.141955 | TASK [health-check/config-update-zuul : Ensure project exists in Zuul (fail if not)] 2026-06-10 00:51:51.190283 | controller | skipping: Conditional result was False 2026-06-10 00:51:51.212650 | 2026-06-10 00:51:51.212837 | TASK [health-check/check-logjuicer : Get logjuicer status] 2026-06-10 00:51:51.705046 | controller | ok: OK (2 bytes) 2026-06-10 00:51:51.722284 | 2026-06-10 00:51:51.722454 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 00:51:51.751704 | controller | ok 2026-06-10 00:51:51.780602 | 2026-06-10 00:51:51.780743 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the demo-tenant tenant] 2026-06-10 00:51:52.323403 | controller | ok: OK (2 bytes) 2026-06-10 00:51:52.335320 | 2026-06-10 00:51:52.335454 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the demo-tenant tenant] 2026-06-10 00:51:52.363262 | controller | skipping: Conditional result was False 2026-06-10 00:51:52.381452 | 2026-06-10 00:51:52.381625 | TASK [health-check/zuul-demo-tenant-workflow : Fetch and reset demo-project to latest origin for fast-forward submit] 2026-06-09 20:51:52.740634 | controller | HEAD is now at 4ab7d04 Automatic secret re-encryption 2026-06-10 00:51:52.922238 | controller | ok: Runtime: 0:00:00.084416 2026-06-10 00:51:52.931440 | 2026-06-10 00:51:52.931621 | TASK [health-check/zuul-demo-tenant-workflow : Update the demo-job to sleep 1 second] 2026-06-10 00:51:53.412015 | controller | changed: 1 replacements made 2026-06-10 00:51:53.422381 | 2026-06-10 00:51:53.422518 | LOOP [health-check/zuul-demo-tenant-workflow : Commit file in the demo-project repo] 2026-06-10 00:51:53.678652 | controller | ok: Item: git add playbooks/run.yaml Runtime: 0:00:00.005305 2026-06-10 00:51:53.679244 | controller | changed: All items complete 2026-06-10 00:51:53.679323 | 2026-06-10 00:51:53.884828 | controller | [master 34b5843] Update demo-job to sleep 1 second 2026-06-10 00:51:53.884979 | controller | 1 file changed, 1 insertion(+), 1 deletion(-) 2026-06-10 00:51:53.885044 | controller | ok: Item: git commit -m 'Update demo-job to sleep 1 second' Runtime: 0:00:00.006981 2026-06-10 00:51:53.897107 | 2026-06-10 00:51:53.897228 | TASK [Submit, merge and verify post-merge job] 2026-06-10 00:51:53.936833 | controller | ok 2026-06-10 00:51:53.975357 | 2026-06-10 00:51:53.975467 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 00:51:54.003929 | controller | ok 2026-06-10 00:51:54.012102 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 00:51:54.019389 | 2026-06-10 00:51:54.019477 | TASK [health-check/repo-submit-change : Submit change] 2026-06-09 20:51:54.995369 | controller | Your change was committed before the commit hook was installed. 2026-06-09 20:51:54.995456 | controller | Amending the commit to add a gerrit change id. 2026-06-09 20:51:54.995468 | controller | remote: 2026-06-09 20:51:54.995478 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 20:51:54.995491 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 20:51:54.995501 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-09 20:51:54.995513 | controller | remote: 2026-06-09 20:51:54.995521 | controller | remote: SUCCESS 2026-06-09 20:51:54.995529 | controller | remote: 2026-06-09 20:51:54.995539 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/2 Update demo-job to sleep 1 second [NEW] 2026-06-09 20:51:54.995549 | controller | remote: 2026-06-09 20:51:54.995557 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-09 20:51:54.995566 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 00:51:55.065482 | controller | ok: Runtime: 0:00:00.717215 2026-06-10 00:51:55.075527 | 2026-06-10 00:51:55.075755 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-09 20:51:55.423532 | controller | I48f96db3114f57f796247269d56453b21a10cd08 2026-06-10 00:51:55.684450 | controller | ok: Runtime: 0:00:00.008421 2026-06-10 00:51:55.694513 | 2026-06-10 00:51:55.694678 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 00:51:56.379869 | controller | ok: OK (47 bytes) 2026-06-10 00:51:56.394096 | 2026-06-10 00:51:56.394238 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 00:51:56.423727 | controller | skipping: Conditional result was False 2026-06-10 00:51:56.436520 | 2026-06-10 00:51:56.436703 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 00:51:56.465438 | controller | skipping: Conditional result was False 2026-06-10 00:51:56.479827 | 2026-06-10 00:51:56.480006 | TASK [health-check/repo-submit-change : Ensure a success for publish-job post job] 2026-06-10 00:51:56.526439 | controller | ok 2026-06-10 00:51:56.539141 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 00:51:56.552717 | 2026-06-10 00:51:56.552859 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-09 20:51:56.829686 | controller | bb037cc610ee38bde70c3eeeb925fe0206daeccb 2026-06-10 00:51:57.093899 | controller | ok: Runtime: 0:00:00.008468 2026-06-10 00:51:57.106129 | 2026-06-10 00:51:57.106283 | TASK [health-check/repo-submit-change : Wait publish-job SUCCESS result using zuul-web API] 2026-06-10 00:52:50.078146 | controller | ok: OK (927 bytes) 2026-06-10 00:52:50.087648 | 2026-06-10 00:52:50.087743 | TASK [health-check/repo-submit-change : Ensure a success for publish-job job] 2026-06-10 00:52:50.122524 | controller | skipping: Conditional result was False 2026-06-10 00:52:50.149508 | 2026-06-10 00:52:50.149698 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale down executor statefulset to ensure 1 replica only] 2026-06-09 20:52:50.522832 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 00:52:50.692665 | controller | ok: Runtime: 0:00:00.073339 2026-06-10 00:52:50.702024 | 2026-06-10 00:52:50.702252 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for only 1 executor running] 2026-06-09 20:52:51.039723 | controller | 2 2026-06-10 00:53:01.363390 | controller | ok: Runtime: 0:00:00.048101 2026-06-10 00:53:01.373404 | 2026-06-10 00:53:01.373547 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure we are on the default branch] 2026-06-09 20:53:01.663177 | controller | Already on 'master' 2026-06-09 20:53:01.663523 | controller | Your branch is ahead of 'origin/master' by 1 commit. 2026-06-09 20:53:01.663542 | controller | (use "git push" to publish your local commits) 2026-06-10 00:53:01.921429 | controller | ok: Runtime: 0:00:00.005218 2026-06-10 00:53:01.932156 | 2026-06-10 00:53:01.932300 | TASK [health-check/zuul-executor-ensure-graceful-stop : Create a test branch] 2026-06-09 20:53:02.227195 | controller | Switched to a new branch 'test_check_url' 2026-06-10 00:53:02.476321 | controller | ok: Runtime: 0:00:00.005629 2026-06-10 00:53:02.480874 | 2026-06-10 00:53:02.480973 | TASK [health-check/zuul-executor-ensure-graceful-stop : Replace demo-job with demo-job-check-url] 2026-06-10 00:53:02.758550 | controller | changed: 2 replacements made 2026-06-10 00:53:02.768681 | 2026-06-10 00:53:02.768833 | TASK [health-check/zuul-executor-ensure-graceful-stop : create the review] 2026-06-10 00:53:02.804328 | controller | ok 2026-06-10 00:53:02.815401 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-executor-ensure-graceful-stop/tasks/create-review.yaml 2026-06-10 00:53:02.824769 | 2026-06-10 00:53:02.824859 | LOOP [health-check/zuul-executor-ensure-graceful-stop : Create review] 2026-06-10 00:53:03.126806 | controller | ok: Item: git add --all Runtime: 0:00:00.006260 2026-06-10 00:53:03.360765 | controller | [test_check_url aa2bf83] test check url 2026-06-10 00:53:03.360904 | controller | 1 file changed, 2 insertions(+), 2 deletions(-) 2026-06-10 00:53:03.360937 | controller | ok: Item: git commit -m 'test check url' Runtime: 0:00:00.045153 2026-06-10 00:53:04.151155 | controller | remote: 2026-06-10 00:53:04.151334 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 00:53:04.151385 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 00:53:04.151424 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 00:53:04.151461 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 00:53:04.151497 | controller | remote: 2026-06-10 00:53:04.151533 | controller | remote: SUCCESS 2026-06-10 00:53:04.151607 | controller | remote: 2026-06-10 00:53:04.151658 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/3 test check url [NEW] 2026-06-10 00:53:04.151697 | controller | remote: 2026-06-10 00:53:04.151733 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 00:53:04.151769 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 00:53:04.151820 | controller | ok: Item: git review Runtime: 0:00:00.514570 2026-06-10 00:53:04.166357 | 2026-06-10 00:53:04.166645 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get Change-id] 2026-06-09 20:53:04.472093 | controller | I0c7c30a991787117c1f0f4983be442737277ecee 2026-06-10 00:53:04.716955 | controller | ok: Runtime: 0:00:00.008818 2026-06-10 00:53:04.727275 | 2026-06-10 00:53:04.727412 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 00:53:04.774477 | controller | ok 2026-06-10 00:53:04.784206 | 2026-06-10 00:53:04.784360 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get review number] 2026-06-09 20:53:05.175682 | controller | 3 2026-06-10 00:53:05.419077 | controller | ok: Runtime: 0:00:00.058930 2026-06-10 00:53:05.428800 | 2026-06-10 00:53:05.428952 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 00:53:05.475478 | controller | ok 2026-06-10 00:53:05.485954 | 2026-06-10 00:53:05.486090 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.set_fact] 2026-06-10 00:53:05.530630 | controller | ok 2026-06-10 00:53:05.539473 | 2026-06-10 00:53:05.539649 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for job to start] 2026-06-09 20:53:05.876841 | controller | null 2026-06-10 00:53:16.256565 | controller | ok: Runtime: 0:00:00.056485 2026-06-10 00:53:16.279414 | 2026-06-10 00:53:16.279732 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale down executor statefulset to trigger a graceful stop] 2026-06-09 20:53:16.643183 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 00:53:16.818238 | controller | ok: Runtime: 0:00:00.110715 2026-06-10 00:53:16.827266 | 2026-06-10 00:53:16.827400 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure executor component is marked for shutdown in Zuul API] 2026-06-09 20:53:17.134019 | controller | paused 2026-06-10 00:53:17.375591 | controller | ok: Runtime: 0:00:00.036471 2026-06-10 00:53:17.385560 | 2026-06-10 00:53:17.385754 | TASK [health-check/zuul-executor-ensure-graceful-stop : show executor state] 2026-06-10 00:53:17.431918 | controller | ok: paused 2026-06-10 00:53:17.441312 | 2026-06-10 00:53:17.441461 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get back on default branch] 2026-06-09 20:53:17.716495 | controller | Switched to branch 'master' 2026-06-09 20:53:17.717164 | controller | Your branch is ahead of 'origin/master' by 1 commit. 2026-06-09 20:53:17.717178 | controller | (use "git push" to publish your local commits) 2026-06-10 00:53:17.986970 | controller | ok: Runtime: 0:00:00.008554 2026-06-10 00:53:17.996140 | 2026-06-10 00:53:17.996289 | TASK [health-check/zuul-executor-ensure-graceful-stop : Create a test branch] 2026-06-09 20:53:18.284843 | controller | Switched to a new branch 'test_short_job' 2026-06-10 00:53:18.545799 | controller | ok: Runtime: 0:00:00.004207 2026-06-10 00:53:18.554934 | 2026-06-10 00:53:18.555063 | TASK [health-check/zuul-executor-ensure-graceful-stop : Update the demo-job to sleep 2 seconds] 2026-06-10 00:53:18.828286 | controller | changed: 1 replacements made 2026-06-10 00:53:18.837146 | 2026-06-10 00:53:18.837280 | TASK [health-check/zuul-executor-ensure-graceful-stop : create the review] 2026-06-10 00:53:18.872163 | controller | ok 2026-06-10 00:53:18.887564 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-executor-ensure-graceful-stop/tasks/create-review.yaml 2026-06-10 00:53:18.904224 | 2026-06-10 00:53:18.904307 | LOOP [health-check/zuul-executor-ensure-graceful-stop : Create review] 2026-06-10 00:53:19.203646 | controller | ok: Item: git add --all Runtime: 0:00:00.010295 2026-06-10 00:53:19.204122 | controller | changed: All items complete 2026-06-10 00:53:19.204180 | 2026-06-10 00:53:19.444295 | controller | [test_short_job a769aa0] test short job 2026-06-10 00:53:19.444476 | controller | 1 file changed, 1 insertion(+), 1 deletion(-) 2026-06-10 00:53:19.444543 | controller | ok: Item: git commit -m 'test short job' Runtime: 0:00:00.053270 2026-06-10 00:53:20.243549 | controller | remote: 2026-06-10 00:53:20.243751 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 00:53:20.243792 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 00:53:20.243825 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 00:53:20.243856 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 00:53:20.243887 | controller | remote: 2026-06-10 00:53:20.243930 | controller | remote: SUCCESS 2026-06-10 00:53:20.243962 | controller | remote: 2026-06-10 00:53:20.243993 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/4 test short job [NEW] 2026-06-10 00:53:20.244025 | controller | remote: 2026-06-10 00:53:20.244055 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 00:53:20.244086 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 00:53:20.244131 | controller | ok: Item: git review Runtime: 0:00:00.561721 2026-06-10 00:53:20.257914 | 2026-06-10 00:53:20.257997 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get Change-id] 2026-06-09 20:53:20.518157 | controller | I41752f403a7beaa20770b16f8c1b83a99d33413c 2026-06-10 00:53:20.792093 | controller | ok: Runtime: 0:00:00.010193 2026-06-10 00:53:20.801930 | 2026-06-10 00:53:20.802073 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 00:53:20.847403 | controller | ok 2026-06-10 00:53:20.857395 | 2026-06-10 00:53:20.857534 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get review number] 2026-06-09 20:53:21.161280 | controller | 4 2026-06-10 00:53:21.399330 | controller | ok: Runtime: 0:00:00.055520 2026-06-10 00:53:21.405155 | 2026-06-10 00:53:21.405242 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 00:53:21.461605 | controller | ok 2026-06-10 00:53:21.472347 | 2026-06-10 00:53:21.472503 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.set_fact] 2026-06-10 00:53:21.554273 | controller | ok 2026-06-10 00:53:21.564304 | 2026-06-10 00:53:21.564436 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure new review is queued] 2026-06-10 00:53:32.282558 | controller | ok: Runtime: 0:00:00.063653 2026-06-10 00:53:32.291725 | 2026-06-10 00:53:32.291809 | TASK [Create switch file on logserver so that first job can complete] 2026-06-10 00:53:32.334743 | controller | ok 2026-06-10 00:53:32.385232 | 2026-06-10 00:53:32.385445 | TASK [create-file-on-logserver : copy file on log server] 2026-06-09 20:53:32.729787 | controller | Defaulted container "logserver" out of: logserver, logserver-sshd, purgelogs, logserver-nodeexporter 2026-06-10 00:53:32.925083 | controller | ok: Runtime: 0:00:00.188617 2026-06-10 00:53:32.941789 | 2026-06-10 00:53:32.941921 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for executor pod to terminate] 2026-06-09 20:53:33.284828 | controller | NAME READY STATUS RESTARTS AGE 2026-06-09 20:53:33.284947 | controller | config-base-secret-1-2801-ns-4fdwk 0/1 Completed 0 28m 2026-06-09 20:53:33.284963 | controller | config-base-secret-1-945-ns-85xbq 0/1 Completed 0 38m 2026-06-09 20:53:33.284971 | controller | config-base-secret-1-945-ns-v2xnk 0/1 Completed 0 33m 2026-06-09 20:53:33.284979 | controller | gateway-6bbbd758d9-ws9mf 1/1 Running 0 27m 2026-06-09 20:53:33.284985 | controller | gerrit-0 2/2 Running 0 45m 2026-06-09 20:53:33.284992 | controller | git-server-0 2/2 Running 0 11m 2026-06-09 20:53:33.284998 | controller | hound-search-0 1/1 Running 0 44m 2026-06-09 20:53:33.285005 | controller | logjuicer-6ff84764b7-mv8gq 1/1 Running 0 44m 2026-06-09 20:53:33.285012 | controller | logserver-0 4/4 Running 0 22m 2026-06-09 20:53:33.285019 | controller | mariadb-0 3/3 Running 0 19m 2026-06-09 20:53:33.285026 | controller | nodepool-builder-0 5/5 Running 0 19m 2026-06-09 20:53:33.285033 | controller | nodepool-launcher-688bf4bbff-6mqg5 3/3 Running 0 19m 2026-06-09 20:53:33.285040 | controller | post-init-wjpvj 0/1 Completed 0 44m 2026-06-09 20:53:33.285071 | controller | test-fluentbit 1/1 Running 0 36m 2026-06-09 20:53:33.285081 | controller | test-loki-8cd4fd56-sbxfw 1/1 Running 0 36m 2026-06-09 20:53:33.285086 | controller | zookeeper-0 3/3 Running 0 19m 2026-06-09 20:53:33.285095 | controller | zuul-db-provision-52vwg 0/1 Completed 0 42m 2026-06-09 20:53:33.285100 | controller | zuul-db-provision-gt92h 0/1 Completed 0 30m 2026-06-09 20:53:33.285105 | controller | zuul-executor-0 2/2 Terminating 0 10m 2026-06-09 20:53:33.285109 | controller | zuul-merger-0 2/2 Running 0 10m 2026-06-09 20:53:33.285114 | controller | zuul-merger-1 2/2 Running 0 10m 2026-06-09 20:53:33.285119 | controller | zuul-scheduler-0 3/3 Running 0 6m43s 2026-06-09 20:53:33.285124 | controller | zuul-web-575996f59f-g4fd2 1/1 Running 0 6m45s 2026-06-09 20:53:33.285130 | controller | zuul-weeder-5fb87885d-9kqmb 1/1 Running 0 9m48s 2026-06-10 00:53:54.070786 | controller | ok: Runtime: 0:00:00.132958 2026-06-10 00:53:54.080460 | 2026-06-10 00:53:54.080628 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure first job completed] 2026-06-10 00:53:54.632756 | controller | ok: Runtime: 0:00:00.051484 2026-06-10 00:53:54.644527 | 2026-06-10 00:53:54.644835 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure second job hasn't completed] 2026-06-10 00:53:55.187853 | controller | ok: Runtime: 0:00:00.055703 2026-06-10 00:53:55.198033 | 2026-06-10 00:53:55.198222 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale up executor statefulset] 2026-06-09 20:53:55.558627 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 00:53:55.749619 | controller | ok: Runtime: 0:00:00.072724 2026-06-10 00:53:55.760110 | 2026-06-10 00:53:55.760270 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait until executor pod is up] 2026-06-09 20:53:56.137152 | controller | NAME READY STATUS RESTARTS AGE 2026-06-09 20:53:56.137253 | controller | zuul-executor-0 0/2 ContainerCreating 0 1s 2026-06-10 00:54:06.512672 | controller | ok: Runtime: 0:00:00.109211 2026-06-10 00:54:06.521631 | 2026-06-10 00:54:06.521769 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure executor component is running] 2026-06-09 20:54:06.864083 | controller | running 2026-06-10 00:54:07.065277 | controller | ok: Runtime: 0:00:00.038267 2026-06-10 00:54:07.074945 | 2026-06-10 00:54:07.075093 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure second review's job completed] 2026-06-10 00:54:38.392359 | controller | ok: Runtime: 0:00:00.057910 2026-06-10 00:54:38.411418 | 2026-06-10 00:54:38.411681 | TASK [health-check/test-nodepool-providers-secrets : Ensure an empty /tmp/kube.config] 2026-06-10 00:54:38.712625 | controller | changed 2026-06-10 00:54:38.724459 | 2026-06-10 00:54:38.724624 | TASK [health-check/test-nodepool-providers-secrets : Get existing kube.config from the secret] 2026-06-10 00:54:39.270912 | controller | ok: Runtime: 0:00:00.104760 2026-06-10 00:54:39.281976 | 2026-06-10 00:54:39.282124 | TASK [health-check/test-nodepool-providers-secrets : Create a fake clouds.yaml] 2026-06-10 00:54:40.051803 | controller | changed 2026-06-10 00:54:40.061489 | 2026-06-10 00:54:40.061664 | TASK [health-check/test-nodepool-providers-secrets : Update the secret clouds.yaml key] 2026-06-09 20:54:40.518278 | controller | Warning: resource secrets/nodepool-providers-secrets is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. 2026-06-09 20:54:40.656770 | controller | secret/nodepool-providers-secrets configured 2026-06-10 00:54:41.108433 | controller | ok: Runtime: 0:00:00.325434 2026-06-10 00:54:41.117567 | 2026-06-10 00:54:41.117732 | TASK [Wait for secrets to be updated] 2026-06-10 00:54:41.146875 | controller | ok 2026-06-10 00:54:41.175198 | 2026-06-10 00:54:41.175294 | TASK [run the operator in standalone mode] 2026-06-10 00:54:41.201278 | controller | ok 2026-06-10 00:54:41.220735 | 2026-06-10 00:54:41.220858 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 00:54:41.248727 | controller | skipping: Conditional result was False 2026-06-10 00:54:41.260302 | 2026-06-10 00:54:41.260447 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 00:54:41.288096 | controller | skipping: Conditional result was False 2026-06-10 00:54:41.299145 | 2026-06-10 00:54:41.299282 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 20:54:42.077423 | controller | 2026-06-09T20:54:42-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:54:42.077516 | controller | 2026-06-09 20:54:42.087989 | controller | 2026-06-09T20:54:42-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:54:42.088052 | controller | 2026-06-09 20:54:42.102756 | controller | 2026-06-09T20:54:42-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:54:42.123434 | controller | 2026-06-09T20:54:42-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:54:42.138299 | controller | 2026-06-09T20:54:42-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:54:42.710751 | controller | 2026-06-09T20:54:42-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:54:42.716023 | controller | 2026-06-09T20:54:42-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:54:42.867243 | controller | 2026-06-09T20:54:42-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:54:42.874514 | controller | 2026-06-09T20:54:42-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:54:42.874543 | controller | 2026-06-09T20:54:42-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:54:42.882976 | controller | 2026-06-09T20:54:42-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:54:42.893096 | controller | 2026-06-09T20:54:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:54:42.893121 | controller | 2026-06-09T20:54:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:54:43.116283 | controller | 2026-06-09T20:54:43-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:54:43.118785 | controller | 2026-06-09T20:54:43-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:54:43.508311 | controller | 2026-06-09T20:54:43-04:00 INFO Updating configmap, name: np-statsd-config-map 2026-06-09 20:54:43.508409 | controller | 2026-06-09T20:54:43-04:00 INFO Updating object, name: np-statsd-config-map 2026-06-09 20:54:43.529940 | controller | 2026-06-09T20:54:43-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-09 20:54:43.530000 | controller | 2026-06-09T20:54:43-04:00 INFO Updating object, name: nodepool-launcher 2026-06-09 20:54:44.109012 | controller | 2026-06-09T20:54:44-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-09 20:54:44.109072 | controller | 2026-06-09T20:54:44-04:00 INFO Updating object, name: nodepool-builder 2026-06-09 20:54:44.905602 | controller | 2026-06-09T20:54:44-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:54:44.913788 | controller | 2026-06-09T20:54:44-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:54:44.913837 | controller | 2026-06-09T20:54:44-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:54:45.714725 | controller | 2026-06-09T20:54:45-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:54:46.503409 | controller | 2026-06-09T20:54:46-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:54:46.503681 | controller | 2026-06-09T20:54:46-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:54:47.310954 | controller | 2026-06-09T20:54:47-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:54:47.714495 | controller | 2026-06-09T20:54:47-04:00 INFO 2026-06-09 20:54:47.714570 | controller | - Config: OK 2026-06-09 20:54:47.714582 | controller | - Gateway: OK 2026-06-09 20:54:47.714591 | controller | - GitServer: OK 2026-06-09 20:54:47.714600 | controller | - HoundSearch: OK 2026-06-09 20:54:47.714609 | controller | - LogJuicer: OK 2026-06-09 20:54:47.714618 | controller | - Logserver: OK 2026-06-09 20:54:47.714626 | controller | - MariaDB: OK 2026-06-09 20:54:47.714635 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:54:47.714645 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:54:47.714653 | controller | - Zookeeper: OK 2026-06-09 20:54:47.714662 | controller | - Zuul: OK 2026-06-09 20:54:47.714670 | controller | 2026-06-09 20:54:47.714679 | controller | 2026-06-09T20:54:47-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:54:52.723767 | controller | 2026-06-09T20:54:52-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:54:52.743993 | controller | 2026-06-09T20:54:52-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:54:52.762699 | controller | 2026-06-09T20:54:52-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:54:53.333027 | controller | 2026-06-09T20:54:53-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:54:53.345995 | controller | 2026-06-09T20:54:53-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:54:53.346075 | controller | 2026-06-09T20:54:53-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:54:53.356140 | controller | 2026-06-09T20:54:53-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:54:53.373841 | controller | 2026-06-09T20:54:53-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:54:53.373896 | controller | 2026-06-09T20:54:53-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:54:53.742846 | controller | 2026-06-09T20:54:53-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:54:53.746330 | controller | 2026-06-09T20:54:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:54:54.148050 | controller | 2026-06-09T20:54:54-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:54:54.148117 | controller | 2026-06-09T20:54:54-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:54:54.333631 | controller | 2026-06-09T20:54:54-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:54:54.333696 | controller | 2026-06-09T20:54:54-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:54:55.145646 | controller | 2026-06-09T20:54:55-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:54:55.153836 | controller | 2026-06-09T20:54:55-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:54:55.153873 | controller | 2026-06-09T20:54:55-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:54:55.935297 | controller | 2026-06-09T20:54:55-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:54:56.731710 | controller | 2026-06-09T20:54:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:54:56.731790 | controller | 2026-06-09T20:54:56-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:54:57.533101 | controller | 2026-06-09T20:54:57-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:54:57.961310 | controller | 2026-06-09T20:54:57-04:00 INFO 2026-06-09 20:54:57.961388 | controller | - Config: OK 2026-06-09 20:54:57.961399 | controller | - Gateway: OK 2026-06-09 20:54:57.961408 | controller | - GitServer: OK 2026-06-09 20:54:57.961416 | controller | - HoundSearch: OK 2026-06-09 20:54:57.961424 | controller | - LogJuicer: OK 2026-06-09 20:54:57.961431 | controller | - Logserver: OK 2026-06-09 20:54:57.961439 | controller | - MariaDB: OK 2026-06-09 20:54:57.961447 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:54:57.961455 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:54:57.961463 | controller | - Zookeeper: OK 2026-06-09 20:54:57.961471 | controller | - Zuul: OK 2026-06-09 20:54:57.961479 | controller | 2026-06-09 20:54:57.961494 | controller | 2026-06-09T20:54:57-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 20:55:02.971884 | controller | 2026-06-09T20:55:02-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:55:03.004150 | controller | 2026-06-09T20:55:03-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:55:03.029638 | controller | 2026-06-09T20:55:03-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:55:03.581857 | controller | 2026-06-09T20:55:03-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:55:03.592639 | controller | 2026-06-09T20:55:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:55:03.592681 | controller | 2026-06-09T20:55:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:55:03.602850 | controller | 2026-06-09T20:55:03-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:55:03.614942 | controller | 2026-06-09T20:55:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:55:03.615028 | controller | 2026-06-09T20:55:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:55:03.988583 | controller | 2026-06-09T20:55:03-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:55:03.991663 | controller | 2026-06-09T20:55:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:55:04.386680 | controller | 2026-06-09T20:55:04-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:55:04.386739 | controller | 2026-06-09T20:55:04-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:55:04.593082 | controller | 2026-06-09T20:55:04-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:55:04.593463 | controller | 2026-06-09T20:55:04-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:55:05.395173 | controller | 2026-06-09T20:55:05-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:55:05.404995 | controller | 2026-06-09T20:55:05-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:55:05.405068 | controller | 2026-06-09T20:55:05-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:55:06.180376 | controller | 2026-06-09T20:55:06-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:55:06.975373 | controller | 2026-06-09T20:55:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:55:06.975425 | controller | 2026-06-09T20:55:06-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:55:07.784374 | controller | 2026-06-09T20:55:07-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:55:08.182860 | controller | 2026-06-09T20:55:08-04:00 INFO 2026-06-09 20:55:08.182931 | controller | - Config: OK 2026-06-09 20:55:08.182944 | controller | - Gateway: OK 2026-06-09 20:55:08.182953 | controller | - GitServer: OK 2026-06-09 20:55:08.182962 | controller | - HoundSearch: OK 2026-06-09 20:55:08.182971 | controller | - LogJuicer: OK 2026-06-09 20:55:08.182979 | controller | - Logserver: OK 2026-06-09 20:55:08.182988 | controller | - MariaDB: OK 2026-06-09 20:55:08.182997 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:55:08.183006 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:55:08.183014 | controller | - Zookeeper: OK 2026-06-09 20:55:08.183023 | controller | - Zuul: OK 2026-06-09 20:55:08.183032 | controller | 2026-06-09 20:55:08.183049 | controller | 2026-06-09T20:55:08-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 20:55:13.193983 | controller | 2026-06-09T20:55:13-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:55:13.224119 | controller | 2026-06-09T20:55:13-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:55:13.247011 | controller | 2026-06-09T20:55:13-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:55:13.801312 | controller | 2026-06-09T20:55:13-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:55:13.812175 | controller | 2026-06-09T20:55:13-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:55:13.812295 | controller | 2026-06-09T20:55:13-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:55:13.820478 | controller | 2026-06-09T20:55:13-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:55:13.830914 | controller | 2026-06-09T20:55:13-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:55:13.830977 | controller | 2026-06-09T20:55:13-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:55:14.202971 | controller | 2026-06-09T20:55:14-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:55:14.205774 | controller | 2026-06-09T20:55:14-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:55:14.606020 | controller | 2026-06-09T20:55:14-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:55:14.606095 | controller | 2026-06-09T20:55:14-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 20:55:14.804459 | controller | 2026-06-09T20:55:14-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:55:14.804511 | controller | 2026-06-09T20:55:14-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:55:15.621047 | controller | 2026-06-09T20:55:15-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:55:15.629809 | controller | 2026-06-09T20:55:15-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:55:15.629862 | controller | 2026-06-09T20:55:15-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:55:16.406626 | controller | 2026-06-09T20:55:16-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:55:17.198409 | controller | 2026-06-09T20:55:17-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:55:17.198490 | controller | 2026-06-09T20:55:17-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:55:18.005686 | controller | 2026-06-09T20:55:18-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:55:18.406853 | controller | 2026-06-09T20:55:18-04:00 INFO 2026-06-09 20:55:18.406941 | controller | - Config: OK 2026-06-09 20:55:18.406952 | controller | - Gateway: OK 2026-06-09 20:55:18.406976 | controller | - GitServer: OK 2026-06-09 20:55:18.406985 | controller | - HoundSearch: OK 2026-06-09 20:55:18.406993 | controller | - LogJuicer: OK 2026-06-09 20:55:18.407001 | controller | - Logserver: OK 2026-06-09 20:55:18.407009 | controller | - MariaDB: OK 2026-06-09 20:55:18.407017 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:55:18.407025 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 20:55:18.407033 | controller | - Zookeeper: OK 2026-06-09 20:55:18.407040 | controller | - Zuul: OK 2026-06-09 20:55:18.407049 | controller | 2026-06-09 20:55:18.407076 | controller | 2026-06-09T20:55:18-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-09 20:55:23.416518 | controller | 2026-06-09T20:55:23-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:55:23.443864 | controller | 2026-06-09T20:55:23-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:55:23.470068 | controller | 2026-06-09T20:55:23-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:55:24.022212 | controller | 2026-06-09T20:55:24-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:55:24.033771 | controller | 2026-06-09T20:55:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:55:24.033829 | controller | 2026-06-09T20:55:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:55:24.042605 | controller | 2026-06-09T20:55:24-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:55:24.052626 | controller | 2026-06-09T20:55:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:55:24.052658 | controller | 2026-06-09T20:55:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:55:24.432951 | controller | 2026-06-09T20:55:24-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:55:24.436441 | controller | 2026-06-09T20:55:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:55:24.840766 | controller | 2026-06-09T20:55:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:55:24.840835 | controller | 2026-06-09T20:55:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:55:25.037334 | controller | 2026-06-09T20:55:25-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:55:25.037411 | controller | 2026-06-09T20:55:25-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 20:55:25.846819 | controller | 2026-06-09T20:55:25-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:55:25.856914 | controller | 2026-06-09T20:55:25-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:55:25.856986 | controller | 2026-06-09T20:55:25-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:55:26.628617 | controller | 2026-06-09T20:55:26-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:55:27.420825 | controller | 2026-06-09T20:55:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:55:27.420899 | controller | 2026-06-09T20:55:27-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:55:28.225008 | controller | 2026-06-09T20:55:28-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:55:28.629399 | controller | 2026-06-09T20:55:28-04:00 INFO 2026-06-09 20:55:28.629470 | controller | - Config: OK 2026-06-09 20:55:28.629481 | controller | - Gateway: OK 2026-06-09 20:55:28.629490 | controller | - GitServer: OK 2026-06-09 20:55:28.629498 | controller | - HoundSearch: OK 2026-06-09 20:55:28.629506 | controller | - LogJuicer: OK 2026-06-09 20:55:28.629514 | controller | - Logserver: OK 2026-06-09 20:55:28.629522 | controller | - MariaDB: OK 2026-06-09 20:55:28.629530 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 20:55:28.629539 | controller | - NodePoolLauncher: OK 2026-06-09 20:55:28.629547 | controller | - Zookeeper: OK 2026-06-09 20:55:28.629554 | controller | - Zuul: OK 2026-06-09 20:55:28.629563 | controller | 2026-06-09 20:55:28.629571 | controller | 2026-06-09T20:55:28-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-09 20:55:33.640251 | controller | 2026-06-09T20:55:33-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:55:33.660686 | controller | 2026-06-09T20:55:33-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:55:33.681481 | controller | 2026-06-09T20:55:33-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:55:34.247310 | controller | 2026-06-09T20:55:34-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:55:34.257649 | controller | 2026-06-09T20:55:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:55:34.257702 | controller | 2026-06-09T20:55:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:55:34.265691 | controller | 2026-06-09T20:55:34-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:55:34.275975 | controller | 2026-06-09T20:55:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:55:34.276007 | controller | 2026-06-09T20:55:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:55:34.655885 | controller | 2026-06-09T20:55:34-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 20:55:34.659114 | controller | 2026-06-09T20:55:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 20:55:35.062435 | controller | 2026-06-09T20:55:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 20:55:35.062484 | controller | 2026-06-09T20:55:35-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 20:55:35.258712 | controller | 2026-06-09T20:55:35-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 20:55:36.064388 | controller | 2026-06-09T20:55:36-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 20:55:36.072574 | controller | 2026-06-09T20:55:36-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 20:55:36.072631 | controller | 2026-06-09T20:55:36-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 20:55:36.851592 | controller | 2026-06-09T20:55:36-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 20:55:37.644304 | controller | 2026-06-09T20:55:37-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 20:55:37.644401 | controller | 2026-06-09T20:55:37-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 20:55:38.449996 | controller | 2026-06-09T20:55:38-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 20:55:38.853007 | controller | 2026-06-09T20:55:38-04:00 INFO 2026-06-09 20:55:38.853060 | controller | - Config: OK 2026-06-09 20:55:38.853072 | controller | - Gateway: OK 2026-06-09 20:55:38.853081 | controller | - GitServer: OK 2026-06-09 20:55:38.853089 | controller | - HoundSearch: OK 2026-06-09 20:55:38.853097 | controller | - LogJuicer: OK 2026-06-09 20:55:38.853117 | controller | - Logserver: OK 2026-06-09 20:55:38.853125 | controller | - MariaDB: OK 2026-06-09 20:55:38.853132 | controller | - NodePoolBuilder: OK 2026-06-09 20:55:38.853140 | controller | - NodePoolLauncher: OK 2026-06-09 20:55:38.853148 | controller | - Zookeeper: OK 2026-06-09 20:55:38.853156 | controller | - Zuul: OK 2026-06-09 20:55:38.853164 | controller | 2026-06-09 20:55:38.853172 | controller | 2026-06-09T20:55:38-04:00 INFO Updating controller configmap ... 2026-06-09 20:55:39.243688 | controller | 2026-06-09T20:55:39-04:00 INFO Standalone reconcile done. 2026-06-10 00:55:39.546964 | controller | ok: Runtime: 0:00:57.534126 2026-06-10 00:55:39.559181 | 2026-06-10 00:55:39.559322 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 00:55:40.099216 | controller | skipping: Conditional result was False 2026-06-10 00:55:40.119213 | 2026-06-10 00:55:40.119372 | TASK [health-check/test-nodepool-providers-secrets : Check that clouds.yaml available in nodepool-builder] 2026-06-09 20:55:40.465924 | controller | Defaulted container "nodepool-builder" out of: nodepool-builder, fluentbit, np-statsd, nodepool-builder-nodeexporter, build-logs-httpd, nodepool-builder-init (init) 2026-06-09 20:55:40.598271 | controller | /var/lib/nodepool/.config/openstack/clouds.yaml 2026-06-10 00:55:40.654665 | controller | ok: Runtime: 0:00:00.207491 2026-06-10 00:55:40.666466 | 2026-06-10 00:55:40.666715 | TASK [health-check/test-nodepool-providers-secrets : Check that clouds.yaml available in nodepool-launcher containers] 2026-06-09 20:55:41.157925 | controller | Defaulted container "launcher" out of: launcher, np-statsd, zuul-capacity, nodepool-launcher-init (init) 2026-06-09 20:55:41.283403 | controller | /var/lib/nodepool/.config/openstack/clouds.yaml 2026-06-10 00:55:41.721393 | controller | ok: Runtime: 0:00:00.325408 2026-06-10 00:55:41.739370 | 2026-06-10 00:55:41.739522 | TASK [health-check/config-update-nodepool-launcher : Set random value] 2026-06-10 00:55:41.845964 | controller | ok 2026-06-10 00:55:41.855117 | 2026-06-10 00:55:41.855255 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool directory exists] 2026-06-10 00:55:42.126344 | controller | changed 2026-06-10 00:55:42.135989 | 2026-06-10 00:55:42.136130 | TASK [health-check/config-update-nodepool-launcher : Set bad Nodepool config] 2026-06-10 00:55:42.396036 | controller | changed: File created 2026-06-10 00:55:42.405693 | 2026-06-10 00:55:42.405902 | LOOP [health-check/config-update-nodepool-launcher : Commit file in the config repo] 2026-06-10 00:55:42.655020 | controller | ok: Item: git add nodepool/nodepool.yaml Runtime: 0:00:00.008138 2026-06-10 00:55:42.920884 | controller | [master 780a8fa] Add nodepool/nodepool.yaml 2026-06-10 00:55:42.921071 | controller | 1 file changed, 3 insertions(+) 2026-06-10 00:55:42.921125 | controller | create mode 100644 nodepool/nodepool.yaml 2026-06-10 00:55:42.921181 | controller | ok: Item: git commit -m 'Add nodepool/nodepool.yaml' Runtime: 0:00:00.052487 2026-06-10 00:55:42.934152 | 2026-06-10 00:55:42.934366 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 00:55:42.970798 | controller | ok 2026-06-10 00:55:43.013260 | 2026-06-10 00:55:43.013408 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 00:55:43.044618 | controller | ok 2026-06-10 00:55:43.053034 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 00:55:43.060187 | 2026-06-10 00:55:43.060267 | TASK [health-check/repo-submit-change : Submit change] 2026-06-09 20:55:43.821088 | controller | remote: 2026-06-09 20:55:43.821175 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 20:55:43.821187 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-09 20:55:43.821196 | controller | remote: 2026-06-09 20:55:43.821205 | controller | remote: SUCCESS 2026-06-09 20:55:43.821214 | controller | remote: 2026-06-09 20:55:43.821227 | controller | remote: https://gerrit.sfop.me/c/config/+/5 Add nodepool/nodepool.yaml [NEW] 2026-06-09 20:55:43.821276 | controller | remote: 2026-06-09 20:55:43.821285 | controller | To https://gerrit.sfop.me/a/config 2026-06-09 20:55:43.821297 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 00:55:44.098892 | controller | ok: Runtime: 0:00:00.566754 2026-06-10 00:55:44.109303 | 2026-06-10 00:55:44.109443 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-09 20:55:44.452316 | controller | I27f9a671978f26f9eaa4c788569c387363ef5efe 2026-06-10 00:55:44.660207 | controller | ok: Runtime: 0:00:00.010946 2026-06-10 00:55:44.670054 | 2026-06-10 00:55:44.670238 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 00:55:45.306328 | controller | ok: OK (47 bytes) 2026-06-10 00:55:45.318351 | 2026-06-10 00:55:45.318528 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 00:55:45.356440 | controller | ok 2026-06-10 00:55:45.369657 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 00:55:45.383466 | 2026-06-10 00:55:45.383648 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 00:55:45.894894 | controller | ok: OK (7150 bytes) 2026-06-10 00:55:45.905000 | 2026-06-10 00:55:45.905140 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 00:55:45.960361 | controller | ok 2026-06-10 00:55:45.977826 | 2026-06-10 00:55:45.977966 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 00:56:39.086787 | controller | ok: OK (1163 bytes) 2026-06-10 00:56:39.098638 | 2026-06-10 00:56:39.098777 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 00:56:39.131678 | controller | skipping: Conditional result was False 2026-06-10 00:56:39.150880 | 2026-06-10 00:56:39.151032 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 00:56:39.181729 | controller | skipping: Conditional result was False 2026-06-10 00:56:39.192903 | 2026-06-10 00:56:39.193085 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 00:56:39.222852 | controller | skipping: Conditional result was False 2026-06-10 00:56:39.237334 | 2026-06-10 00:56:39.237476 | TASK [health-check/config-update-nodepool-launcher : Reset the config repo] 2026-06-09 20:56:39.515434 | controller | HEAD is now at a33bd73 Add zuul/main.yaml 2026-06-10 00:56:39.782750 | controller | ok: Runtime: 0:00:00.004894 2026-06-10 00:56:39.793475 | 2026-06-10 00:56:39.793682 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool directory exists] 2026-06-10 00:56:40.165328 | controller | changed 2026-06-10 00:56:40.169554 | 2026-06-10 00:56:40.169636 | TASK [health-check/config-update-nodepool-launcher : Check nodepool.yaml exists] 2026-06-10 00:56:40.447084 | controller | ok 2026-06-10 00:56:40.456750 | 2026-06-10 00:56:40.456912 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool.yaml default content] 2026-06-10 00:56:41.263191 | controller | changed 2026-06-10 00:56:41.272445 | 2026-06-10 00:56:41.272635 | TASK [health-check/config-update-nodepool-launcher : Load current nodepool launcher config] 2026-06-09 20:56:41.558207 | controller | labels: [] 2026-06-09 20:56:41.558319 | controller | providers: [] 2026-06-10 00:56:41.821418 | controller | ok: Runtime: 0:00:00.003790 2026-06-10 00:56:41.835321 | 2026-06-10 00:56:41.835490 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 00:56:41.883766 | controller | ok 2026-06-10 00:56:41.894904 | 2026-06-10 00:56:41.895105 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 00:56:41.942295 | controller | ok 2026-06-10 00:56:41.951460 | 2026-06-10 00:56:41.951666 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 00:56:41.980513 | controller | skipping: Conditional result was False 2026-06-10 00:56:41.992014 | 2026-06-10 00:56:41.992173 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 00:56:42.027680 | controller | ok 2026-06-10 00:56:42.036288 | 2026-06-10 00:56:42.036424 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 00:56:42.053874 | controller | skipping: Conditional result was False 2026-06-10 00:56:42.069383 | 2026-06-10 00:56:42.069624 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 00:56:42.108535 | controller | ok 2026-06-10 00:56:42.117488 | 2026-06-10 00:56:42.117658 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 00:56:42.168364 | controller | ok 2026-06-10 00:56:42.181473 | 2026-06-10 00:56:42.181657 | TASK [health-check/config-update-nodepool-launcher : Set dummy Nodepool config] 2026-06-10 00:56:42.987356 | controller | changed 2026-06-10 00:56:42.996929 | 2026-06-10 00:56:42.997069 | LOOP [health-check/config-update-nodepool-launcher : Commit file in the config repo] 2026-06-10 00:56:43.297712 | controller | ok: Item: git add nodepool/nodepool.yaml Runtime: 0:00:00.007026 2026-06-10 00:56:43.298243 | controller | changed: All items complete 2026-06-10 00:56:43.298307 | 2026-06-10 00:56:43.607481 | controller | [master 355afe5] Add nodepool/nodepool.yaml 2026-06-10 00:56:43.607691 | controller | 1 file changed, 12 insertions(+) 2026-06-10 00:56:43.607751 | controller | create mode 100644 nodepool/nodepool.yaml 2026-06-10 00:56:43.607810 | controller | ok: Item: git commit -m 'Add nodepool/nodepool.yaml' Runtime: 0:00:00.062525 2026-06-10 00:56:43.618241 | 2026-06-10 00:56:43.618422 | TASK [Update config repo change] 2026-06-10 00:56:43.651752 | controller | ok 2026-06-10 00:56:43.705852 | 2026-06-10 00:56:43.706085 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 00:56:43.744459 | controller | ok 2026-06-10 00:56:43.758405 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 00:56:43.770392 | 2026-06-10 00:56:43.770526 | TASK [health-check/repo-submit-change : Submit change] 2026-06-09 20:56:44.610158 | controller | remote: 2026-06-09 20:56:44.610266 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 20:56:44.610282 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 20:56:44.610291 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 20:56:44.610303 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-09 20:56:44.610312 | controller | remote: 2026-06-09 20:56:44.610321 | controller | remote: SUCCESS 2026-06-09 20:56:44.610330 | controller | remote: 2026-06-09 20:56:44.610339 | controller | remote: https://gerrit.sfop.me/c/config/+/6 Add nodepool/nodepool.yaml [NEW] 2026-06-09 20:56:44.610348 | controller | remote: 2026-06-09 20:56:44.610356 | controller | To https://gerrit.sfop.me/a/config 2026-06-09 20:56:44.610367 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 00:56:44.812657 | controller | ok: Runtime: 0:00:00.567594 2026-06-10 00:56:44.822760 | 2026-06-10 00:56:44.822908 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-09 20:56:45.117153 | controller | I46d9be3b3a986f8e8425a7278349ffa8ca76a721 2026-06-10 00:56:45.372262 | controller | ok: Runtime: 0:00:00.010681 2026-06-10 00:56:45.382533 | 2026-06-10 00:56:45.382719 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 00:56:45.932680 | controller | ok: OK (47 bytes) 2026-06-10 00:56:45.938906 | 2026-06-10 00:56:45.938999 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 00:56:45.962230 | controller | skipping: Conditional result was False 2026-06-10 00:56:45.967996 | 2026-06-10 00:56:45.968068 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 00:56:46.003262 | controller | skipping: Conditional result was False 2026-06-10 00:56:46.013865 | 2026-06-10 00:56:46.013961 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 00:56:46.058022 | controller | ok 2026-06-10 00:56:46.070807 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 00:56:46.080374 | 2026-06-10 00:56:46.080463 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-09 20:56:46.345497 | controller | 355afe5148178206dc7d0f3ac3dbce99057f5c51 2026-06-10 00:56:46.618092 | controller | ok: Runtime: 0:00:00.008757 2026-06-10 00:56:46.629214 | 2026-06-10 00:56:46.629351 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 00:59:24.590953 | controller | ok: OK (1143 bytes) 2026-06-10 00:59:24.604546 | 2026-06-10 00:59:24.604776 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 00:59:24.642991 | controller | skipping: Conditional result was False 2026-06-10 00:59:24.665134 | 2026-06-10 00:59:24.665326 | TASK [health-check/config-update-nodepool-launcher : Get nodepool-launcher pod name] 2026-06-09 20:59:25.034553 | controller | nodepool-launcher-775f788b46-fpxx9 2026-06-10 00:59:25.210430 | controller | ok: Runtime: 0:00:00.096723 2026-06-10 00:59:25.221369 | 2026-06-10 00:59:25.221513 | LOOP [health-check/config-update-nodepool-launcher : Check that new zuul-worker-sf-operator-ci-9vz6apf6 label exists in the destination file] 2026-06-10 00:59:25.813074 | controller | - name: zuul-worker-sf-operator-ci-9vz6apf6 2026-06-10 00:59:25.813752 | controller | name: zuul-worker-sf-operator-ci-9vz6apf6 2026-06-10 00:59:25.813856 | controller | ok: Item: zuul-worker-sf-operator-ci-9vz6apf6 Runtime: 0:00:00.279291 2026-06-10 00:59:26.245237 | controller | name: microshiftLocal 2026-06-10 00:59:26.245490 | controller | ok: Item: microshiftLocal Runtime: 0:00:00.242049 2026-06-10 00:59:26.264133 | 2026-06-10 00:59:26.264367 | TASK [health-check/config-update-nodepool-launcher : Ensure the new label listed in the Nodepool API] 2026-06-10 00:59:26.872040 | controller | ok: OK (199 bytes) 2026-06-10 00:59:26.876676 | 2026-06-10 00:59:26.876741 | TASK [health-check/config-update-nodepool-launcher : Ensure the new label listed in the Zuul API] 2026-06-10 00:59:27.441643 | controller | ok: OK (49 bytes) 2026-06-10 00:59:27.459462 | 2026-06-10 00:59:27.459635 | TASK [health-check/test-deploy-zero-replica : Set executor replica set to 0] 2026-06-09 20:59:27.821926 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 00:59:28.003057 | controller | ok: Runtime: 0:00:00.102422 2026-06-10 00:59:28.012603 | 2026-06-10 00:59:28.012744 | TASK [health-check/test-deploy-zero-replica : Ensure executor pod is down] 2026-06-09 20:59:28.418354 | controller | NAME READY STATUS RESTARTS AGE 2026-06-09 20:59:28.418458 | controller | config-base-secret-1-2801-ns-4fdwk 0/1 Completed 0 34m 2026-06-09 20:59:28.418470 | controller | config-base-secret-1-945-ns-85xbq 0/1 Completed 0 44m 2026-06-09 20:59:28.418487 | controller | config-base-secret-1-945-ns-v2xnk 0/1 Completed 0 39m 2026-06-09 20:59:28.418500 | controller | gateway-6bbbd758d9-ws9mf 1/1 Running 0 33m 2026-06-09 20:59:28.418509 | controller | gerrit-0 2/2 Running 0 51m 2026-06-09 20:59:28.418519 | controller | git-server-0 2/2 Running 0 17m 2026-06-09 20:59:28.418528 | controller | hound-search-0 1/1 Running 0 49m 2026-06-09 20:59:28.418535 | controller | logjuicer-6ff84764b7-mv8gq 1/1 Running 0 49m 2026-06-09 20:59:28.418543 | controller | logserver-0 4/4 Running 0 28m 2026-06-09 20:59:28.418551 | controller | mariadb-0 3/3 Running 0 25m 2026-06-09 20:59:28.418559 | controller | nodepool-builder-0 5/5 Running 0 4m43s 2026-06-09 20:59:28.418568 | controller | nodepool-launcher-775f788b46-fpxx9 3/3 Running 0 4m45s 2026-06-09 20:59:28.418576 | controller | post-init-wjpvj 0/1 Completed 0 50m 2026-06-09 20:59:28.418587 | controller | test-fluentbit 1/1 Running 0 42m 2026-06-09 20:59:28.418613 | controller | test-loki-8cd4fd56-sbxfw 1/1 Running 0 42m 2026-06-09 20:59:28.418624 | controller | zookeeper-0 3/3 Running 0 25m 2026-06-09 20:59:28.418640 | controller | zuul-db-provision-52vwg 0/1 Completed 0 48m 2026-06-09 20:59:28.418652 | controller | zuul-db-provision-gt92h 0/1 Completed 0 36m 2026-06-09 20:59:28.418663 | controller | zuul-executor-0 2/2 Terminating 0 5m33s 2026-06-09 20:59:28.418674 | controller | zuul-merger-0 2/2 Running 0 16m 2026-06-09 20:59:28.418684 | controller | zuul-merger-1 2/2 Running 0 16m 2026-06-09 20:59:28.418695 | controller | zuul-scheduler-0 3/3 Running 0 12m 2026-06-09 20:59:28.418706 | controller | zuul-web-575996f59f-g4fd2 1/1 Running 0 12m 2026-06-09 20:59:28.418716 | controller | zuul-weeder-5fb87885d-9kqmb 1/1 Running 0 15m 2026-06-10 00:59:38.842340 | controller | ok: Runtime: 0:00:00.114918 2026-06-10 00:59:38.852154 | 2026-06-10 00:59:38.852301 | TASK [health-check/test-deploy-zero-replica : Set zuul-web replica set to 0] 2026-06-09 20:59:39.216819 | controller | deployment.apps/zuul-web scaled 2026-06-10 00:59:39.395545 | controller | ok: Runtime: 0:00:00.097717 2026-06-10 00:59:39.404670 | 2026-06-10 00:59:39.404808 | TASK [health-check/test-deploy-zero-replica : Ensure zuul-web pod is down] 2026-06-09 20:59:39.816862 | controller | NAME READY STATUS RESTARTS AGE 2026-06-09 20:59:39.817113 | controller | config-base-secret-1-2801-ns-4fdwk 0/1 Completed 0 35m 2026-06-09 20:59:39.817332 | controller | config-base-secret-1-945-ns-85xbq 0/1 Completed 0 44m 2026-06-09 20:59:39.817471 | controller | config-base-secret-1-945-ns-v2xnk 0/1 Completed 0 39m 2026-06-09 20:59:39.817615 | controller | gateway-6bbbd758d9-ws9mf 1/1 Running 0 33m 2026-06-09 20:59:39.817769 | controller | gerrit-0 2/2 Running 0 51m 2026-06-09 20:59:39.817921 | controller | git-server-0 2/2 Running 0 17m 2026-06-09 20:59:39.818063 | controller | hound-search-0 1/1 Running 0 50m 2026-06-09 20:59:39.818195 | controller | logjuicer-6ff84764b7-mv8gq 1/1 Running 0 50m 2026-06-09 20:59:39.818376 | controller | logserver-0 4/4 Running 0 28m 2026-06-09 20:59:39.818538 | controller | mariadb-0 3/3 Running 0 25m 2026-06-09 20:59:39.818682 | controller | nodepool-builder-0 5/5 Running 0 4m54s 2026-06-09 20:59:39.818804 | controller | nodepool-launcher-775f788b46-fpxx9 3/3 Running 0 4m56s 2026-06-09 20:59:39.818947 | controller | post-init-wjpvj 0/1 Completed 0 50m 2026-06-09 20:59:39.819089 | controller | test-fluentbit 1/1 Running 0 42m 2026-06-09 20:59:39.819221 | controller | test-loki-8cd4fd56-sbxfw 1/1 Running 0 42m 2026-06-09 20:59:39.819411 | controller | zookeeper-0 3/3 Running 0 25m 2026-06-09 20:59:39.819548 | controller | zuul-db-provision-52vwg 0/1 Completed 0 48m 2026-06-09 20:59:39.819684 | controller | zuul-db-provision-gt92h 0/1 Completed 0 36m 2026-06-09 20:59:39.819826 | controller | zuul-merger-0 2/2 Running 0 16m 2026-06-09 20:59:39.819969 | controller | zuul-merger-1 2/2 Running 0 16m 2026-06-09 20:59:39.820112 | controller | zuul-scheduler-0 3/3 Running 0 12m 2026-06-09 20:59:39.820268 | controller | zuul-web-575996f59f-g4fd2 1/1 Terminating 0 12m 2026-06-09 20:59:39.820408 | controller | zuul-weeder-5fb87885d-9kqmb 1/1 Running 0 15m 2026-06-10 00:59:50.223130 | controller | ok: Runtime: 0:00:00.108203 2026-06-10 00:59:50.232889 | 2026-06-10 00:59:50.233032 | TASK [health-check/test-deploy-zero-replica : Run sf-operator and expect reconciliation] 2026-06-09 20:59:50.777227 | controller | 2026-06-09T20:59:50-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 20:59:50.777359 | controller | 2026-06-09 20:59:50.787425 | controller | 2026-06-09T20:59:50-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 20:59:50.787461 | controller | 2026-06-09 20:59:50.802562 | controller | 2026-06-09T20:59:50-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:59:50.825933 | controller | 2026-06-09T20:59:50-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:59:50.838166 | controller | 2026-06-09T20:59:50-04:00 INFO Statefulset "mariadb" configuration changed, applying... 2026-06-09 20:59:50.838209 | controller | 2026-06-09T20:59:50-04:00 INFO Updating object, name: mariadb 2026-06-09 20:59:51.210566 | controller | 2026-06-09T20:59:51-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 20:59:51.219890 | controller | 2026-06-09T20:59:51-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 20:59:51.319982 | controller | 2026-06-09T20:59:51-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 20:59:51.326771 | controller | 2026-06-09T20:59:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 20:59:51.326835 | controller | 2026-06-09T20:59:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 20:59:51.333872 | controller | 2026-06-09T20:59:51-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 20:59:51.345479 | controller | 2026-06-09T20:59:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 20:59:51.345545 | controller | 2026-06-09T20:59:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 20:59:51.610467 | controller | 2026-06-09T20:59:51-04:00 INFO Statefulset "zookeeper" configuration changed, applying... 2026-06-09 20:59:51.610532 | controller | 2026-06-09T20:59:51-04:00 INFO Updating object, name: zookeeper 2026-06-09 20:59:51.621186 | controller | 2026-06-09T20:59:51-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-09 20:59:51.621282 | controller | 2026-06-09T20:59:51-04:00 INFO 2026-06-09 20:59:51.621298 | controller | - Config: Waiting ... 2026-06-09 20:59:51.621306 | controller | - Gateway: OK 2026-06-09 20:59:51.621313 | controller | - GitServer: OK 2026-06-09 20:59:51.621320 | controller | - HoundSearch: OK 2026-06-09 20:59:51.621326 | controller | - LogJuicer: OK 2026-06-09 20:59:51.621332 | controller | - Logserver: OK 2026-06-09 20:59:51.621376 | controller | - MariaDB: Waiting ... 2026-06-09 20:59:51.621384 | controller | - Zookeeper: Waiting ... 2026-06-09 20:59:51.621391 | controller | 2026-06-09 20:59:51.621398 | controller | 2026-06-09T20:59:51-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 20:59:56.629663 | controller | 2026-06-09T20:59:56-04:00 INFO Deleting object, name: ca-cert 2026-06-09 20:59:56.653431 | controller | 2026-06-09T20:59:56-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 20:59:56.678509 | controller | 2026-06-09T20:59:56-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 20:59:56.678566 | controller | 2026-06-09T20:59:56-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 21:00:01.686058 | controller | 2026-06-09T21:00:01-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:00:01.686136 | controller | 2026-06-09T21:00:01-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 21:00:06.696384 | controller | 2026-06-09T21:00:06-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:00:06.738129 | controller | 2026-06-09T21:00:06-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:00:06.748605 | controller | 2026-06-09T21:00:06-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:00:06.748707 | controller | 2026-06-09T21:00:06-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:00:06.758294 | controller | 2026-06-09T21:00:06-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:00:06.772801 | controller | 2026-06-09T21:00:06-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:00:06.772873 | controller | 2026-06-09T21:00:06-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:00:06.789935 | controller | 2026-06-09T21:00:06-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:00:06.790006 | controller | 2026-06-09T21:00:06-04:00 INFO statefulset rollout not ok, name: zookeeper 2026-06-09 21:00:11.800214 | controller | 2026-06-09T21:00:11-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:00:11.804426 | controller | 2026-06-09T21:00:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:00:11.821641 | controller | 2026-06-09T21:00:11-04:00 INFO Updating configmap, name: nodepool-launcher-extra-config-config-map 2026-06-09 21:00:11.821697 | controller | 2026-06-09T21:00:11-04:00 INFO Updating object, name: nodepool-launcher-extra-config-config-map 2026-06-09 21:00:11.835169 | controller | 2026-06-09T21:00:11-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-09 21:00:11.835218 | controller | 2026-06-09T21:00:11-04:00 INFO Updating object, name: nodepool-launcher 2026-06-09 21:00:11.856812 | controller | 2026-06-09T21:00:11-04:00 INFO Updating configmap, name: nodepool-builder-extra-config-config-map 2026-06-09 21:00:11.856900 | controller | 2026-06-09T21:00:11-04:00 INFO Updating object, name: nodepool-builder-extra-config-config-map 2026-06-09 21:00:11.878852 | controller | 2026-06-09T21:00:11-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-09 21:00:11.878953 | controller | 2026-06-09T21:00:11-04:00 INFO Updating object, name: nodepool-builder 2026-06-09 21:00:11.929074 | controller | 2026-06-09T21:00:11-04:00 INFO Updating secret, name: zuul-config 2026-06-09 21:00:11.929144 | controller | 2026-06-09T21:00:11-04:00 INFO Updating object, name: zuul-config 2026-06-09 21:00:12.416193 | controller | 2026-06-09T21:00:12-04:00 INFO Updating configmap, name: zuul-logging-config-map 2026-06-09 21:00:12.416299 | controller | 2026-06-09T21:00:12-04:00 INFO Updating object, name: zuul-logging-config-map 2026-06-09 21:00:12.622044 | controller | 2026-06-09T21:00:12-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-09 21:00:12.622116 | controller | 2026-06-09T21:00:12-04:00 INFO Updating object, name: zuul-scheduler 2026-06-09 21:00:13.422001 | controller | 2026-06-09T21:00:13-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-09 21:00:13.422066 | controller | 2026-06-09T21:00:13-04:00 INFO Updating object, name: zuul-executor 2026-06-09 21:00:14.219464 | controller | 2026-06-09T21:00:14-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-09 21:00:14.219525 | controller | 2026-06-09T21:00:14-04:00 INFO Updating object, name: zuul-web 2026-06-09 21:00:15.022326 | controller | 2026-06-09T21:00:15-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-09 21:00:15.022403 | controller | 2026-06-09T21:00:15-04:00 INFO Updating object, name: zuul-merger 2026-06-09 21:00:15.035167 | controller | 2026-06-09T21:00:15-04:00 INFO 2026-06-09 21:00:15.035266 | controller | - Config: Waiting ... 2026-06-09 21:00:15.035290 | controller | - Gateway: OK 2026-06-09 21:00:15.035302 | controller | - GitServer: OK 2026-06-09 21:00:15.035311 | controller | - HoundSearch: OK 2026-06-09 21:00:15.035319 | controller | - LogJuicer: OK 2026-06-09 21:00:15.035339 | controller | - Logserver: OK 2026-06-09 21:00:15.035348 | controller | - MariaDB: OK 2026-06-09 21:00:15.035356 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 21:00:15.035365 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 21:00:15.035373 | controller | - Zookeeper: OK 2026-06-09 21:00:15.035381 | controller | - Zuul: Waiting ... 2026-06-09 21:00:15.035389 | controller | - |_ Config Secrets: OK 2026-06-09 21:00:15.035397 | controller | - |_ Executor: Waiting ... 2026-06-09 21:00:15.035405 | controller | - |_ Merger: Waiting ... 2026-06-09 21:00:15.035413 | controller | - |_ Scheduler: Waiting ... 2026-06-09 21:00:15.035421 | controller | - |_ Web: Waiting ... 2026-06-09 21:00:15.035429 | controller | 2026-06-09 21:00:15.035437 | controller | 2026-06-09T21:00:15-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 21:00:20.046700 | controller | 2026-06-09T21:00:20-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:00:20.101828 | controller | 2026-06-09T21:00:20-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:00:20.128993 | controller | 2026-06-09T21:00:20-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:00:20.650529 | controller | 2026-06-09T21:00:20-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:00:20.662600 | controller | 2026-06-09T21:00:20-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:00:20.662655 | controller | 2026-06-09T21:00:20-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:00:20.672322 | controller | 2026-06-09T21:00:20-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:00:20.683902 | controller | 2026-06-09T21:00:20-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:00:20.683947 | controller | 2026-06-09T21:00:20-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:00:21.059870 | controller | 2026-06-09T21:00:21-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:00:21.062574 | controller | 2026-06-09T21:00:21-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:00:21.463969 | controller | 2026-06-09T21:00:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:00:21.464048 | controller | 2026-06-09T21:00:21-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 21:00:21.659171 | controller | 2026-06-09T21:00:21-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:00:21.659314 | controller | 2026-06-09T21:00:21-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 21:00:22.465960 | controller | 2026-06-09T21:00:22-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:00:22.466053 | controller | 2026-06-09T21:00:22-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:00:27.478514 | controller | 2026-06-09T21:00:27-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:00:27.478606 | controller | 2026-06-09T21:00:27-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:00:32.488005 | controller | 2026-06-09T21:00:32-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:00:32.488097 | controller | 2026-06-09T21:00:32-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:00:37.497099 | controller | 2026-06-09T21:00:37-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:00:37.497192 | controller | 2026-06-09T21:00:37-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:00:42.508858 | controller | 2026-06-09T21:00:42-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:00:42.509006 | controller | 2026-06-09T21:00:42-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:00:47.518688 | controller | 2026-06-09T21:00:47-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:00:47.518793 | controller | 2026-06-09T21:00:47-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:00:52.525477 | controller | 2026-06-09T21:00:52-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 21:00:52.565471 | controller | 2026-06-09T21:00:52-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:00:52.586094 | controller | 2026-06-09T21:00:52-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:00:52.941441 | controller | 2026-06-09T21:00:52-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:00:52.941577 | controller | 2026-06-09T21:00:52-04:00 INFO statefulset rollout not ok, name: zuul-merger 2026-06-09 21:00:57.954210 | controller | 2026-06-09T21:00:57-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:00:57.954320 | controller | 2026-06-09T21:00:57-04:00 INFO statefulset rollout not ok, name: zuul-merger 2026-06-09 21:01:02.963193 | controller | 2026-06-09T21:01:02-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:01:02.963282 | controller | 2026-06-09T21:01:02-04:00 INFO statefulset rollout not ok, name: zuul-merger 2026-06-09 21:01:07.971201 | controller | 2026-06-09T21:01:07-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:01:07.971307 | controller | 2026-06-09T21:01:07-04:00 INFO statefulset rollout not ok, name: zuul-merger 2026-06-09 21:01:12.977931 | controller | 2026-06-09T21:01:12-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:01:12.977999 | controller | 2026-06-09T21:01:12-04:00 INFO statefulset rollout not ok, name: zuul-merger 2026-06-09 21:01:17.989157 | controller | 2026-06-09T21:01:17-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:01:17.992131 | controller | 2026-06-09T21:01:17-04:00 INFO 2026-06-09 21:01:17.992172 | controller | - Config: Waiting ... 2026-06-09 21:01:17.992183 | controller | - Gateway: OK 2026-06-09 21:01:17.992192 | controller | - GitServer: OK 2026-06-09 21:01:17.992200 | controller | - HoundSearch: OK 2026-06-09 21:01:17.992208 | controller | - LogJuicer: OK 2026-06-09 21:01:17.992216 | controller | - Logserver: OK 2026-06-09 21:01:17.992225 | controller | - MariaDB: OK 2026-06-09 21:01:17.992271 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 21:01:17.992281 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 21:01:17.992305 | controller | - Zookeeper: OK 2026-06-09 21:01:17.992313 | controller | - Zuul: Waiting ... 2026-06-09 21:01:17.992321 | controller | - |_ Config Secrets: OK 2026-06-09 21:01:17.992329 | controller | - |_ Executor: OK 2026-06-09 21:01:17.992337 | controller | - |_ Merger: OK 2026-06-09 21:01:17.992345 | controller | - |_ Scheduler: Waiting ... 2026-06-09 21:01:17.992353 | controller | - |_ Web: OK 2026-06-09 21:01:17.992362 | controller | 2026-06-09 21:01:17.992370 | controller | 2026-06-09T21:01:17-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 21:01:23.001066 | controller | 2026-06-09T21:01:23-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:01:23.029804 | controller | 2026-06-09T21:01:23-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:01:23.047616 | controller | 2026-06-09T21:01:23-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:01:23.609470 | controller | 2026-06-09T21:01:23-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:01:23.620082 | controller | 2026-06-09T21:01:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:01:23.620119 | controller | 2026-06-09T21:01:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:01:23.631771 | controller | 2026-06-09T21:01:23-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:01:23.643054 | controller | 2026-06-09T21:01:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:01:23.643129 | controller | 2026-06-09T21:01:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:01:24.013775 | controller | 2026-06-09T21:01:24-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:01:24.016961 | controller | 2026-06-09T21:01:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:01:24.420857 | controller | 2026-06-09T21:01:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:01:24.420951 | controller | 2026-06-09T21:01:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:01:24.615467 | controller | 2026-06-09T21:01:24-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:01:25.423864 | controller | 2026-06-09T21:01:25-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:01:25.423940 | controller | 2026-06-09T21:01:25-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:01:30.430334 | controller | 2026-06-09T21:01:30-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:01:30.430439 | controller | 2026-06-09T21:01:30-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:01:35.440071 | controller | 2026-06-09T21:01:35-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:01:35.440157 | controller | 2026-06-09T21:01:35-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:01:40.448406 | controller | 2026-06-09T21:01:40-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:01:40.448483 | controller | 2026-06-09T21:01:40-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:01:45.460628 | controller | 2026-06-09T21:01:45-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:01:45.470626 | controller | 2026-06-09T21:01:45-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-09 21:01:45.471015 | controller | 2026-06-09T21:01:45-04:00 INFO Updating object, name: zuul-weeder 2026-06-09 21:01:45.493972 | controller | 2026-06-09T21:01:45-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:01:45.507071 | controller | 2026-06-09T21:01:45-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:01:45.890827 | controller | 2026-06-09T21:01:45-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:01:45.893857 | controller | 2026-06-09T21:01:45-04:00 INFO 2026-06-09 21:01:45.893884 | controller | - Config: Waiting ... 2026-06-09 21:01:45.893892 | controller | - Gateway: OK 2026-06-09 21:01:45.893898 | controller | - GitServer: OK 2026-06-09 21:01:45.893903 | controller | - HoundSearch: OK 2026-06-09 21:01:45.893909 | controller | - LogJuicer: OK 2026-06-09 21:01:45.893914 | controller | - Logserver: OK 2026-06-09 21:01:45.893920 | controller | - MariaDB: OK 2026-06-09 21:01:45.893925 | controller | - NodePoolBuilder: OK 2026-06-09 21:01:45.893930 | controller | - NodePoolLauncher: OK 2026-06-09 21:01:45.893936 | controller | - Zookeeper: OK 2026-06-09 21:01:45.893941 | controller | - Zuul: Waiting ... 2026-06-09 21:01:45.893946 | controller | - |_ Config Secrets: OK 2026-06-09 21:01:45.893964 | controller | - |_ Executor: OK 2026-06-09 21:01:45.893971 | controller | - |_ Merger: OK 2026-06-09 21:01:45.893977 | controller | - |_ Scheduler: Waiting ... 2026-06-09 21:01:45.893983 | controller | - |_ Web: OK 2026-06-09 21:01:45.893989 | controller | 2026-06-09 21:01:45.893998 | controller | 2026-06-09T21:01:45-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-09 21:01:50.901436 | controller | 2026-06-09T21:01:50-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:01:50.928030 | controller | 2026-06-09T21:01:50-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:01:50.947494 | controller | 2026-06-09T21:01:50-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:01:51.506771 | controller | 2026-06-09T21:01:51-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:01:51.514217 | controller | 2026-06-09T21:01:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:01:51.514279 | controller | 2026-06-09T21:01:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:01:51.523776 | controller | 2026-06-09T21:01:51-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:01:51.541220 | controller | 2026-06-09T21:01:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:01:51.541331 | controller | 2026-06-09T21:01:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:01:51.914205 | controller | 2026-06-09T21:01:51-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:01:51.917658 | controller | 2026-06-09T21:01:51-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:01:52.315680 | controller | 2026-06-09T21:01:52-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:01:52.315748 | controller | 2026-06-09T21:01:52-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:01:52.516706 | controller | 2026-06-09T21:01:52-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:01:53.323659 | controller | 2026-06-09T21:01:53-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:01:53.332176 | controller | 2026-06-09T21:01:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:01:53.332293 | controller | 2026-06-09T21:01:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:01:53.511791 | controller | 2026-06-09T21:01:53-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:01:54.305516 | controller | 2026-06-09T21:01:54-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:01:55.113505 | controller | 2026-06-09T21:01:55-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:01:55.519007 | controller | 2026-06-09T21:01:55-04:00 INFO 2026-06-09 21:01:55.519119 | controller | - Config: OK 2026-06-09 21:01:55.519137 | controller | - Gateway: OK 2026-06-09 21:01:55.519149 | controller | - GitServer: OK 2026-06-09 21:01:55.519160 | controller | - HoundSearch: OK 2026-06-09 21:01:55.519171 | controller | - LogJuicer: OK 2026-06-09 21:01:55.519182 | controller | - Logserver: OK 2026-06-09 21:01:55.519194 | controller | - MariaDB: OK 2026-06-09 21:01:55.519205 | controller | - NodePoolBuilder: OK 2026-06-09 21:01:55.519217 | controller | - NodePoolLauncher: OK 2026-06-09 21:01:55.519227 | controller | - Zookeeper: OK 2026-06-09 21:01:55.519311 | controller | - Zuul: OK 2026-06-09 21:01:55.519321 | controller | 2026-06-09 21:01:55.519378 | controller | 2026-06-09T21:01:55-04:00 INFO Updating controller configmap ... 2026-06-09 21:01:55.904067 | controller | 2026-06-09T21:01:55-04:00 INFO Standalone reconcile done. 2026-06-10 01:01:55.958695 | controller | ok: Runtime: 0:02:05.401345 2026-06-10 01:01:55.972225 | 2026-06-10 01:01:55.972370 | TASK [health-check/test-deploy-zero-replica : Set executor replica set to 1] 2026-06-09 21:01:56.354331 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 01:01:56.515426 | controller | ok: Runtime: 0:00:00.111539 2026-06-10 01:01:56.525701 | 2026-06-10 01:01:56.525865 | TASK [health-check/test-deploy-zero-replica : Set web replica set to 1] 2026-06-09 21:01:56.898711 | controller | deployment.apps/zuul-web scaled 2026-06-10 01:01:57.073452 | controller | ok: Runtime: 0:00:00.085952 2026-06-10 01:01:57.084491 | 2026-06-10 01:01:57.084673 | TASK [health-check/test-deploy-zero-replica : Ensure executor pod is up] 2026-06-09 21:01:57.443980 | controller | NAME READY STATUS RESTARTS AGE 2026-06-09 21:01:57.444609 | controller | zuul-executor-0 0/2 ContainerCreating 0 1s 2026-06-10 01:02:18.147819 | controller | ok: Runtime: 0:00:00.092538 2026-06-10 01:02:18.157372 | 2026-06-10 01:02:18.157518 | TASK [health-check/test-deploy-zero-replica : Ensure web pod is up] 2026-06-09 21:02:18.510584 | controller | NAME READY STATUS RESTARTS AGE 2026-06-09 21:02:18.510710 | controller | gateway-6bbbd758d9-ws9mf 1/1 Running 0 36m 2026-06-09 21:02:18.510729 | controller | gerrit-0 2/2 Running 0 54m 2026-06-09 21:02:18.510742 | controller | git-server-0 2/2 Running 0 20m 2026-06-09 21:02:18.510764 | controller | hound-search-0 1/1 Running 0 52m 2026-06-09 21:02:18.510782 | controller | logjuicer-6ff84764b7-mv8gq 1/1 Running 0 52m 2026-06-09 21:02:18.510796 | controller | logserver-0 4/4 Running 0 30m 2026-06-09 21:02:18.510810 | controller | mariadb-0 2/2 Running 0 2m26s 2026-06-09 21:02:18.510821 | controller | nodepool-builder-0 4/4 Running 0 2m5s 2026-06-09 21:02:18.510829 | controller | nodepool-launcher-6cffc7496d-p78l8 3/3 Running 0 2m7s 2026-06-09 21:02:18.510837 | controller | test-fluentbit 1/1 Running 0 45m 2026-06-09 21:02:18.510845 | controller | test-loki-8cd4fd56-sbxfw 1/1 Running 0 45m 2026-06-09 21:02:18.510854 | controller | zookeeper-0 2/2 Running 0 2m25s 2026-06-09 21:02:18.510862 | controller | zuul-executor-0 2/2 Running 0 22s 2026-06-09 21:02:18.510874 | controller | zuul-merger-0 2/2 Running 0 77s 2026-06-09 21:02:18.510882 | controller | zuul-merger-1 2/2 Running 0 2m1s 2026-06-09 21:02:18.510890 | controller | zuul-scheduler-0 3/3 Running 0 95s 2026-06-09 21:02:18.510898 | controller | zuul-web-594856d466-47nx8 0/1 Running 0 22s 2026-06-09 21:02:18.510906 | controller | zuul-weeder-57c6567f5f-gzfkl 1/1 Running 0 33s 2026-06-10 01:02:18.706982 | controller | ok: Runtime: 0:00:00.108406 2026-06-10 01:02:18.717434 | 2026-06-10 01:02:18.717567 | TASK [Run dry-run update detection test] 2026-06-10 01:02:18.753801 | controller | ok 2026-06-10 01:02:18.801173 | 2026-06-10 01:02:18.801418 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-executor] 2026-06-10 01:02:19.338795 | controller | ok: Runtime: 0:00:00.101801 2026-06-10 01:02:19.348111 | 2026-06-10 01:02:19.348261 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-web] 2026-06-10 01:02:19.895372 | controller | ok: Runtime: 0:00:00.101825 2026-06-10 01:02:19.902091 | 2026-06-10 01:02:19.902281 | TASK [test-dry-run-update-detection : Patch zuul-executor statefulset template to trigger reconciliation] 2026-06-09 21:02:20.361400 | controller | statefulset.apps/zuul-executor patched 2026-06-10 01:02:20.526368 | controller | ok: Runtime: 0:00:00.118397 2026-06-10 01:02:20.536456 | 2026-06-10 01:02:20.536626 | TASK [test-dry-run-update-detection : Patch zuul-web deployment template to trigger reconciliation] 2026-06-09 21:02:20.897314 | controller | deployment.apps/zuul-web patched 2026-06-10 01:02:21.085948 | controller | ok: Runtime: 0:00:00.088601 2026-06-10 01:02:21.095420 | 2026-06-10 01:02:21.095547 | TASK [Run deploy with --dry-run to check for update detection] 2026-06-10 01:02:21.133399 | controller | ok 2026-06-10 01:02:21.168120 | 2026-06-10 01:02:21.168226 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 01:02:21.405693 | controller | ok 2026-06-10 01:02:21.416003 | 2026-06-10 01:02:21.416137 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 01:02:21.645254 | controller | ok 2026-06-10 01:02:21.655730 | 2026-06-10 01:02:21.655869 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 21:02:22.220663 | controller | 2026-06-09T21:02:22-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 21:02:22.220753 | controller | 2026-06-09 21:02:22.228578 | controller | 2026-06-09T21:02:22-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 21:02:22.228622 | controller | 2026-06-09 21:02:22.243258 | controller | 2026-06-09T21:02:22-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-09 21:02:22.857016 | controller | 2026-06-09T21:02:22-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 21:02:22.965356 | controller | 2026-06-09T21:02:22-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 21:02:23.253869 | controller | 2026-06-09T21:02:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:02:25.043475 | controller | 2026-06-09T21:02:25-04:00 INFO [Dry Run] Statefulset "zuul-executor" configuration changed, applying... 2026-06-09 21:02:25.043518 | controller | 2026-06-09T21:02:25-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '11' to '0' 2026-06-09 21:02:25.043524 | controller | 2026-06-09T21:02:25-04:00 INFO Updating object, name: zuul-executor (server dry run) 2026-06-09 21:02:25.845257 | controller | 2026-06-09T21:02:25-04:00 INFO [Dry Run] Statefulset "zuul-web" configuration changed, applying... 2026-06-09 21:02:25.845344 | controller | 2026-06-09T21:02:25-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-09 21:02:25.845358 | controller | 2026-06-09T21:02:25-04:00 INFO Updating object, name: zuul-web (server dry run) 2026-06-09 21:02:26.650455 | controller | 2026-06-09T21:02:26-04:00 INFO 2026-06-09 21:02:26.650532 | controller | - Config: Waiting ... 2026-06-09 21:02:26.650543 | controller | - Gateway: OK 2026-06-09 21:02:26.650552 | controller | - GitServer: OK 2026-06-09 21:02:26.650560 | controller | - HoundSearch: OK 2026-06-09 21:02:26.650568 | controller | - LogJuicer: OK 2026-06-09 21:02:26.650576 | controller | - Logserver: OK 2026-06-09 21:02:26.650584 | controller | - MariaDB: OK 2026-06-09 21:02:26.650592 | controller | - NodePoolBuilder: OK 2026-06-09 21:02:26.650600 | controller | - NodePoolLauncher: OK 2026-06-09 21:02:26.650608 | controller | - Zookeeper: OK 2026-06-09 21:02:26.650616 | controller | - Zuul: Waiting ... 2026-06-09 21:02:26.650624 | controller | - |_ Config Secrets: OK 2026-06-09 21:02:26.650632 | controller | - |_ Executor: Waiting ... 2026-06-09 21:02:26.650640 | controller | - |_ Merger: OK 2026-06-09 21:02:26.650648 | controller | - |_ Scheduler: OK 2026-06-09 21:02:26.650656 | controller | - |_ Web: Waiting ... 2026-06-09 21:02:26.650664 | controller | 2026-06-09 21:02:26.650674 | controller | 2026-06-09T21:02:26-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 01:02:26.711643 | controller | ok: Runtime: 0:00:04.750371 2026-06-10 01:02:26.723429 | 2026-06-10 01:02:26.723565 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 01:02:27.272370 | controller | ok: Runtime: 0:00:00.005507 2026-06-10 01:02:27.292225 | 2026-06-10 01:02:27.292400 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-executor] 2026-06-09 21:02:27.578080 | controller | 2026-06-09T21:02:25-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '11' to '0' 2026-06-10 01:02:27.844635 | controller | ok: Runtime: 0:00:00.008037 2026-06-10 01:02:27.854036 | 2026-06-10 01:02:27.854174 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-web] 2026-06-09 21:02:28.144954 | controller | 2026-06-09T21:02:25-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 01:02:28.406249 | controller | ok: Runtime: 0:00:00.004378 2026-06-10 01:02:28.416031 | 2026-06-10 01:02:28.416172 | TASK [test-dry-run-update-detection : Restore original serial for zuul-executor] 2026-06-09 21:02:28.827399 | controller | statefulset.apps/zuul-executor patched 2026-06-10 01:02:28.961524 | controller | ok: Runtime: 0:00:00.115308 2026-06-10 01:02:28.971048 | 2026-06-10 01:02:28.971186 | TASK [test-dry-run-update-detection : Restore original serial for zuul-web] 2026-06-09 21:02:29.342458 | controller | deployment.apps/zuul-web patched 2026-06-10 01:02:29.518086 | controller | ok: Runtime: 0:00:00.098836 2026-06-10 01:02:29.541974 | 2026-06-10 01:02:29.542109 | TASK [health-check/test-dry-run-lifecycle : Create a temporary directory for generated CRs] 2026-06-10 01:02:29.822010 | controller | changed 2026-06-10 01:02:29.831535 | 2026-06-10 01:02:29.831700 | TASK [health-check/test-dry-run-lifecycle : Generate CR with zuul-executor disabled] 2026-06-10 01:02:30.590832 | controller | changed 2026-06-10 01:02:30.600774 | 2026-06-10 01:02:30.600899 | TASK [Deploy SF without zuul-executor to ensure its removal] 2026-06-10 01:02:30.637275 | controller | ok 2026-06-10 01:02:30.668158 | 2026-06-10 01:02:30.668246 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 01:02:30.945991 | controller | ok 2026-06-10 01:02:30.957642 | 2026-06-10 01:02:30.957780 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 01:02:31.195412 | controller | changed 2026-06-10 01:02:31.205336 | 2026-06-10 01:02:31.205478 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 21:02:31.805881 | controller | 2026-06-09T21:02:31-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 21:02:31.805944 | controller | 2026-06-09 21:02:31.815537 | controller | 2026-06-09T21:02:31-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 21:02:31.815564 | controller | 2026-06-09 21:02:31.832988 | controller | 2026-06-09T21:02:31-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:02:31.857928 | controller | 2026-06-09T21:02:31-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:02:31.872373 | controller | 2026-06-09T21:02:31-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:02:32.442557 | controller | 2026-06-09T21:02:32-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:02:32.450126 | controller | 2026-06-09T21:02:32-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 21:02:32.588686 | controller | 2026-06-09T21:02:32-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 21:02:32.598036 | controller | 2026-06-09T21:02:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:02:32.598113 | controller | 2026-06-09T21:02:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:02:32.608857 | controller | 2026-06-09T21:02:32-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:02:32.621931 | controller | 2026-06-09T21:02:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:02:32.621991 | controller | 2026-06-09T21:02:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:02:32.844029 | controller | 2026-06-09T21:02:32-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:02:32.846325 | controller | 2026-06-09T21:02:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:02:33.252826 | controller | 2026-06-09T21:02:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:02:33.252891 | controller | 2026-06-09T21:02:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:02:33.449364 | controller | 2026-06-09T21:02:33-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:02:33.455533 | controller | 2026-06-09T21:02:33-04:00 INFO zuul-executor is disabled but running. Deleting the executor ... 2026-06-09 21:02:33.455561 | controller | 2026-06-09T21:02:33-04:00 INFO Deleting object, name: zuul-executor 2026-06-09 21:02:34.253852 | controller | 2026-06-09T21:02:34-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:02:34.261557 | controller | 2026-06-09T21:02:34-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:02:34.261628 | controller | 2026-06-09T21:02:34-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:02:34.632588 | controller | 2026-06-09T21:02:34-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:02:34.632661 | controller | 2026-06-09T21:02:34-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:02:35.438112 | controller | 2026-06-09T21:02:35-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:02:35.442394 | controller | 2026-06-09T21:02:35-04:00 INFO 2026-06-09 21:02:35.442451 | controller | - Config: Waiting ... 2026-06-09 21:02:35.442465 | controller | - Gateway: OK 2026-06-09 21:02:35.442477 | controller | - GitServer: OK 2026-06-09 21:02:35.442488 | controller | - HoundSearch: OK 2026-06-09 21:02:35.442499 | controller | - LogJuicer: OK 2026-06-09 21:02:35.442510 | controller | - Logserver: OK 2026-06-09 21:02:35.442521 | controller | - MariaDB: OK 2026-06-09 21:02:35.442534 | controller | - NodePoolBuilder: OK 2026-06-09 21:02:35.442545 | controller | - NodePoolLauncher: OK 2026-06-09 21:02:35.442557 | controller | - Zookeeper: OK 2026-06-09 21:02:35.442568 | controller | - Zuul: Waiting ... 2026-06-09 21:02:35.442579 | controller | - |_ Config Secrets: OK 2026-06-09 21:02:35.442590 | controller | - |_ Merger: OK 2026-06-09 21:02:35.442600 | controller | - |_ Scheduler: OK 2026-06-09 21:02:35.442611 | controller | - |_ Web: Waiting ... 2026-06-09 21:02:35.442623 | controller | 2026-06-09 21:02:35.442634 | controller | 2026-06-09T21:02:35-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 21:02:40.448890 | controller | 2026-06-09T21:02:40-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:02:40.475699 | controller | 2026-06-09T21:02:40-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:02:40.491481 | controller | 2026-06-09T21:02:40-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:02:41.057604 | controller | 2026-06-09T21:02:41-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:02:41.067390 | controller | 2026-06-09T21:02:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:02:41.067446 | controller | 2026-06-09T21:02:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:02:41.076722 | controller | 2026-06-09T21:02:41-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:02:41.088590 | controller | 2026-06-09T21:02:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:02:41.088638 | controller | 2026-06-09T21:02:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:02:41.460403 | controller | 2026-06-09T21:02:41-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:02:41.463452 | controller | 2026-06-09T21:02:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:02:41.865550 | controller | 2026-06-09T21:02:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:02:41.865646 | controller | 2026-06-09T21:02:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:02:42.054510 | controller | 2026-06-09T21:02:42-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:02:42.867134 | controller | 2026-06-09T21:02:42-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:02:42.876142 | controller | 2026-06-09T21:02:42-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:02:42.876265 | controller | 2026-06-09T21:02:42-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:02:43.052400 | controller | 2026-06-09T21:02:43-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:02:43.052457 | controller | 2026-06-09T21:02:43-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:02:43.859859 | controller | 2026-06-09T21:02:43-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:02:43.863986 | controller | 2026-06-09T21:02:43-04:00 INFO 2026-06-09 21:02:43.864025 | controller | - Config: Waiting ... 2026-06-09 21:02:43.864036 | controller | - Gateway: OK 2026-06-09 21:02:43.864045 | controller | - GitServer: OK 2026-06-09 21:02:43.864053 | controller | - HoundSearch: OK 2026-06-09 21:02:43.864061 | controller | - LogJuicer: OK 2026-06-09 21:02:43.864069 | controller | - Logserver: OK 2026-06-09 21:02:43.864077 | controller | - MariaDB: OK 2026-06-09 21:02:43.864085 | controller | - NodePoolBuilder: OK 2026-06-09 21:02:43.864092 | controller | - NodePoolLauncher: OK 2026-06-09 21:02:43.864100 | controller | - Zookeeper: OK 2026-06-09 21:02:43.864108 | controller | - Zuul: Waiting ... 2026-06-09 21:02:43.864116 | controller | - |_ Config Secrets: OK 2026-06-09 21:02:43.864124 | controller | - |_ Merger: OK 2026-06-09 21:02:43.864132 | controller | - |_ Scheduler: OK 2026-06-09 21:02:43.864142 | controller | - |_ Web: Waiting ... 2026-06-09 21:02:43.864151 | controller | 2026-06-09 21:02:43.864165 | controller | 2026-06-09T21:02:43-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 21:02:48.871730 | controller | 2026-06-09T21:02:48-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:02:48.899202 | controller | 2026-06-09T21:02:48-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:02:48.916783 | controller | 2026-06-09T21:02:48-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:02:49.479728 | controller | 2026-06-09T21:02:49-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:02:49.489948 | controller | 2026-06-09T21:02:49-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:02:49.490043 | controller | 2026-06-09T21:02:49-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:02:49.498995 | controller | 2026-06-09T21:02:49-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:02:49.511671 | controller | 2026-06-09T21:02:49-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:02:49.511781 | controller | 2026-06-09T21:02:49-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:02:49.883742 | controller | 2026-06-09T21:02:49-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:02:49.885800 | controller | 2026-06-09T21:02:49-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:02:50.291510 | controller | 2026-06-09T21:02:50-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:02:50.291600 | controller | 2026-06-09T21:02:50-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:02:50.483728 | controller | 2026-06-09T21:02:50-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:02:51.290913 | controller | 2026-06-09T21:02:51-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:02:51.298503 | controller | 2026-06-09T21:02:51-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:02:51.298549 | controller | 2026-06-09T21:02:51-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:02:51.476140 | controller | 2026-06-09T21:02:51-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:02:51.476204 | controller | 2026-06-09T21:02:51-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:02:52.281316 | controller | 2026-06-09T21:02:52-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:02:52.283527 | controller | 2026-06-09T21:02:52-04:00 INFO 2026-06-09 21:02:52.283552 | controller | - Config: Waiting ... 2026-06-09 21:02:52.283563 | controller | - Gateway: OK 2026-06-09 21:02:52.283572 | controller | - GitServer: OK 2026-06-09 21:02:52.283581 | controller | - HoundSearch: OK 2026-06-09 21:02:52.283589 | controller | - LogJuicer: OK 2026-06-09 21:02:52.283598 | controller | - Logserver: OK 2026-06-09 21:02:52.283607 | controller | - MariaDB: OK 2026-06-09 21:02:52.283615 | controller | - NodePoolBuilder: OK 2026-06-09 21:02:52.283624 | controller | - NodePoolLauncher: OK 2026-06-09 21:02:52.283632 | controller | - Zookeeper: OK 2026-06-09 21:02:52.283641 | controller | - Zuul: Waiting ... 2026-06-09 21:02:52.283649 | controller | - |_ Config Secrets: OK 2026-06-09 21:02:52.283658 | controller | - |_ Merger: OK 2026-06-09 21:02:52.283667 | controller | - |_ Scheduler: OK 2026-06-09 21:02:52.283675 | controller | - |_ Web: Waiting ... 2026-06-09 21:02:52.283684 | controller | 2026-06-09 21:02:52.283695 | controller | 2026-06-09T21:02:52-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 21:02:57.293202 | controller | 2026-06-09T21:02:57-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:02:57.324913 | controller | 2026-06-09T21:02:57-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:02:57.342381 | controller | 2026-06-09T21:02:57-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:02:57.899886 | controller | 2026-06-09T21:02:57-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:02:57.910913 | controller | 2026-06-09T21:02:57-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:02:57.910967 | controller | 2026-06-09T21:02:57-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:02:57.920688 | controller | 2026-06-09T21:02:57-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:02:57.932323 | controller | 2026-06-09T21:02:57-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:02:57.932369 | controller | 2026-06-09T21:02:57-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:02:58.302951 | controller | 2026-06-09T21:02:58-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:02:58.306857 | controller | 2026-06-09T21:02:58-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:02:58.713891 | controller | 2026-06-09T21:02:58-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:02:58.713981 | controller | 2026-06-09T21:02:58-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:02:58.909733 | controller | 2026-06-09T21:02:58-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:02:59.717984 | controller | 2026-06-09T21:02:59-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:02:59.726704 | controller | 2026-06-09T21:02:59-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:02:59.726768 | controller | 2026-06-09T21:02:59-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:02:59.896781 | controller | 2026-06-09T21:02:59-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:02:59.896862 | controller | 2026-06-09T21:02:59-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 21:03:00.702576 | controller | 2026-06-09T21:03:00-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:03:01.110465 | controller | 2026-06-09T21:03:01-04:00 INFO 2026-06-09 21:03:01.110520 | controller | - Config: OK 2026-06-09 21:03:01.110531 | controller | - Gateway: OK 2026-06-09 21:03:01.110539 | controller | - GitServer: OK 2026-06-09 21:03:01.110547 | controller | - HoundSearch: OK 2026-06-09 21:03:01.110555 | controller | - LogJuicer: OK 2026-06-09 21:03:01.110563 | controller | - Logserver: OK 2026-06-09 21:03:01.110571 | controller | - MariaDB: OK 2026-06-09 21:03:01.110593 | controller | - NodePoolBuilder: OK 2026-06-09 21:03:01.110602 | controller | - NodePoolLauncher: OK 2026-06-09 21:03:01.110610 | controller | - Zookeeper: OK 2026-06-09 21:03:01.110618 | controller | - Zuul: OK 2026-06-09 21:03:01.110626 | controller | 2026-06-09 21:03:01.110637 | controller | 2026-06-09T21:03:01-04:00 INFO Updating controller configmap ... 2026-06-09 21:03:01.496468 | controller | 2026-06-09T21:03:01-04:00 INFO Standalone reconcile done. 2026-06-10 01:03:01.793121 | controller | ok: Runtime: 0:00:30.035328 2026-06-10 01:03:01.804515 | 2026-06-10 01:03:01.804717 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 01:03:02.351903 | controller | ok: Runtime: 0:00:00.007693 2026-06-10 01:03:02.370755 | 2026-06-10 01:03:02.370929 | TASK [health-check/test-dry-run-lifecycle : Ensure zuul-executor StatefulSet is not present] 2026-06-09 21:03:02.723014 | controller | Error from server (NotFound): statefulsets.apps "zuul-executor" not found 2026-06-10 01:03:02.919810 | controller | ok: Runtime: 0:00:00.098368 2026-06-10 01:03:02.929781 | 2026-06-10 01:03:02.929914 | TASK [Run update detection test on zuul-web only] 2026-06-10 01:03:02.967317 | controller | ok 2026-06-10 01:03:03.131678 | 2026-06-10 01:03:03.131850 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-executor] 2026-06-10 01:03:03.654149 | controller | skipping: Conditional result was False 2026-06-10 01:03:03.665622 | 2026-06-10 01:03:03.665809 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-web] 2026-06-10 01:03:04.212949 | controller | ok: Runtime: 0:00:00.098705 2026-06-10 01:03:04.222470 | 2026-06-10 01:03:04.222657 | TASK [test-dry-run-update-detection : Patch zuul-executor statefulset template to trigger reconciliation] 2026-06-10 01:03:04.766200 | controller | skipping: Conditional result was False 2026-06-10 01:03:04.777890 | 2026-06-10 01:03:04.778041 | TASK [test-dry-run-update-detection : Patch zuul-web deployment template to trigger reconciliation] 2026-06-09 21:03:05.187183 | controller | deployment.apps/zuul-web patched 2026-06-10 01:03:05.328186 | controller | ok: Runtime: 0:00:00.095188 2026-06-10 01:03:05.337835 | 2026-06-10 01:03:05.338067 | TASK [Run deploy with --dry-run to check for update detection] 2026-06-10 01:03:05.375716 | controller | ok 2026-06-10 01:03:05.413089 | 2026-06-10 01:03:05.413245 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 01:03:05.701211 | controller | ok 2026-06-10 01:03:05.710999 | 2026-06-10 01:03:05.711129 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 01:03:06.002925 | controller | changed 2026-06-10 01:03:06.013292 | 2026-06-10 01:03:06.013452 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 21:03:06.613152 | controller | 2026-06-09T21:03:06-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 21:03:06.613245 | controller | 2026-06-09 21:03:06.624862 | controller | 2026-06-09T21:03:06-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 21:03:06.624888 | controller | 2026-06-09 21:03:06.636745 | controller | 2026-06-09T21:03:06-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-09 21:03:07.252276 | controller | 2026-06-09T21:03:07-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 21:03:07.387978 | controller | 2026-06-09T21:03:07-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 21:03:07.654115 | controller | 2026-06-09T21:03:07-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:03:09.437651 | controller | 2026-06-09T21:03:09-04:00 INFO [Dry Run] Statefulset "zuul-web" configuration changed, applying... 2026-06-09 21:03:09.437974 | controller | 2026-06-09T21:03:09-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-09 21:03:09.438006 | controller | 2026-06-09T21:03:09-04:00 INFO Updating object, name: zuul-web (server dry run) 2026-06-09 21:03:10.240399 | controller | 2026-06-09T21:03:10-04:00 INFO 2026-06-09 21:03:10.240463 | controller | - Config: Waiting ... 2026-06-09 21:03:10.240475 | controller | - Gateway: OK 2026-06-09 21:03:10.240486 | controller | - GitServer: OK 2026-06-09 21:03:10.240499 | controller | - HoundSearch: OK 2026-06-09 21:03:10.240512 | controller | - LogJuicer: OK 2026-06-09 21:03:10.240523 | controller | - Logserver: OK 2026-06-09 21:03:10.240535 | controller | - MariaDB: OK 2026-06-09 21:03:10.240543 | controller | - NodePoolBuilder: OK 2026-06-09 21:03:10.240552 | controller | - NodePoolLauncher: OK 2026-06-09 21:03:10.240569 | controller | - Zookeeper: OK 2026-06-09 21:03:10.240577 | controller | - Zuul: Waiting ... 2026-06-09 21:03:10.240585 | controller | - |_ Config Secrets: OK 2026-06-09 21:03:10.240593 | controller | - |_ Merger: OK 2026-06-09 21:03:10.240601 | controller | - |_ Scheduler: OK 2026-06-09 21:03:10.240609 | controller | - |_ Web: Waiting ... 2026-06-09 21:03:10.240617 | controller | 2026-06-09 21:03:10.240626 | controller | 2026-06-09T21:03:10-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 01:03:10.567657 | controller | ok: Runtime: 0:00:03.964173 2026-06-10 01:03:10.577975 | 2026-06-10 01:03:10.578115 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 01:03:11.127996 | controller | ok: Runtime: 0:00:00.007442 2026-06-10 01:03:11.148015 | 2026-06-10 01:03:11.148159 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-executor] 2026-06-10 01:03:11.686560 | controller | skipping: Conditional result was False 2026-06-10 01:03:11.698126 | 2026-06-10 01:03:11.698306 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-web] 2026-06-09 21:03:11.999057 | controller | 2026-06-09T21:03:09-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 01:03:12.248223 | controller | ok: Runtime: 0:00:00.006175 2026-06-10 01:03:12.258681 | 2026-06-10 01:03:12.258867 | TASK [test-dry-run-update-detection : Restore original serial for zuul-executor] 2026-06-10 01:03:12.805140 | controller | skipping: Conditional result was False 2026-06-10 01:03:12.816397 | 2026-06-10 01:03:12.816547 | TASK [test-dry-run-update-detection : Restore original serial for zuul-web] 2026-06-09 21:03:13.176272 | controller | deployment.apps/zuul-web patched 2026-06-10 01:03:13.364388 | controller | ok: Runtime: 0:00:00.068422 2026-06-10 01:03:13.383927 | 2026-06-10 01:03:13.384195 | TASK [health-check/test-dry-run-lifecycle : Generate CR with zuul-executor enabled] 2026-06-10 01:03:14.159851 | controller | changed 2026-06-10 01:03:14.171752 | 2026-06-10 01:03:14.171889 | TASK [Run deploy with --dry-run to test from-scratch creation] 2026-06-10 01:03:14.207505 | controller | ok 2026-06-10 01:03:14.253664 | 2026-06-10 01:03:14.253835 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 01:03:14.521839 | controller | ok 2026-06-10 01:03:14.531556 | 2026-06-10 01:03:14.531721 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 01:03:14.814158 | controller | ok 2026-06-10 01:03:14.823292 | 2026-06-10 01:03:14.823429 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 21:03:15.488712 | controller | 2026-06-09T21:03:15-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 21:03:15.488795 | controller | 2026-06-09 21:03:15.500695 | controller | 2026-06-09T21:03:15-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 21:03:15.500738 | controller | 2026-06-09 21:03:15.513123 | controller | 2026-06-09T21:03:15-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-09 21:03:16.125702 | controller | 2026-06-09T21:03:16-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 21:03:16.265321 | controller | 2026-06-09T21:03:16-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 21:03:16.527446 | controller | 2026-06-09T21:03:16-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:03:18.314810 | controller | 2026-06-09T21:03:18-04:00 INFO Creating object, name: zuul-executor (server dry run) 2026-06-09 21:03:19.919903 | controller | 2026-06-09T21:03:19-04:00 INFO 2026-06-09 21:03:19.919961 | controller | - Config: Waiting ... 2026-06-09 21:03:19.919966 | controller | - Gateway: OK 2026-06-09 21:03:19.919970 | controller | - GitServer: OK 2026-06-09 21:03:19.919974 | controller | - HoundSearch: OK 2026-06-09 21:03:19.919978 | controller | - LogJuicer: OK 2026-06-09 21:03:19.919982 | controller | - Logserver: OK 2026-06-09 21:03:19.919985 | controller | - MariaDB: OK 2026-06-09 21:03:19.919988 | controller | - NodePoolBuilder: OK 2026-06-09 21:03:19.919992 | controller | - NodePoolLauncher: OK 2026-06-09 21:03:19.919995 | controller | - Zookeeper: OK 2026-06-09 21:03:19.919999 | controller | - Zuul: Waiting ... 2026-06-09 21:03:19.920002 | controller | - |_ Config Secrets: OK 2026-06-09 21:03:19.920006 | controller | - |_ Executor: Waiting ... 2026-06-09 21:03:19.920010 | controller | - |_ Merger: OK 2026-06-09 21:03:19.920013 | controller | - |_ Scheduler: OK 2026-06-09 21:03:19.920017 | controller | - |_ Web: OK 2026-06-09 21:03:19.920021 | controller | 2026-06-09 21:03:19.920025 | controller | 2026-06-09T21:03:19-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 01:03:20.496342 | controller | ok: Runtime: 0:00:04.784336 2026-06-10 01:03:20.500655 | 2026-06-10 01:03:20.500717 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 01:03:21.029827 | controller | ok: Runtime: 0:00:00.007521 2026-06-10 01:03:21.047556 | 2026-06-10 01:03:21.047744 | TASK [health-check/test-dry-run-lifecycle : Check for 'create' dry-run messages for zuul-executor resources] 2026-06-09 21:03:21.361348 | controller | 2026-06-09T21:03:18-04:00 INFO Creating object, name: zuul-executor (server dry run) 2026-06-10 01:03:21.603082 | controller | ok: Runtime: 0:00:00.007334 2026-06-10 01:03:21.612736 | 2026-06-10 01:03:21.612871 | TASK [Restore original state by deploying with zuul-executor enabled] 2026-06-10 01:03:21.660284 | controller | ok 2026-06-10 01:03:21.709220 | 2026-06-10 01:03:21.709410 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 01:03:21.997161 | controller | ok 2026-06-10 01:03:22.007378 | 2026-06-10 01:03:22.007516 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 01:03:22.270271 | controller | changed 2026-06-10 01:03:22.280229 | 2026-06-10 01:03:22.280362 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-09 21:03:22.900350 | controller | 2026-06-09T21:03:22-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 21:03:22.900423 | controller | 2026-06-09 21:03:22.911477 | controller | 2026-06-09T21:03:22-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 21:03:22.911506 | controller | 2026-06-09 21:03:22.926143 | controller | 2026-06-09T21:03:22-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:03:22.949319 | controller | 2026-06-09T21:03:22-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:03:22.965303 | controller | 2026-06-09T21:03:22-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:03:23.533330 | controller | 2026-06-09T21:03:23-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:03:23.538007 | controller | 2026-06-09T21:03:23-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 21:03:23.652570 | controller | 2026-06-09T21:03:23-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 21:03:23.660618 | controller | 2026-06-09T21:03:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:03:23.660662 | controller | 2026-06-09T21:03:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:03:23.670199 | controller | 2026-06-09T21:03:23-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:03:23.681387 | controller | 2026-06-09T21:03:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:03:23.681427 | controller | 2026-06-09T21:03:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:03:23.939864 | controller | 2026-06-09T21:03:23-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:03:23.943180 | controller | 2026-06-09T21:03:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:03:24.346852 | controller | 2026-06-09T21:03:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:03:24.346939 | controller | 2026-06-09T21:03:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:03:24.539735 | controller | 2026-06-09T21:03:24-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:03:25.349343 | controller | 2026-06-09T21:03:25-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:03:25.358078 | controller | 2026-06-09T21:03:25-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:03:25.358163 | controller | 2026-06-09T21:03:25-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:03:25.726190 | controller | 2026-06-09T21:03:25-04:00 INFO Creating object, name: zuul-executor 2026-06-09 21:03:26.525838 | controller | 2026-06-09T21:03:26-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:03:26.525910 | controller | 2026-06-09T21:03:26-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 21:03:27.331482 | controller | 2026-06-09T21:03:27-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:03:27.334622 | controller | 2026-06-09T21:03:27-04:00 INFO 2026-06-09 21:03:27.334663 | controller | - Config: Waiting ... 2026-06-09 21:03:27.334674 | controller | - Gateway: OK 2026-06-09 21:03:27.334684 | controller | - GitServer: OK 2026-06-09 21:03:27.334692 | controller | - HoundSearch: OK 2026-06-09 21:03:27.334700 | controller | - LogJuicer: OK 2026-06-09 21:03:27.334708 | controller | - Logserver: OK 2026-06-09 21:03:27.334715 | controller | - MariaDB: OK 2026-06-09 21:03:27.334723 | controller | - NodePoolBuilder: OK 2026-06-09 21:03:27.334731 | controller | - NodePoolLauncher: OK 2026-06-09 21:03:27.334739 | controller | - Zookeeper: OK 2026-06-09 21:03:27.334746 | controller | - Zuul: Waiting ... 2026-06-09 21:03:27.334754 | controller | - |_ Config Secrets: OK 2026-06-09 21:03:27.334762 | controller | - |_ Executor: Waiting ... 2026-06-09 21:03:27.334770 | controller | - |_ Merger: OK 2026-06-09 21:03:27.334777 | controller | - |_ Scheduler: OK 2026-06-09 21:03:27.334786 | controller | - |_ Web: OK 2026-06-09 21:03:27.334794 | controller | 2026-06-09 21:03:27.334803 | controller | 2026-06-09T21:03:27-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 21:03:32.346605 | controller | 2026-06-09T21:03:32-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:03:32.368571 | controller | 2026-06-09T21:03:32-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:03:32.386276 | controller | 2026-06-09T21:03:32-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:03:32.954989 | controller | 2026-06-09T21:03:32-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:03:32.967058 | controller | 2026-06-09T21:03:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:03:32.967147 | controller | 2026-06-09T21:03:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:03:32.978663 | controller | 2026-06-09T21:03:32-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:03:32.991503 | controller | 2026-06-09T21:03:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:03:32.991574 | controller | 2026-06-09T21:03:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:03:33.354665 | controller | 2026-06-09T21:03:33-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:03:33.357605 | controller | 2026-06-09T21:03:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:03:33.758429 | controller | 2026-06-09T21:03:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:03:33.758501 | controller | 2026-06-09T21:03:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:03:33.958512 | controller | 2026-06-09T21:03:33-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:03:34.759279 | controller | 2026-06-09T21:03:34-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:03:34.765685 | controller | 2026-06-09T21:03:34-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:03:34.765725 | controller | 2026-06-09T21:03:34-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:03:34.956906 | controller | 2026-06-09T21:03:34-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:03:34.956996 | controller | 2026-06-09T21:03:34-04:00 INFO statefulset rollout not ok, name: zuul-executor 2026-06-09 21:03:39.967002 | controller | 2026-06-09T21:03:39-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:03:39.985347 | controller | 2026-06-09T21:03:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:03:39.985411 | controller | 2026-06-09T21:03:39-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 21:03:40.006975 | controller | 2026-06-09T21:03:40-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:03:40.030601 | controller | 2026-06-09T21:03:40-04:00 INFO 2026-06-09 21:03:40.030651 | controller | - Config: OK 2026-06-09 21:03:40.030665 | controller | - Gateway: OK 2026-06-09 21:03:40.030677 | controller | - GitServer: OK 2026-06-09 21:03:40.030688 | controller | - HoundSearch: OK 2026-06-09 21:03:40.030699 | controller | - LogJuicer: OK 2026-06-09 21:03:40.030710 | controller | - Logserver: OK 2026-06-09 21:03:40.030722 | controller | - MariaDB: OK 2026-06-09 21:03:40.030733 | controller | - NodePoolBuilder: OK 2026-06-09 21:03:40.030746 | controller | - NodePoolLauncher: OK 2026-06-09 21:03:40.030757 | controller | - Zookeeper: OK 2026-06-09 21:03:40.030768 | controller | - Zuul: OK 2026-06-09 21:03:40.030780 | controller | 2026-06-09 21:03:40.030791 | controller | 2026-06-09T21:03:40-04:00 INFO Updating controller configmap ... 2026-06-09 21:03:40.375302 | controller | 2026-06-09T21:03:40-04:00 INFO Standalone reconcile done. 2026-06-10 01:03:40.852741 | controller | ok: Runtime: 0:00:17.828314 2026-06-10 01:03:40.863323 | 2026-06-10 01:03:40.863455 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 01:03:41.410116 | controller | ok: Runtime: 0:00:00.004492 2026-06-10 01:03:41.426936 | 2026-06-10 01:03:41.427073 | TASK [health-check/test-dry-run-lifecycle : Remove temporary directory] 2026-06-10 01:03:41.719694 | controller | changed 2026-06-10 01:03:41.735745 | 2026-06-10 01:03:41.735882 | TASK [health-check/test-backup-restore : Backup the Software Factory deployment] 2026-06-09 21:03:42.291267 | controller | 2026-06-09T21:03:42-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 21:03:42.291338 | controller | 2026-06-09 21:03:42.301194 | controller | 2026-06-09T21:03:42-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 21:03:42.301211 | controller | 2026-06-09 21:03:42.305306 | controller | 2026-06-09T21:03:42-04:00 INFO Starting backup process for services in namespace: sf 2026-06-09 21:03:42.305344 | controller | 2026-06-09T21:03:42-04:00 INFO Creating secrets backup... 2026-06-09 21:03:42.331482 | controller | 2026-06-09T21:03:42-04:00 INFO Finished doing secret backup! 2026-06-09 21:03:42.331527 | controller | 2026-06-09T21:03:42-04:00 INFO Doing Zuul keys backup... 2026-06-09 21:03:42.334772 | controller | 2026-06-09T21:03:42-04:00 INFO Creating object, name: zuul-kazoo 2026-06-09 21:03:43.343614 | controller | 2026-06-09T21:03:43-04:00 INFO Running pod execution pod: zuul-kazoo, command: [zuul-admin export-keys /tmp/zuul-backup] 2026-06-09 21:03:44.914187 | controller | INFO:kazoo.client:Connecting to zookeeper-0.zookeeper-headless.sf(10.244.0.98):2281, use_ssl: True 2026-06-09 21:03:44.974987 | controller | INFO:kazoo.client:Zookeeper connection established, state: CONNECTED 2026-06-09 21:03:45.023792 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh 2026-06-09 21:03:45.023829 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 2026-06-09 21:03:45.023837 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh 2026-06-09 21:03:45.023934 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 2026-06-09 21:03:45.023963 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-project/demo-project/ssh 2026-06-09 21:03:45.024001 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-09 21:03:45.024009 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/ichiban-config-tC3E2wmd/ichiban-config-tC3E2wmd/ssh 2026-06-09 21:03:45.024015 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/ichiban-config-tC3E2wmd/ichiban-config-tC3E2wmd/secrets 2026-06-09 21:03:45.024020 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/config/config/ssh 2026-06-09 21:03:45.024081 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/config/config/secrets 2026-06-09 21:03:45.024088 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/git-server/system-config/system-config/ssh 2026-06-09 21:03:45.024093 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/git-server/system-config/system-config/secrets 2026-06-09 21:03:45.024213 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/ichiban-tenant-tC3E2wmd/ssh 2026-06-09 21:03:45.024221 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/internal/ssh 2026-06-09 21:03:45.024226 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/demo-tenant/ssh 2026-06-09 21:03:45.512788 | controller | 2026-06-09T21:03:45-04:00 INFO Running pod execution pod: zuul-kazoo, command: [cat /tmp/zuul-backup] 2026-06-09 21:03:45.628193 | controller | 2026-06-09T21:03:45-04:00 INFO Running pod execution pod: zuul-kazoo, command: [rm /tmp/zuul-backup] 2026-06-09 21:03:45.732852 | controller | 2026-06-09T21:03:45-04:00 INFO Finished doing Zuul private keys backup! 2026-06-09 21:03:45.736178 | controller | 2026-06-09T21:03:45-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-09 21:03:45.741327 | controller | 2026-06-09T21:03:45-04:00 INFO Doing DB backup... 2026-06-09 21:03:45.744265 | controller | 2026-06-09T21:03:45-04:00 INFO Running pod execution pod: mariadb-0, command: [mysqldump --databases zuul --single-transaction] 2026-06-09 21:03:45.874459 | controller | mysqldump: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb-dump' instead 2026-06-09 21:03:45.874534 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-09 21:03:46.066178 | controller | 2026-06-09T21:03:46-04:00 INFO Finished doing DBs backup! 2026-06-10 01:03:46.287203 | controller | ok: Runtime: 0:00:04.079908 2026-06-10 01:03:46.300693 | 2026-06-10 01:03:46.300989 | TASK [health-check/test-backup-restore : Fetch the last build ID reported by Zuul web] 2026-06-10 01:03:46.906346 | controller | ok: OK (698 bytes) 2026-06-10 01:03:46.917107 | 2026-06-10 01:03:46.917246 | TASK [health-check/test-backup-restore : Wipe Software Factory deployment] 2026-06-09 21:03:47.558153 | controller | 2026-06-09T21:03:47-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 21:03:47.558291 | controller | 2026-06-09 21:03:47.571399 | controller | 2026-06-09T21:03:47-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 21:03:47.571456 | controller | 2026-06-09 21:03:47.607158 | controller | 2026-06-09T21:03:47-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-09 21:03:47.607215 | controller | 2026-06-09T21:03:47-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-09 21:03:47.775554 | controller | 2026-06-09T21:03:47-04:00 INFO Deleting object, name: nodepool-builder 2026-06-09 21:03:47.783567 | controller | 2026-06-09T21:03:47-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-6cffc7496d-p78l8 launcher 2026-06-09 21:03:47.783630 | controller | 2026-06-09T21:03:47-04:00 INFO Running pod execution pod: nodepool-launcher-6cffc7496d-p78l8, command: [kill -9 1] 2026-06-09 21:03:47.912348 | controller | 2026-06-09T21:03:47-04:00 INFO Deleting object, name: launcher 2026-06-09 21:03:47.914500 | controller | 2026-06-09T21:03:47-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-52vwg mariadb-client 2026-06-09 21:03:47.914549 | controller | 2026-06-09T21:03:47-04:00 INFO Running pod execution pod: zuul-db-provision-52vwg, command: [kill -9 1] 2026-06-09 21:03:47.928680 | controller | 2026-06-09T21:03:47-04:00 INFO Deleting object, name: mariadb-client 2026-06-09 21:03:47.930360 | controller | 2026-06-09T21:03:47-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-gt92h mariadb-client 2026-06-09 21:03:47.930376 | controller | 2026-06-09T21:03:47-04:00 INFO Running pod execution pod: zuul-db-provision-gt92h, command: [kill -9 1] 2026-06-09 21:03:47.944367 | controller | 2026-06-09T21:03:47-04:00 INFO Deleting object, name: mariadb-client 2026-06-09 21:03:47.946154 | controller | 2026-06-09T21:03:47-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-09 21:03:47.946191 | controller | 2026-06-09T21:03:47-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-09 21:03:47.992211 | controller | 2026-06-09T21:03:47-04:00 INFO Deleting object, name: zuul-executor 2026-06-09 21:03:47.995804 | controller | 2026-06-09T21:03:47-04:00 INFO Warning: Killing ZooKeeper client: zuul-kazoo zuul-kazoo 2026-06-09 21:03:47.995817 | controller | 2026-06-09T21:03:47-04:00 INFO Running pod execution pod: zuul-kazoo, command: [kill -9 1] 2026-06-09 21:04:15.945732 | controller | 2026-06-09T21:04:15-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-09 21:04:15.948724 | controller | 2026-06-09T21:04:15-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-09 21:04:15.948768 | controller | 2026-06-09T21:04:15-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-09 21:04:16.082371 | controller | 2026-06-09T21:04:16-04:00 INFO Deleting object, name: zuul-merger 2026-06-09 21:04:16.086860 | controller | 2026-06-09T21:04:16-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-1 zuul-merger 2026-06-09 21:04:16.086920 | controller | 2026-06-09T21:04:16-04:00 INFO Running pod execution pod: zuul-merger-1, command: [kill -9 1] 2026-06-09 21:04:17.418194 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-merger 2026-06-09 21:04:17.420710 | controller | 2026-06-09T21:04:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-09 21:04:17.420750 | controller | 2026-06-09T21:04:17-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-09 21:04:17.530269 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-09 21:04:17.533974 | controller | 2026-06-09T21:04:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-594856d466-47nx8 zuul-web 2026-06-09 21:04:17.534018 | controller | 2026-06-09T21:04:17-04:00 INFO Running pod execution pod: zuul-web-594856d466-47nx8, command: [kill -9 1] 2026-06-09 21:04:17.658946 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-web 2026-06-09 21:04:17.662456 | controller | 2026-06-09T21:04:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-gzfkl zuul-weeder 2026-06-09 21:04:17.662485 | controller | 2026-06-09T21:04:17-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-gzfkl, command: [kill -9 1] 2026-06-09 21:04:17.786490 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-weeder 2026-06-09 21:04:17.789698 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zookeeper 2026-06-09 21:04:17.798836 | controller | 2026-06-09T21:04:17-04:00 INFO Standalone mode detected. Deleting owner ConfigMap with foreground propagation. 2026-06-09 21:04:17.798887 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: sf-standalone-owner 2026-06-09 21:04:17.801703 | controller | 2026-06-09T21:04:17-04:00 INFO Cleaning resources... 2026-06-09 21:04:17.805023 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: gateway 2026-06-09 21:04:17.816937 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: hound-search 2026-06-09 21:04:17.825757 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: logjuicer 2026-06-09 21:04:17.837003 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: nodepool-launcher 2026-06-09 21:04:17.844148 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zookeeper 2026-06-09 21:04:17.861525 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zookeeper-headless 2026-06-09 21:04:17.874166 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-capacity 2026-06-09 21:04:17.882730 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-executor-headless 2026-06-09 21:04:17.886621 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-web 2026-06-09 21:04:17.892296 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-weeder 2026-06-09 21:04:17.899922 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: gateway 2026-06-09 21:04:17.902171 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: logjuicer 2026-06-09 21:04:17.904362 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: nodepool-launcher 2026-06-09 21:04:17.908994 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: git-server 2026-06-09 21:04:17.911248 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: hound-search 2026-06-09 21:04:17.913742 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: logserver 2026-06-09 21:04:17.916944 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: mariadb 2026-06-09 21:04:17.925080 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-ws9mf", "status": "Running"} 2026-06-09 21:04:17.925127 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: gateway-6bbbd758d9-ws9mf 2026-06-09 21:04:17.927984 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "git-server-0", "status": "Running"} 2026-06-09 21:04:17.928033 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: git-server-0 2026-06-09 21:04:17.932469 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "hound-search-0", "status": "Running"} 2026-06-09 21:04:17.932538 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: hound-search-0 2026-06-09 21:04:17.937367 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:17.937425 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:17.943431 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:17.943507 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:17.949408 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "mariadb-0", "status": "Running"} 2026-06-09 21:04:17.949454 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: mariadb-0 2026-06-09 21:04:17.952555 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:17.952580 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:17.958660 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "zookeeper-0", "status": "Running"} 2026-06-09 21:04:17.958706 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zookeeper-0 2026-06-09 21:04:17.960912 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "zuul-merger-0", "status": "Failed"} 2026-06-09 21:04:17.960934 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-merger-0 2026-06-09 21:04:17.964626 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "zuul-merger-1", "status": "Failed"} 2026-06-09 21:04:17.964645 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-merger-1 2026-06-09 21:04:17.966181 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:17.966199 | controller | 2026-06-09T21:04:17-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:18.128054 | controller | 2026-06-09T21:04:18-04:00 INFO Deleting pod {"name": "zuul-web-594856d466-47nx8", "status": "Running"} 2026-06-09 21:04:18.128102 | controller | 2026-06-09T21:04:18-04:00 INFO Deleting object, name: zuul-web-594856d466-47nx8 2026-06-09 21:04:18.333556 | controller | 2026-06-09T21:04:18-04:00 INFO Deleting pod {"name": "zuul-weeder-57c6567f5f-gzfkl", "status": "Running"} 2026-06-09 21:04:18.333622 | controller | 2026-06-09T21:04:18-04:00 INFO Deleting object, name: zuul-weeder-57c6567f5f-gzfkl 2026-06-09 21:04:20.532983 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-qvckf", "status": "Running"} 2026-06-09 21:04:20.533061 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting object, name: gateway-6bbbd758d9-qvckf 2026-06-09 21:04:20.536869 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:20.536917 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:20.540159 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:20.540192 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:20.544046 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:20.544108 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:20.546355 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:20.546417 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:20.550742 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:20.550803 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:20.553897 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:20.553938 | controller | 2026-06-09T21:04:20-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:22.567377 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-qtck8", "status": "Pending"} 2026-06-09 21:04:22.567439 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting object, name: gateway-6bbbd758d9-qtck8 2026-06-09 21:04:22.570377 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:22.570402 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:22.572072 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:22.572103 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:22.574587 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:22.574611 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:22.576706 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:22.576730 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:22.579096 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:22.579135 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:22.581501 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-xtl7k", "status": "Pending"} 2026-06-09 21:04:22.581536 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-xtl7k 2026-06-09 21:04:22.583743 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:22.583767 | controller | 2026-06-09T21:04:22-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:24.590981 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:24.591575 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:24.594433 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:24.594515 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:24.596721 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:24.596753 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:24.600179 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:24.600213 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:24.603351 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:24.603407 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:24.606162 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:24.606219 | controller | 2026-06-09T21:04:24-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:26.619782 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:26.619865 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:26.622852 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:26.622907 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:26.625600 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:26.625637 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:26.628285 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:26.628331 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:26.631461 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:26.631497 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:26.634317 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:26.634379 | controller | 2026-06-09T21:04:26-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:28.643305 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:28.643420 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:28.646551 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:28.646661 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:28.650209 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:28.650309 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:28.652907 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:28.652993 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:28.656049 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:28.656070 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:28.659256 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:28.659274 | controller | 2026-06-09T21:04:28-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:30.668961 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:30.669061 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:30.672306 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:30.672351 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:30.675344 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:30.675387 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:30.677994 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:30.678038 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:30.681524 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:30.681554 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:30.684598 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:30.684633 | controller | 2026-06-09T21:04:30-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:32.696749 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:32.696819 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:32.699895 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:32.699915 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:32.702704 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:32.702756 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:32.705261 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:32.705346 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:32.709965 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:32.710035 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:32.714608 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:32.714666 | controller | 2026-06-09T21:04:32-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:34.722379 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:34.722443 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:34.724735 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:34.724799 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:34.726927 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:34.726947 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:34.729289 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:34.729308 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:34.732515 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:34.732632 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:34.735172 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:34.735267 | controller | 2026-06-09T21:04:34-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:36.743703 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:36.743812 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:36.746518 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:36.746562 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:36.748682 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:36.748774 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:36.750736 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:36.750797 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:36.753486 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:36.753548 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:36.755605 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:36.755647 | controller | 2026-06-09T21:04:36-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:38.769327 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:38.769404 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:38.772069 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:38.772135 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:38.774717 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:38.774780 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:38.777453 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-8msg8", "status": "Pending"} 2026-06-09 21:04:38.777524 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-8msg8 2026-06-09 21:04:38.780278 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:38.780355 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:38.783658 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:38.783699 | controller | 2026-06-09T21:04:38-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:40.795563 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:40.795742 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:40.798994 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:40.799039 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:40.802037 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:40.802082 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:40.804967 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:40.805006 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:40.808174 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:40.808213 | controller | 2026-06-09T21:04:40-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:42.820115 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:42.820206 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:42.823134 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:42.823179 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:42.826319 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:42.826357 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:42.829139 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:42.829176 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:42.831720 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:42.831756 | controller | 2026-06-09T21:04:42-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:44.842832 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:44.842943 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:44.845364 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:44.845430 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:44.847662 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:44.847733 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:44.850031 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:44.850056 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:44.852090 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:44.852115 | controller | 2026-06-09T21:04:44-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:46.861391 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:46.861495 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:46.864087 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-mv8gq", "status": "Running"} 2026-06-09 21:04:46.864126 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-mv8gq 2026-06-09 21:04:46.866812 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-09 21:04:46.866861 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting object, name: logserver-0 2026-06-09 21:04:46.870263 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting pod {"name": "nodepool-launcher-6cffc7496d-p78l8", "status": "Running"} 2026-06-09 21:04:46.870311 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting object, name: nodepool-launcher-6cffc7496d-p78l8 2026-06-09 21:04:46.874610 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-09 21:04:46.874701 | controller | 2026-06-09T21:04:46-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-09 21:04:48.882815 | controller | 2026-06-09T21:04:48-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:48.882882 | controller | 2026-06-09T21:04:48-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:50.890641 | controller | 2026-06-09T21:04:50-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Running"} 2026-06-09 21:04:50.890718 | controller | 2026-06-09T21:04:50-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:52.899667 | controller | 2026-06-09T21:04:52-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-5dp49", "status": "Failed"} 2026-06-09 21:04:52.899806 | controller | 2026-06-09T21:04:52-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-5dp49 2026-06-09 21:04:54.908562 | controller | 2026-06-09T21:04:54-04:00 INFO Removing dangling persistent volume claims if any... 2026-06-09 21:04:54.912808 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "git-server-git-server-0", "status": "Bound"} 2026-06-09 21:04:54.912935 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: git-server-git-server-0 2026-06-09 21:04:54.917951 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "hound-search-data-hound-search-0", "status": "Bound"} 2026-06-09 21:04:54.918000 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: hound-search-data-hound-search-0 2026-06-09 21:04:54.922118 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "logserver-logserver-0", "status": "Bound"} 2026-06-09 21:04:54.922158 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: logserver-logserver-0 2026-06-09 21:04:54.925562 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "mariadb-logs-mariadb-0", "status": "Bound"} 2026-06-09 21:04:54.925608 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: mariadb-logs-mariadb-0 2026-06-09 21:04:54.929104 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "mariadb-mariadb-0", "status": "Bound"} 2026-06-09 21:04:54.929130 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: mariadb-mariadb-0 2026-06-09 21:04:54.932249 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "nodepool-builder-nodepool-builder-0", "status": "Bound"} 2026-06-09 21:04:54.932288 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: nodepool-builder-nodepool-builder-0 2026-06-09 21:04:54.935400 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "zookeeper-data-zookeeper-0", "status": "Bound"} 2026-06-09 21:04:54.935426 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: zookeeper-data-zookeeper-0 2026-06-09 21:04:54.938863 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "zookeeper-logs-zookeeper-0", "status": "Bound"} 2026-06-09 21:04:54.938884 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: zookeeper-logs-zookeeper-0 2026-06-09 21:04:54.942260 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "zuul-executor-zuul-executor-0", "status": "Bound"} 2026-06-09 21:04:54.942282 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: zuul-executor-zuul-executor-0 2026-06-09 21:04:54.945897 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "zuul-executor-zuul-executor-1", "status": "Bound"} 2026-06-09 21:04:54.945918 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: zuul-executor-zuul-executor-1 2026-06-09 21:04:54.948808 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting pvc {"name": "zuul-merger-zuul-merger-0", "status": "Bound"} 2026-06-09 21:04:54.948832 | controller | 2026-06-09T21:04:54-04:00 INFO Deleting object, name: zuul-merger-zuul-merger-0 2026-06-09 21:04:55.118912 | controller | 2026-06-09T21:04:55-04:00 INFO Deleting pvc {"name": "zuul-merger-zuul-merger-1", "status": "Bound"} 2026-06-09 21:04:55.119000 | controller | 2026-06-09T21:04:55-04:00 INFO Deleting object, name: zuul-merger-zuul-merger-1 2026-06-09 21:04:55.316386 | controller | 2026-06-09T21:04:55-04:00 INFO Deleting pvc {"name": "zuul-scheduler-zuul-scheduler-0", "status": "Bound"} 2026-06-09 21:04:55.316447 | controller | 2026-06-09T21:04:55-04:00 INFO Deleting object, name: zuul-scheduler-zuul-scheduler-0 2026-06-10 01:04:57.580228 | controller | ok: Runtime: 0:01:10.329582 2026-06-10 01:04:57.595955 | 2026-06-10 01:04:57.596089 | TASK [health-check/test-backup-restore : Restore backup of the Software Factory previous deployment] 2026-06-09 21:04:58.180277 | controller | 2026-06-09T21:04:58-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-09 21:04:58.180363 | controller | 2026-06-09 21:04:58.194258 | controller | 2026-06-09T21:04:58-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-09 21:04:58.194322 | controller | 2026-06-09 21:04:58.204111 | controller | 2026-06-09T21:04:58-04:00 INFO Restoring secrets... 2026-06-09 21:04:58.205250 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-09 21:04:58.208968 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-09 21:04:58.211777 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: nodepool-builder-ssh-key 2026-06-09 21:04:58.214285 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: zuul-ssh-key 2026-06-09 21:04:58.217959 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: zuul-spare-ssh-key 2026-06-09 21:04:58.220321 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: zuul-keystore-password 2026-06-09 21:04:58.222153 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-09 21:04:58.224808 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: logserver-keys 2026-06-09 21:04:58.227731 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: logserver-uploader-keys 2026-06-09 21:04:58.230722 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: logserver-uploader-spare-keys 2026-06-09 21:04:58.233005 | controller | 2026-06-09T21:04:58-04:00 INFO Spawning backend services... 2026-06-09 21:04:58.409768 | controller | 2026-06-09T21:04:58-04:00 INFO Creating secret, name: mariadb-root-password 2026-06-09 21:04:58.409852 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: mariadb-root-password 2026-06-09 21:04:58.808475 | controller | 2026-06-09T21:04:58-04:00 INFO Creating secret, name: mariadb-config-secrets 2026-06-09 21:04:58.808583 | controller | 2026-06-09T21:04:58-04:00 INFO Creating object, name: mariadb-config-secrets 2026-06-09 21:04:59.208979 | controller | 2026-06-09T21:04:59-04:00 INFO Creating secret, name: mariadb-initdb-secrets 2026-06-09 21:04:59.209057 | controller | 2026-06-09T21:04:59-04:00 INFO Creating object, name: mariadb-initdb-secrets 2026-06-09 21:04:59.414973 | controller | 2026-06-09T21:04:59-04:00 INFO Creating object, name: mariadb 2026-06-09 21:04:59.810504 | controller | 2026-06-09T21:04:59-04:00 INFO Creating config map name: zookeeper-pi-config-map 2026-06-09 21:04:59.810592 | controller | 2026-06-09T21:04:59-04:00 INFO Creating object, name: zookeeper-pi-config-map 2026-06-09 21:04:59.814094 | controller | 2026-06-09T21:04:59-04:00 INFO Creating service, name: zookeeper-headless 2026-06-09 21:04:59.814150 | controller | 2026-06-09T21:04:59-04:00 INFO Creating object, name: zookeeper-headless 2026-06-09 21:04:59.817630 | controller | 2026-06-09T21:04:59-04:00 INFO Creating service, name: zookeeper 2026-06-09 21:04:59.817698 | controller | 2026-06-09T21:04:59-04:00 INFO Creating object, name: zookeeper 2026-06-09 21:04:59.824469 | controller | 2026-06-09T21:04:59-04:00 INFO Creating object, name: zookeeper 2026-06-09 21:04:59.827438 | controller | 2026-06-09T21:04:59-04:00 INFO Waiting for backend services... 2026-06-09 21:05:00.415894 | controller | 2026-06-09T21:05:00-04:00 INFO Creating service, name: mariadb 2026-06-09 21:05:00.415982 | controller | 2026-06-09T21:05:00-04:00 INFO Creating object, name: mariadb 2026-06-09 21:05:00.427857 | controller | 2026-06-09T21:05:00-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:00.427945 | controller | 2026-06-09T21:05:00-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 21:05:05.434803 | controller | 2026-06-09T21:05:05-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:05.435189 | controller | 2026-06-09T21:05:05-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 21:05:10.443837 | controller | 2026-06-09T21:05:10-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:10.443901 | controller | 2026-06-09T21:05:10-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 21:05:15.449505 | controller | 2026-06-09T21:05:15-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:15.449596 | controller | 2026-06-09T21:05:15-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 21:05:20.457587 | controller | 2026-06-09T21:05:20-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:20.457697 | controller | 2026-06-09T21:05:20-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 21:05:25.468933 | controller | 2026-06-09T21:05:25-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:25.469023 | controller | 2026-06-09T21:05:25-04:00 INFO statefulset rollout not ok, name: mariadb 2026-06-09 21:05:30.476672 | controller | 2026-06-09T21:05:30-04:00 INFO Statefulset "mariadb" not ready after 30s 2026-06-09 21:05:31.494842 | controller | 2026-06-09T21:05:31-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:31.500593 | controller | 2026-06-09T21:05:31-04:00 INFO Creating object, name: zuul-db-connection 2026-06-09 21:05:31.507734 | controller | 2026-06-09T21:05:31-04:00 INFO Creating object, name: zuul-db-provision 2026-06-09 21:05:32.528976 | controller | 2026-06-09T21:05:32-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:33.554768 | controller | 2026-06-09T21:05:33-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:34.580482 | controller | 2026-06-09T21:05:34-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:35.606508 | controller | 2026-06-09T21:05:35-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:36.627072 | controller | 2026-06-09T21:05:36-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:36.644357 | controller | 2026-06-09T21:05:36-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:05:37.217083 | controller | 2026-06-09T21:05:37-04:00 INFO Creating secret, name: zuul-config 2026-06-09 21:05:37.217157 | controller | 2026-06-09T21:05:37-04:00 INFO Creating object, name: zuul-config 2026-06-09 21:05:37.420401 | controller | 2026-06-09T21:05:37-04:00 INFO Creating config map name: zuul-scheduler-tooling-config-map 2026-06-09 21:05:37.420477 | controller | 2026-06-09T21:05:37-04:00 INFO Creating object, name: zuul-scheduler-tooling-config-map 2026-06-09 21:05:37.426170 | controller | 2026-06-09T21:05:37-04:00 INFO Creating object, name: zuul-kazoo 2026-06-09 21:05:39.438265 | controller | 2026-06-09T21:05:39-04:00 INFO Restoring Zuul... 2026-06-09 21:05:39.438342 | controller | 2026-06-09T21:05:39-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c rm -rf /tmp/zuul-import && mkdir -p /tmp/zuul-import] 2026-06-09 21:05:39.584730 | controller | 2026-06-09T21:05:39-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /tmp/zuul.keys && zuul-admin import-keys --force /tmp/zuul.keys] 2026-06-09 21:05:41.168820 | controller | INFO:kazoo.client:Connecting to zookeeper-0.zookeeper-headless.sf(10.244.0.116):2281, use_ssl: True 2026-06-09 21:05:41.375492 | controller | INFO:kazoo.client:Zookeeper connection established, state: CONNECTED 2026-06-09 21:05:41.878346 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh 2026-06-09 21:05:41.962756 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 2026-06-09 21:05:42.078659 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh 2026-06-09 21:05:42.084845 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 2026-06-09 21:05:42.271783 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-project/demo-project/ssh 2026-06-09 21:05:42.361305 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-09 21:05:42.472581 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/ichiban-config-tC3E2wmd/ichiban-config-tC3E2wmd/ssh 2026-06-09 21:05:42.561908 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/ichiban-config-tC3E2wmd/ichiban-config-tC3E2wmd/secrets 2026-06-09 21:05:42.674042 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/config/config/ssh 2026-06-09 21:05:42.678613 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/config/config/secrets 2026-06-09 21:05:42.781717 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/git-server/system-config/system-config/ssh 2026-06-09 21:05:42.863075 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/git-server/system-config/system-config/secrets 2026-06-09 21:05:42.890433 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/ichiban-tenant-tC3E2wmd/ssh 2026-06-09 21:05:42.967014 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/internal/ssh 2026-06-09 21:05:42.981744 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/demo-tenant/ssh 2026-06-09 21:05:43.554214 | controller | 2026-06-09T21:05:43-04:00 INFO Finished doing Zuul private keys restore! 2026-06-09 21:05:43.554311 | controller | 2026-06-09T21:05:43-04:00 INFO Restoring DB... 2026-06-09 21:05:43.558159 | controller | 2026-06-09T21:05:43-04:00 INFO Running pod execution pod: mariadb-0, command: [mysql -e DROP DATABASE zuul;] 2026-06-09 21:05:43.682312 | controller | mysql: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb' instead 2026-06-09 21:05:43.682385 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-09 21:05:43.797195 | controller | 2026-06-09T21:05:43-04:00 INFO Running pod execution pod: mariadb-0, command: [mysql -h0] 2026-06-09 21:05:43.971310 | controller | mysql: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb' instead 2026-06-09 21:05:43.971392 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-09 21:05:44.331298 | controller | 2026-06-09T21:05:44-04:00 INFO Finished restoring DB from backup! 2026-06-09 21:05:44.335031 | controller | 2026-06-09T21:05:44-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-09 21:05:44.357680 | controller | 2026-06-09T21:05:44-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:05:44.373667 | controller | 2026-06-09T21:05:44-04:00 INFO Creating config map name: git-server-pi-config-map 2026-06-09 21:05:44.373729 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: git-server-pi-config-map 2026-06-09 21:05:44.381567 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: git-server 2026-06-09 21:05:44.404329 | controller | 2026-06-09T21:05:44-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:05:44.756363 | controller | 2026-06-09T21:05:44-04:00 INFO Creating config map name: logserver-config-map 2026-06-09 21:05:44.756456 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: logserver-config-map 2026-06-09 21:05:44.760336 | controller | 2026-06-09T21:05:44-04:00 INFO Creating service, name: logserver 2026-06-09 21:05:44.760373 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: logserver 2026-06-09 21:05:44.957390 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: logserver 2026-06-09 21:05:44.965352 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: gateway 2026-06-09 21:05:44.975359 | controller | 2026-06-09T21:05:44-04:00 INFO Creating config map name: gateway-config-map 2026-06-09 21:05:44.975419 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: gateway-config-map 2026-06-09 21:05:44.981335 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: gateway 2026-06-09 21:05:44.988763 | controller | 2026-06-09T21:05:44-04:00 INFO Creating service, name: hound-search 2026-06-09 21:05:44.988803 | controller | 2026-06-09T21:05:44-04:00 INFO Creating object, name: hound-search 2026-06-09 21:05:45.007824 | controller | 2026-06-09T21:05:45-04:00 INFO Creating object, name: hound-search 2026-06-09 21:05:45.019390 | controller | 2026-06-09T21:05:45-04:00 INFO Creating object, name: logjuicer-pvc 2026-06-09 21:05:45.024361 | controller | 2026-06-09T21:05:45-04:00 INFO Creating object, name: logjuicer 2026-06-09 21:05:45.033082 | controller | 2026-06-09T21:05:45-04:00 INFO Creating object, name: logjuicer 2026-06-09 21:05:45.374466 | controller | 2026-06-09T21:05:45-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:05:45.378367 | controller | 2026-06-09T21:05:45-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-09 21:05:45.378413 | controller | 2026-06-09T21:05:45-04:00 INFO 2026-06-09 21:05:45.378425 | controller | - Config: Waiting ... 2026-06-09 21:05:45.378433 | controller | - Gateway: Waiting ... 2026-06-09 21:05:45.378442 | controller | - GitServer: Waiting ... 2026-06-09 21:05:45.378450 | controller | - HoundSearch: Waiting ... 2026-06-09 21:05:45.378458 | controller | - LogJuicer: Waiting ... 2026-06-09 21:05:45.378476 | controller | - Logserver: Waiting ... 2026-06-09 21:05:45.378485 | controller | - MariaDB: OK 2026-06-09 21:05:45.378494 | controller | - Zookeeper: OK 2026-06-09 21:05:45.378502 | controller | 2026-06-09 21:05:45.378514 | controller | 2026-06-09T21:05:45-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-09 21:05:50.384925 | controller | 2026-06-09T21:05:50-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:05:50.408959 | controller | 2026-06-09T21:05:50-04:00 INFO Creating service, name: git-server 2026-06-09 21:05:50.409026 | controller | 2026-06-09T21:05:50-04:00 INFO Creating object, name: git-server 2026-06-09 21:05:50.419002 | controller | 2026-06-09T21:05:50-04:00 INFO Creating service, name: git-server-rw 2026-06-09 21:05:50.419084 | controller | 2026-06-09T21:05:50-04:00 INFO Creating object, name: git-server-rw 2026-06-09 21:05:50.428442 | controller | 2026-06-09T21:05:50-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:05:50.428511 | controller | 2026-06-09T21:05:50-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 21:05:55.438561 | controller | 2026-06-09T21:05:55-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:05:55.438651 | controller | 2026-06-09T21:05:55-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 21:06:00.447540 | controller | 2026-06-09T21:06:00-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:06:00.447633 | controller | 2026-06-09T21:06:00-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 21:06:05.459362 | controller | 2026-06-09T21:06:05-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:06:05.459441 | controller | 2026-06-09T21:06:05-04:00 INFO statefulset rollout not ok, name: git-server 2026-06-09 21:06:10.472520 | controller | 2026-06-09T21:06:10-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:06:10.492795 | controller | 2026-06-09T21:06:10-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:06:10.520019 | controller | 2026-06-09T21:06:10-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:06:10.525541 | controller | 2026-06-09T21:06:10-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-09 21:06:10.636839 | controller | 2026-06-09T21:06:10-04:00 INFO Logserver authorized_keys is up to date 2026-06-09 21:06:10.646620 | controller | 2026-06-09T21:06:10-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:06:10.646664 | controller | 2026-06-09T21:06:10-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:06:10.656010 | controller | 2026-06-09T21:06:10-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:06:10.668303 | controller | 2026-06-09T21:06:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:06:10.668345 | controller | 2026-06-09T21:06:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:06:10.687106 | controller | 2026-06-09T21:06:10-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:06:10.689680 | controller | 2026-06-09T21:06:10-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:06:10.882568 | controller | 2026-06-09T21:06:10-04:00 INFO Creating config map name: np-statsd-config-map 2026-06-09 21:06:10.882655 | controller | 2026-06-09T21:06:10-04:00 INFO Creating object, name: np-statsd-config-map 2026-06-09 21:06:10.887730 | controller | 2026-06-09T21:06:10-04:00 INFO Creating config map name: nodepool-tooling-config-map 2026-06-09 21:06:10.887808 | controller | 2026-06-09T21:06:10-04:00 INFO Creating object, name: nodepool-tooling-config-map 2026-06-09 21:06:10.892347 | controller | 2026-06-09T21:06:10-04:00 INFO Creating config map name: nodepool-launcher-extra-config-config-map 2026-06-09 21:06:10.892391 | controller | 2026-06-09T21:06:10-04:00 INFO Creating object, name: nodepool-launcher-extra-config-config-map 2026-06-09 21:06:10.912722 | controller | 2026-06-09T21:06:10-04:00 INFO Creating object, name: zuul-capacity 2026-06-09 21:06:10.933488 | controller | 2026-06-09T21:06:10-04:00 INFO Creating object, name: nodepool-launcher 2026-06-09 21:06:11.311107 | controller | 2026-06-09T21:06:11-04:00 INFO Creating config map name: nodepool-builder-extra-config-config-map 2026-06-09 21:06:11.311218 | controller | 2026-06-09T21:06:11-04:00 INFO Creating object, name: nodepool-builder-extra-config-config-map 2026-06-09 21:06:11.721309 | controller | 2026-06-09T21:06:11-04:00 INFO Creating service, name: nodepool-builder 2026-06-09 21:06:11.721455 | controller | 2026-06-09T21:06:11-04:00 INFO Creating object, name: nodepool-builder 2026-06-09 21:06:11.731548 | controller | 2026-06-09T21:06:11-04:00 INFO Creating object, name: nodepool-builder 2026-06-09 21:06:11.882753 | controller | 2026-06-09T21:06:11-04:00 INFO Creating object, name: zuul-web 2026-06-09 21:06:11.893490 | controller | 2026-06-09T21:06:11-04:00 INFO Creating object, name: zuul-executor-headless 2026-06-09 21:06:12.110718 | controller | 2026-06-09T21:06:12-04:00 INFO Creating config map name: zuul-extra-config-map 2026-06-09 21:06:12.110783 | controller | 2026-06-09T21:06:12-04:00 INFO Creating object, name: zuul-extra-config-map 2026-06-09 21:06:12.511993 | controller | 2026-06-09T21:06:12-04:00 INFO Creating config map name: zuul-statsd-config-map 2026-06-09 21:06:12.512141 | controller | 2026-06-09T21:06:12-04:00 INFO Creating object, name: zuul-statsd-config-map 2026-06-09 21:06:12.911643 | controller | 2026-06-09T21:06:12-04:00 INFO Creating config map name: zuul-logging-config-map 2026-06-09 21:06:12.911776 | controller | 2026-06-09T21:06:12-04:00 INFO Creating object, name: zuul-logging-config-map 2026-06-09 21:06:13.114028 | controller | 2026-06-09T21:06:13-04:00 INFO Creating object, name: zuul-scheduler 2026-06-09 21:06:13.912670 | controller | 2026-06-09T21:06:13-04:00 INFO Creating object, name: zuul-executor 2026-06-09 21:06:14.717298 | controller | 2026-06-09T21:06:14-04:00 INFO Creating object, name: zuul-web 2026-06-09 21:06:15.514178 | controller | 2026-06-09T21:06:15-04:00 INFO Creating object, name: zuul-merger 2026-06-09 21:06:15.520332 | controller | 2026-06-09T21:06:15-04:00 INFO 2026-06-09 21:06:15.520395 | controller | - Config: Waiting ... 2026-06-09 21:06:15.520409 | controller | - Gateway: OK 2026-06-09 21:06:15.520421 | controller | - GitServer: OK 2026-06-09 21:06:15.520431 | controller | - HoundSearch: OK 2026-06-09 21:06:15.520441 | controller | - LogJuicer: OK 2026-06-09 21:06:15.520452 | controller | - Logserver: OK 2026-06-09 21:06:15.520462 | controller | - MariaDB: OK 2026-06-09 21:06:15.520474 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 21:06:15.520485 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 21:06:15.520495 | controller | - Zookeeper: OK 2026-06-09 21:06:15.520506 | controller | - Zuul: Waiting ... 2026-06-09 21:06:15.520517 | controller | - |_ Config Secrets: OK 2026-06-09 21:06:15.520528 | controller | - |_ Executor: Waiting ... 2026-06-09 21:06:15.520540 | controller | - |_ Merger: Waiting ... 2026-06-09 21:06:15.520551 | controller | - |_ Scheduler: Waiting ... 2026-06-09 21:06:15.520561 | controller | - |_ Web: Waiting ... 2026-06-09 21:06:15.520573 | controller | 2026-06-09 21:06:15.520584 | controller | 2026-06-09T21:06:15-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-09 21:06:20.529613 | controller | 2026-06-09T21:06:20-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:06:20.546951 | controller | 2026-06-09T21:06:20-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:06:20.557660 | controller | 2026-06-09T21:06:20-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:06:21.136254 | controller | 2026-06-09T21:06:21-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:06:21.143542 | controller | 2026-06-09T21:06:21-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:06:21.143581 | controller | 2026-06-09T21:06:21-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:06:21.149308 | controller | 2026-06-09T21:06:21-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:06:21.157247 | controller | 2026-06-09T21:06:21-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:06:21.157277 | controller | 2026-06-09T21:06:21-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:06:21.536978 | controller | 2026-06-09T21:06:21-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:06:21.538852 | controller | 2026-06-09T21:06:21-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:06:21.941875 | controller | 2026-06-09T21:06:21-04:00 INFO Creating object, name: nodepool-launcher 2026-06-09 21:06:21.947221 | controller | 2026-06-09T21:06:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:06:21.947286 | controller | 2026-06-09T21:06:21-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-09 21:06:22.137662 | controller | 2026-06-09T21:06:22-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:06:22.137737 | controller | 2026-06-09T21:06:22-04:00 INFO statefulset rollout not ok, name: nodepool-builder 2026-06-09 21:06:22.956046 | controller | 2026-06-09T21:06:22-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:06:22.956218 | controller | 2026-06-09T21:06:22-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:06:27.967657 | controller | 2026-06-09T21:06:27-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:06:27.967765 | controller | 2026-06-09T21:06:27-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:06:32.980011 | controller | 2026-06-09T21:06:32-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:06:32.980082 | controller | 2026-06-09T21:06:32-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:06:37.990311 | controller | 2026-06-09T21:06:37-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:06:37.990382 | controller | 2026-06-09T21:06:37-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:06:42.999635 | controller | 2026-06-09T21:06:42-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:06:42.999715 | controller | 2026-06-09T21:06:42-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:06:48.011147 | controller | 2026-06-09T21:06:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:06:48.011303 | controller | 2026-06-09T21:06:48-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:06:53.016685 | controller | 2026-06-09T21:06:53-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-09 21:06:53.036526 | controller | 2026-06-09T21:06:53-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:06:53.052930 | controller | 2026-06-09T21:06:53-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:06:53.053026 | controller | 2026-06-09T21:06:53-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:06:53.428153 | controller | 2026-06-09T21:06:53-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:06:53.430722 | controller | 2026-06-09T21:06:53-04:00 INFO 2026-06-09 21:06:53.430747 | controller | - Config: Waiting ... 2026-06-09 21:06:53.430754 | controller | - Gateway: OK 2026-06-09 21:06:53.430759 | controller | - GitServer: OK 2026-06-09 21:06:53.430764 | controller | - HoundSearch: OK 2026-06-09 21:06:53.430769 | controller | - LogJuicer: OK 2026-06-09 21:06:53.430774 | controller | - Logserver: OK 2026-06-09 21:06:53.430779 | controller | - MariaDB: OK 2026-06-09 21:06:53.430796 | controller | - NodePoolBuilder: Waiting ... 2026-06-09 21:06:53.430801 | controller | - NodePoolLauncher: Waiting ... 2026-06-09 21:06:53.430806 | controller | - Zookeeper: OK 2026-06-09 21:06:53.430811 | controller | - Zuul: Waiting ... 2026-06-09 21:06:53.430816 | controller | - |_ Config Secrets: OK 2026-06-09 21:06:53.430821 | controller | - |_ Executor: OK 2026-06-09 21:06:53.430826 | controller | - |_ Merger: OK 2026-06-09 21:06:53.430831 | controller | - |_ Scheduler: Waiting ... 2026-06-09 21:06:53.430836 | controller | - |_ Web: Waiting ... 2026-06-09 21:06:53.430841 | controller | 2026-06-09 21:06:53.430871 | controller | 2026-06-09T21:06:53-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-09 21:06:58.440435 | controller | 2026-06-09T21:06:58-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:06:58.467000 | controller | 2026-06-09T21:06:58-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:06:58.485598 | controller | 2026-06-09T21:06:58-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:06:59.048947 | controller | 2026-06-09T21:06:59-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:06:59.059772 | controller | 2026-06-09T21:06:59-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:06:59.059832 | controller | 2026-06-09T21:06:59-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:06:59.069327 | controller | 2026-06-09T21:06:59-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:06:59.082780 | controller | 2026-06-09T21:06:59-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:06:59.082955 | controller | 2026-06-09T21:06:59-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:06:59.454169 | controller | 2026-06-09T21:06:59-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:06:59.456996 | controller | 2026-06-09T21:06:59-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:06:59.853103 | controller | 2026-06-09T21:06:59-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:06:59.853176 | controller | 2026-06-09T21:06:59-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:07:00.050493 | controller | 2026-06-09T21:07:00-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:07:00.863174 | controller | 2026-06-09T21:07:00-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:07:00.863317 | controller | 2026-06-09T21:07:00-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:07:05.871362 | controller | 2026-06-09T21:07:05-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:07:05.871436 | controller | 2026-06-09T21:07:05-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:07:10.881583 | controller | 2026-06-09T21:07:10-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:07:10.881660 | controller | 2026-06-09T21:07:10-04:00 INFO statefulset rollout not ok, name: zuul-scheduler 2026-06-09 21:07:15.891189 | controller | 2026-06-09T21:07:15-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:07:15.896412 | controller | 2026-06-09T21:07:15-04:00 INFO Creating object, name: zuul-weeder 2026-06-09 21:07:15.907088 | controller | 2026-06-09T21:07:15-04:00 INFO Creating object, name: zuul-weeder 2026-06-09 21:07:15.934030 | controller | 2026-06-09T21:07:15-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:07:15.961026 | controller | 2026-06-09T21:07:15-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:07:15.961116 | controller | 2026-06-09T21:07:15-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 21:07:16.324863 | controller | 2026-06-09T21:07:16-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:07:16.327895 | controller | 2026-06-09T21:07:16-04:00 INFO 2026-06-09 21:07:16.327936 | controller | - Config: Waiting ... 2026-06-09 21:07:16.327947 | controller | - Gateway: OK 2026-06-09 21:07:16.327956 | controller | - GitServer: OK 2026-06-09 21:07:16.327964 | controller | - HoundSearch: OK 2026-06-09 21:07:16.327972 | controller | - LogJuicer: OK 2026-06-09 21:07:16.327980 | controller | - Logserver: OK 2026-06-09 21:07:16.327987 | controller | - MariaDB: OK 2026-06-09 21:07:16.327995 | controller | - NodePoolBuilder: OK 2026-06-09 21:07:16.328003 | controller | - NodePoolLauncher: OK 2026-06-09 21:07:16.328011 | controller | - Zookeeper: OK 2026-06-09 21:07:16.328019 | controller | - Zuul: Waiting ... 2026-06-09 21:07:16.328026 | controller | - |_ Config Secrets: OK 2026-06-09 21:07:16.328034 | controller | - |_ Executor: OK 2026-06-09 21:07:16.328042 | controller | - |_ Merger: OK 2026-06-09 21:07:16.328050 | controller | - |_ Scheduler: Waiting ... 2026-06-09 21:07:16.328059 | controller | - |_ Web: OK 2026-06-09 21:07:16.328067 | controller | 2026-06-09 21:07:16.328111 | controller | 2026-06-09T21:07:16-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-09 21:07:21.337417 | controller | 2026-06-09T21:07:21-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:07:21.369570 | controller | 2026-06-09T21:07:21-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:07:21.388517 | controller | 2026-06-09T21:07:21-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:07:21.947793 | controller | 2026-06-09T21:07:21-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:07:21.958735 | controller | 2026-06-09T21:07:21-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:07:21.958841 | controller | 2026-06-09T21:07:21-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:07:21.968152 | controller | 2026-06-09T21:07:21-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:07:21.980406 | controller | 2026-06-09T21:07:21-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:07:21.980501 | controller | 2026-06-09T21:07:21-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:07:22.348792 | controller | 2026-06-09T21:07:22-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:07:22.351470 | controller | 2026-06-09T21:07:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:07:22.758465 | controller | 2026-06-09T21:07:22-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:07:22.758569 | controller | 2026-06-09T21:07:22-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:07:22.950142 | controller | 2026-06-09T21:07:22-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:07:23.760944 | controller | 2026-06-09T21:07:23-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:07:23.770877 | controller | 2026-06-09T21:07:23-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:07:23.770921 | controller | 2026-06-09T21:07:23-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:07:23.950331 | controller | 2026-06-09T21:07:23-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:07:24.742052 | controller | 2026-06-09T21:07:24-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:07:24.742143 | controller | 2026-06-09T21:07:24-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 21:07:25.547468 | controller | 2026-06-09T21:07:25-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:07:25.738285 | controller | 2026-06-09T21:07:25-04:00 INFO Creating config map name: pymod-sf-operator-config-map 2026-06-09 21:07:25.738422 | controller | 2026-06-09T21:07:25-04:00 INFO Creating object, name: pymod-sf-operator-config-map 2026-06-09 21:07:26.141196 | controller | 2026-06-09T21:07:26-04:00 INFO Creating object, name: config-updater 2026-06-09 21:07:26.148935 | controller | 2026-06-09T21:07:26-04:00 INFO Creating object, name: config-updater-role 2026-06-09 21:07:26.154714 | controller | 2026-06-09T21:07:26-04:00 INFO Creating object, name: config-updater 2026-06-09 21:07:26.160274 | controller | 2026-06-09T21:07:26-04:00 INFO Creating the config-update service account secret 2026-06-09 21:07:26.160356 | controller | 2026-06-09T21:07:26-04:00 INFO Creating object, name: config-update-secrets 2026-06-09 21:07:26.165483 | controller | 2026-06-09T21:07:26-04:00 INFO Creating base secret job 2026-06-09 21:07:26.166095 | controller | 2026-06-09T21:07:26-04:00 INFO Creating object, name: config-base-secret-1-8715-ns 2026-06-09 21:07:26.170918 | controller | 2026-06-09T21:07:26-04:00 INFO 2026-06-09 21:07:26.170960 | controller | - Config: Waiting ... 2026-06-09 21:07:26.171002 | controller | - Gateway: OK 2026-06-09 21:07:26.171011 | controller | - GitServer: OK 2026-06-09 21:07:26.171019 | controller | - HoundSearch: OK 2026-06-09 21:07:26.171027 | controller | - LogJuicer: OK 2026-06-09 21:07:26.171035 | controller | - Logserver: OK 2026-06-09 21:07:26.171043 | controller | - MariaDB: OK 2026-06-09 21:07:26.171050 | controller | - NodePoolBuilder: OK 2026-06-09 21:07:26.171058 | controller | - NodePoolLauncher: OK 2026-06-09 21:07:26.171066 | controller | - Zookeeper: OK 2026-06-09 21:07:26.171074 | controller | - Zuul: OK 2026-06-09 21:07:26.171083 | controller | 2026-06-09 21:07:26.171131 | controller | 2026-06-09T21:07:26-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-09 21:07:31.181255 | controller | 2026-06-09T21:07:31-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:07:31.207425 | controller | 2026-06-09T21:07:31-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:07:31.226889 | controller | 2026-06-09T21:07:31-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:07:31.786351 | controller | 2026-06-09T21:07:31-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:07:31.794808 | controller | 2026-06-09T21:07:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:07:31.794844 | controller | 2026-06-09T21:07:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:07:31.802455 | controller | 2026-06-09T21:07:31-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:07:31.822900 | controller | 2026-06-09T21:07:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:07:31.822982 | controller | 2026-06-09T21:07:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:07:32.194438 | controller | 2026-06-09T21:07:32-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:07:32.197603 | controller | 2026-06-09T21:07:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:07:32.598528 | controller | 2026-06-09T21:07:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:07:32.598637 | controller | 2026-06-09T21:07:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:07:32.797038 | controller | 2026-06-09T21:07:32-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:07:33.601479 | controller | 2026-06-09T21:07:33-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:07:33.608523 | controller | 2026-06-09T21:07:33-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:07:33.608592 | controller | 2026-06-09T21:07:33-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:07:33.790902 | controller | 2026-06-09T21:07:33-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:07:34.585620 | controller | 2026-06-09T21:07:34-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:07:34.585727 | controller | 2026-06-09T21:07:34-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 21:07:35.393510 | controller | 2026-06-09T21:07:35-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:07:35.790127 | controller | 2026-06-09T21:07:35-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-09 21:07:35.982316 | controller | 2026-06-09T21:07:35-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-09 21:07:42.070370 | controller | 2026-06-09T21:07:42-04:00 INFO Deleting object, name: zuul-web 2026-06-09 21:07:42.076845 | controller | 2026-06-09T21:07:42-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-09 21:07:42.082026 | controller | 2026-06-09T21:07:42-04:00 INFO 2026-06-09 21:07:42.082070 | controller | - Config: Waiting ... 2026-06-09 21:07:42.082081 | controller | - Gateway: OK 2026-06-09 21:07:42.082090 | controller | - GitServer: OK 2026-06-09 21:07:42.082098 | controller | - HoundSearch: OK 2026-06-09 21:07:42.082106 | controller | - LogJuicer: OK 2026-06-09 21:07:42.082114 | controller | - Logserver: OK 2026-06-09 21:07:42.082123 | controller | - MariaDB: OK 2026-06-09 21:07:42.082131 | controller | - NodePoolBuilder: OK 2026-06-09 21:07:42.082139 | controller | - NodePoolLauncher: OK 2026-06-09 21:07:42.082147 | controller | - Zookeeper: OK 2026-06-09 21:07:42.082155 | controller | - Zuul: OK 2026-06-09 21:07:42.082164 | controller | 2026-06-09 21:07:42.082208 | controller | 2026-06-09T21:07:42-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-09 21:07:47.091319 | controller | 2026-06-09T21:07:47-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:07:47.121130 | controller | 2026-06-09T21:07:47-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:07:47.139667 | controller | 2026-06-09T21:07:47-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:07:47.699276 | controller | 2026-06-09T21:07:47-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:07:47.713403 | controller | 2026-06-09T21:07:47-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:07:47.713491 | controller | 2026-06-09T21:07:47-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:07:47.722801 | controller | 2026-06-09T21:07:47-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:07:47.736485 | controller | 2026-06-09T21:07:47-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:07:47.736593 | controller | 2026-06-09T21:07:47-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:07:48.105114 | controller | 2026-06-09T21:07:48-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:07:48.108435 | controller | 2026-06-09T21:07:48-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:07:48.512879 | controller | 2026-06-09T21:07:48-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:07:48.512998 | controller | 2026-06-09T21:07:48-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:07:48.705319 | controller | 2026-06-09T21:07:48-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:07:49.514504 | controller | 2026-06-09T21:07:49-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:07:49.522646 | controller | 2026-06-09T21:07:49-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:07:49.522720 | controller | 2026-06-09T21:07:49-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:07:49.700916 | controller | 2026-06-09T21:07:49-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:07:50.494721 | controller | 2026-06-09T21:07:50-04:00 INFO Creating object, name: zuul-web 2026-06-09 21:07:51.300991 | controller | 2026-06-09T21:07:51-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:07:51.303864 | controller | 2026-06-09T21:07:51-04:00 INFO 2026-06-09 21:07:51.303942 | controller | - Config: Waiting ... 2026-06-09 21:07:51.303954 | controller | - Gateway: OK 2026-06-09 21:07:51.303963 | controller | - GitServer: OK 2026-06-09 21:07:51.303972 | controller | - HoundSearch: OK 2026-06-09 21:07:51.303981 | controller | - LogJuicer: OK 2026-06-09 21:07:51.303990 | controller | - Logserver: OK 2026-06-09 21:07:51.303998 | controller | - MariaDB: OK 2026-06-09 21:07:51.304007 | controller | - NodePoolBuilder: OK 2026-06-09 21:07:51.304016 | controller | - NodePoolLauncher: OK 2026-06-09 21:07:51.304024 | controller | - Zookeeper: OK 2026-06-09 21:07:51.304033 | controller | - Zuul: Waiting ... 2026-06-09 21:07:51.304042 | controller | - |_ Config Secrets: OK 2026-06-09 21:07:51.304050 | controller | - |_ Executor: OK 2026-06-09 21:07:51.304059 | controller | - |_ Merger: OK 2026-06-09 21:07:51.304079 | controller | - |_ Scheduler: OK 2026-06-09 21:07:51.304089 | controller | - |_ Web: Waiting ... 2026-06-09 21:07:51.304098 | controller | 2026-06-09 21:07:51.304148 | controller | 2026-06-09T21:07:51-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-09 21:07:56.313470 | controller | 2026-06-09T21:07:56-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:07:56.341373 | controller | 2026-06-09T21:07:56-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:07:56.363188 | controller | 2026-06-09T21:07:56-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:07:56.925610 | controller | 2026-06-09T21:07:56-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:07:56.936374 | controller | 2026-06-09T21:07:56-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:07:56.936511 | controller | 2026-06-09T21:07:56-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:07:56.945517 | controller | 2026-06-09T21:07:56-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:07:56.955687 | controller | 2026-06-09T21:07:56-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:07:56.955784 | controller | 2026-06-09T21:07:56-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:07:57.326139 | controller | 2026-06-09T21:07:57-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:07:57.329093 | controller | 2026-06-09T21:07:57-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:07:57.731920 | controller | 2026-06-09T21:07:57-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:07:57.731996 | controller | 2026-06-09T21:07:57-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:07:57.928495 | controller | 2026-06-09T21:07:57-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:07:58.731797 | controller | 2026-06-09T21:07:58-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:07:58.740279 | controller | 2026-06-09T21:07:58-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:07:58.740333 | controller | 2026-06-09T21:07:58-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:07:58.924563 | controller | 2026-06-09T21:07:58-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:07:59.716588 | controller | 2026-06-09T21:07:59-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:07:59.716669 | controller | 2026-06-09T21:07:59-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:08:00.520907 | controller | 2026-06-09T21:08:00-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:08:00.523304 | controller | 2026-06-09T21:08:00-04:00 INFO 2026-06-09 21:08:00.523331 | controller | - Config: Waiting ... 2026-06-09 21:08:00.523338 | controller | - Gateway: OK 2026-06-09 21:08:00.523344 | controller | - GitServer: OK 2026-06-09 21:08:00.523349 | controller | - HoundSearch: OK 2026-06-09 21:08:00.523353 | controller | - LogJuicer: OK 2026-06-09 21:08:00.523357 | controller | - Logserver: OK 2026-06-09 21:08:00.523361 | controller | - MariaDB: OK 2026-06-09 21:08:00.523364 | controller | - NodePoolBuilder: OK 2026-06-09 21:08:00.523368 | controller | - NodePoolLauncher: OK 2026-06-09 21:08:00.523372 | controller | - Zookeeper: OK 2026-06-09 21:08:00.523375 | controller | - Zuul: Waiting ... 2026-06-09 21:08:00.523379 | controller | - |_ Config Secrets: OK 2026-06-09 21:08:00.523382 | controller | - |_ Executor: OK 2026-06-09 21:08:00.523386 | controller | - |_ Merger: OK 2026-06-09 21:08:00.523390 | controller | - |_ Scheduler: OK 2026-06-09 21:08:00.523393 | controller | - |_ Web: Waiting ... 2026-06-09 21:08:00.523397 | controller | 2026-06-09 21:08:00.523404 | controller | 2026-06-09T21:08:00-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-09 21:08:05.533423 | controller | 2026-06-09T21:08:05-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:08:05.562060 | controller | 2026-06-09T21:08:05-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:08:05.583461 | controller | 2026-06-09T21:08:05-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:08:06.141325 | controller | 2026-06-09T21:08:06-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:08:06.152684 | controller | 2026-06-09T21:08:06-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:08:06.152749 | controller | 2026-06-09T21:08:06-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:08:06.161816 | controller | 2026-06-09T21:08:06-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:08:06.172641 | controller | 2026-06-09T21:08:06-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:08:06.172705 | controller | 2026-06-09T21:08:06-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:08:06.545984 | controller | 2026-06-09T21:08:06-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:08:06.548455 | controller | 2026-06-09T21:08:06-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:08:06.954859 | controller | 2026-06-09T21:08:06-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:08:06.954935 | controller | 2026-06-09T21:08:06-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:08:07.146450 | controller | 2026-06-09T21:08:07-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:08:07.954758 | controller | 2026-06-09T21:08:07-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:08:07.963567 | controller | 2026-06-09T21:08:07-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:08:07.963621 | controller | 2026-06-09T21:08:07-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:08:08.144950 | controller | 2026-06-09T21:08:08-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:08:08.937005 | controller | 2026-06-09T21:08:08-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:08:08.937111 | controller | 2026-06-09T21:08:08-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:08:09.745860 | controller | 2026-06-09T21:08:09-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:08:09.748575 | controller | 2026-06-09T21:08:09-04:00 INFO 2026-06-09 21:08:09.748619 | controller | - Config: Waiting ... 2026-06-09 21:08:09.748634 | controller | - Gateway: OK 2026-06-09 21:08:09.748646 | controller | - GitServer: OK 2026-06-09 21:08:09.748658 | controller | - HoundSearch: OK 2026-06-09 21:08:09.748669 | controller | - LogJuicer: OK 2026-06-09 21:08:09.748681 | controller | - Logserver: OK 2026-06-09 21:08:09.748691 | controller | - MariaDB: OK 2026-06-09 21:08:09.748702 | controller | - NodePoolBuilder: OK 2026-06-09 21:08:09.748713 | controller | - NodePoolLauncher: OK 2026-06-09 21:08:09.748724 | controller | - Zookeeper: OK 2026-06-09 21:08:09.748734 | controller | - Zuul: Waiting ... 2026-06-09 21:08:09.748744 | controller | - |_ Config Secrets: OK 2026-06-09 21:08:09.748755 | controller | - |_ Executor: OK 2026-06-09 21:08:09.748766 | controller | - |_ Merger: OK 2026-06-09 21:08:09.748777 | controller | - |_ Scheduler: OK 2026-06-09 21:08:09.748788 | controller | - |_ Web: Waiting ... 2026-06-09 21:08:09.748800 | controller | 2026-06-09 21:08:09.748812 | controller | 2026-06-09T21:08:09-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-09 21:08:14.757790 | controller | 2026-06-09T21:08:14-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:08:14.782947 | controller | 2026-06-09T21:08:14-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:08:14.803723 | controller | 2026-06-09T21:08:14-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:08:15.361619 | controller | 2026-06-09T21:08:15-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:08:15.370287 | controller | 2026-06-09T21:08:15-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:08:15.370343 | controller | 2026-06-09T21:08:15-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:08:15.377453 | controller | 2026-06-09T21:08:15-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:08:15.388038 | controller | 2026-06-09T21:08:15-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:08:15.388085 | controller | 2026-06-09T21:08:15-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:08:15.772380 | controller | 2026-06-09T21:08:15-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:08:15.775077 | controller | 2026-06-09T21:08:15-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:08:16.177195 | controller | 2026-06-09T21:08:16-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:08:16.177349 | controller | 2026-06-09T21:08:16-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:08:16.374442 | controller | 2026-06-09T21:08:16-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:08:17.183107 | controller | 2026-06-09T21:08:17-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:08:17.197801 | controller | 2026-06-09T21:08:17-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:08:17.197872 | controller | 2026-06-09T21:08:17-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:08:17.369206 | controller | 2026-06-09T21:08:17-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:08:18.161837 | controller | 2026-06-09T21:08:18-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:08:18.161911 | controller | 2026-06-09T21:08:18-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:08:18.969171 | controller | 2026-06-09T21:08:18-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:08:18.971797 | controller | 2026-06-09T21:08:18-04:00 INFO 2026-06-09 21:08:18.971834 | controller | - Config: Waiting ... 2026-06-09 21:08:18.971844 | controller | - Gateway: OK 2026-06-09 21:08:18.971853 | controller | - GitServer: OK 2026-06-09 21:08:18.971861 | controller | - HoundSearch: OK 2026-06-09 21:08:18.971869 | controller | - LogJuicer: OK 2026-06-09 21:08:18.971877 | controller | - Logserver: OK 2026-06-09 21:08:18.971885 | controller | - MariaDB: OK 2026-06-09 21:08:18.971893 | controller | - NodePoolBuilder: OK 2026-06-09 21:08:18.971900 | controller | - NodePoolLauncher: OK 2026-06-09 21:08:18.971908 | controller | - Zookeeper: OK 2026-06-09 21:08:18.971916 | controller | - Zuul: Waiting ... 2026-06-09 21:08:18.971924 | controller | - |_ Config Secrets: OK 2026-06-09 21:08:18.971932 | controller | - |_ Executor: OK 2026-06-09 21:08:18.971939 | controller | - |_ Merger: OK 2026-06-09 21:08:18.971947 | controller | - |_ Scheduler: OK 2026-06-09 21:08:18.971955 | controller | - |_ Web: Waiting ... 2026-06-09 21:08:18.971963 | controller | 2026-06-09 21:08:18.971971 | controller | 2026-06-09T21:08:18-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-09 21:08:23.978174 | controller | 2026-06-09T21:08:23-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:08:24.007160 | controller | 2026-06-09T21:08:24-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:08:24.025770 | controller | 2026-06-09T21:08:24-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:08:24.585592 | controller | 2026-06-09T21:08:24-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:08:24.598982 | controller | 2026-06-09T21:08:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:08:24.599035 | controller | 2026-06-09T21:08:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:08:24.607748 | controller | 2026-06-09T21:08:24-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:08:24.619692 | controller | 2026-06-09T21:08:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:08:24.619758 | controller | 2026-06-09T21:08:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:08:24.990869 | controller | 2026-06-09T21:08:24-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:08:24.992882 | controller | 2026-06-09T21:08:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:08:25.397210 | controller | 2026-06-09T21:08:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:08:25.397293 | controller | 2026-06-09T21:08:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:08:25.593067 | controller | 2026-06-09T21:08:25-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:08:26.402675 | controller | 2026-06-09T21:08:26-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:08:26.411304 | controller | 2026-06-09T21:08:26-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:08:26.411357 | controller | 2026-06-09T21:08:26-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:08:26.589390 | controller | 2026-06-09T21:08:26-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:08:27.382294 | controller | 2026-06-09T21:08:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:08:27.382374 | controller | 2026-06-09T21:08:27-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:08:28.186058 | controller | 2026-06-09T21:08:28-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:08:28.189144 | controller | 2026-06-09T21:08:28-04:00 INFO 2026-06-09 21:08:28.189199 | controller | - Config: Waiting ... 2026-06-09 21:08:28.189215 | controller | - Gateway: OK 2026-06-09 21:08:28.189227 | controller | - GitServer: OK 2026-06-09 21:08:28.189290 | controller | - HoundSearch: OK 2026-06-09 21:08:28.189303 | controller | - LogJuicer: OK 2026-06-09 21:08:28.189333 | controller | - Logserver: OK 2026-06-09 21:08:28.189346 | controller | - MariaDB: OK 2026-06-09 21:08:28.189355 | controller | - NodePoolBuilder: OK 2026-06-09 21:08:28.189364 | controller | - NodePoolLauncher: OK 2026-06-09 21:08:28.189372 | controller | - Zookeeper: OK 2026-06-09 21:08:28.189379 | controller | - Zuul: Waiting ... 2026-06-09 21:08:28.189387 | controller | - |_ Config Secrets: OK 2026-06-09 21:08:28.189395 | controller | - |_ Executor: OK 2026-06-09 21:08:28.189403 | controller | - |_ Merger: OK 2026-06-09 21:08:28.189411 | controller | - |_ Scheduler: OK 2026-06-09 21:08:28.189419 | controller | - |_ Web: Waiting ... 2026-06-09 21:08:28.189427 | controller | 2026-06-09 21:08:28.189480 | controller | 2026-06-09T21:08:28-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-09 21:08:33.200595 | controller | 2026-06-09T21:08:33-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:08:33.228095 | controller | 2026-06-09T21:08:33-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:08:33.243741 | controller | 2026-06-09T21:08:33-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:08:33.809145 | controller | 2026-06-09T21:08:33-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:08:33.821727 | controller | 2026-06-09T21:08:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:08:33.821807 | controller | 2026-06-09T21:08:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:08:33.832358 | controller | 2026-06-09T21:08:33-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:08:33.845163 | controller | 2026-06-09T21:08:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:08:33.845258 | controller | 2026-06-09T21:08:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:08:34.210494 | controller | 2026-06-09T21:08:34-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:08:34.213278 | controller | 2026-06-09T21:08:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:08:34.620330 | controller | 2026-06-09T21:08:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:08:34.620421 | controller | 2026-06-09T21:08:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:08:34.808295 | controller | 2026-06-09T21:08:34-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:08:35.623319 | controller | 2026-06-09T21:08:35-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:08:35.632400 | controller | 2026-06-09T21:08:35-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:08:35.632425 | controller | 2026-06-09T21:08:35-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:08:35.811199 | controller | 2026-06-09T21:08:35-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:08:36.604777 | controller | 2026-06-09T21:08:36-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:08:36.604855 | controller | 2026-06-09T21:08:36-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:08:37.411270 | controller | 2026-06-09T21:08:37-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:08:37.414583 | controller | 2026-06-09T21:08:37-04:00 INFO 2026-06-09 21:08:37.414635 | controller | - Config: Waiting ... 2026-06-09 21:08:37.414646 | controller | - Gateway: OK 2026-06-09 21:08:37.414654 | controller | - GitServer: OK 2026-06-09 21:08:37.414662 | controller | - HoundSearch: OK 2026-06-09 21:08:37.414670 | controller | - LogJuicer: OK 2026-06-09 21:08:37.414678 | controller | - Logserver: OK 2026-06-09 21:08:37.414686 | controller | - MariaDB: OK 2026-06-09 21:08:37.414694 | controller | - NodePoolBuilder: OK 2026-06-09 21:08:37.414702 | controller | - NodePoolLauncher: OK 2026-06-09 21:08:37.414709 | controller | - Zookeeper: OK 2026-06-09 21:08:37.414717 | controller | - Zuul: Waiting ... 2026-06-09 21:08:37.414725 | controller | - |_ Config Secrets: OK 2026-06-09 21:08:37.414733 | controller | - |_ Executor: OK 2026-06-09 21:08:37.414741 | controller | - |_ Merger: OK 2026-06-09 21:08:37.414748 | controller | - |_ Scheduler: OK 2026-06-09 21:08:37.414756 | controller | - |_ Web: Waiting ... 2026-06-09 21:08:37.414764 | controller | 2026-06-09 21:08:37.414777 | controller | 2026-06-09T21:08:37-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-09 21:08:42.424205 | controller | 2026-06-09T21:08:42-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:08:42.452958 | controller | 2026-06-09T21:08:42-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:08:42.472037 | controller | 2026-06-09T21:08:42-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:08:43.030377 | controller | 2026-06-09T21:08:43-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:08:43.041185 | controller | 2026-06-09T21:08:43-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:08:43.041323 | controller | 2026-06-09T21:08:43-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:08:43.050076 | controller | 2026-06-09T21:08:43-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:08:43.064917 | controller | 2026-06-09T21:08:43-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:08:43.065034 | controller | 2026-06-09T21:08:43-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:08:43.436730 | controller | 2026-06-09T21:08:43-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:08:43.440763 | controller | 2026-06-09T21:08:43-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:08:43.840533 | controller | 2026-06-09T21:08:43-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:08:43.840611 | controller | 2026-06-09T21:08:43-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:08:44.040830 | controller | 2026-06-09T21:08:44-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:08:44.838164 | controller | 2026-06-09T21:08:44-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:08:44.846128 | controller | 2026-06-09T21:08:44-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:08:44.846170 | controller | 2026-06-09T21:08:44-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:08:45.033449 | controller | 2026-06-09T21:08:45-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:08:45.826746 | controller | 2026-06-09T21:08:45-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:08:45.826807 | controller | 2026-06-09T21:08:45-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-09 21:08:46.634505 | controller | 2026-06-09T21:08:46-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:08:46.637729 | controller | 2026-06-09T21:08:46-04:00 INFO 2026-06-09 21:08:46.637765 | controller | - Config: Waiting ... 2026-06-09 21:08:46.637776 | controller | - Gateway: OK 2026-06-09 21:08:46.637785 | controller | - GitServer: OK 2026-06-09 21:08:46.637793 | controller | - HoundSearch: OK 2026-06-09 21:08:46.637801 | controller | - LogJuicer: OK 2026-06-09 21:08:46.637808 | controller | - Logserver: OK 2026-06-09 21:08:46.637816 | controller | - MariaDB: OK 2026-06-09 21:08:46.637824 | controller | - NodePoolBuilder: OK 2026-06-09 21:08:46.637832 | controller | - NodePoolLauncher: OK 2026-06-09 21:08:46.637839 | controller | - Zookeeper: OK 2026-06-09 21:08:46.637847 | controller | - Zuul: Waiting ... 2026-06-09 21:08:46.637855 | controller | - |_ Config Secrets: OK 2026-06-09 21:08:46.637863 | controller | - |_ Executor: OK 2026-06-09 21:08:46.637871 | controller | - |_ Merger: OK 2026-06-09 21:08:46.637878 | controller | - |_ Scheduler: OK 2026-06-09 21:08:46.637886 | controller | - |_ Web: Waiting ... 2026-06-09 21:08:46.637894 | controller | 2026-06-09 21:08:46.637903 | controller | 2026-06-09T21:08:46-04:00 INFO [attempt #13] Waiting 5s for the next reconcile call ... 2026-06-09 21:08:51.644373 | controller | 2026-06-09T21:08:51-04:00 INFO Deleting object, name: ca-cert 2026-06-09 21:08:51.673263 | controller | 2026-06-09T21:08:51-04:00 INFO Waiting for statefulset, name: git-server 2026-06-09 21:08:51.689138 | controller | 2026-06-09T21:08:51-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-09 21:08:52.249988 | controller | 2026-06-09T21:08:52-04:00 INFO Waiting for statefulset, name: logserver 2026-06-09 21:08:52.258571 | controller | 2026-06-09T21:08:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-09 21:08:52.258594 | controller | 2026-06-09T21:08:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-09 21:08:52.265761 | controller | 2026-06-09T21:08:52-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-09 21:08:52.274964 | controller | 2026-06-09T21:08:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-09 21:08:52.274999 | controller | 2026-06-09T21:08:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-09 21:08:52.658432 | controller | 2026-06-09T21:08:52-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-09 21:08:52.660611 | controller | 2026-06-09T21:08:52-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-09 21:08:53.062458 | controller | 2026-06-09T21:08:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-09 21:08:53.062558 | controller | 2026-06-09T21:08:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-09 21:08:53.257010 | controller | 2026-06-09T21:08:53-04:00 INFO Waiting for statefulset, name: nodepool-builder 2026-06-09 21:08:54.070746 | controller | 2026-06-09T21:08:54-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-09 21:08:54.080176 | controller | 2026-06-09T21:08:54-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-09 21:08:54.080296 | controller | 2026-06-09T21:08:54-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-09 21:08:54.254984 | controller | 2026-06-09T21:08:54-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-09 21:08:55.048470 | controller | 2026-06-09T21:08:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-09 21:08:55.048558 | controller | 2026-06-09T21:08:55-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-09 21:08:55.855683 | controller | 2026-06-09T21:08:55-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-09 21:08:56.259908 | controller | 2026-06-09T21:08:56-04:00 INFO 2026-06-09 21:08:56.259987 | controller | - Config: OK 2026-06-09 21:08:56.259998 | controller | - Gateway: OK 2026-06-09 21:08:56.260006 | controller | - GitServer: OK 2026-06-09 21:08:56.260018 | controller | - HoundSearch: OK 2026-06-09 21:08:56.260029 | controller | - LogJuicer: OK 2026-06-09 21:08:56.260040 | controller | - Logserver: OK 2026-06-09 21:08:56.260052 | controller | - MariaDB: OK 2026-06-09 21:08:56.260063 | controller | - NodePoolBuilder: OK 2026-06-09 21:08:56.260073 | controller | - NodePoolLauncher: OK 2026-06-09 21:08:56.260084 | controller | - Zookeeper: OK 2026-06-09 21:08:56.260096 | controller | - Zuul: OK 2026-06-09 21:08:56.260107 | controller | 2026-06-09 21:08:56.260115 | controller | 2026-06-09T21:08:56-04:00 INFO Updating controller configmap ... 2026-06-09 21:08:56.647534 | controller | 2026-06-09T21:08:56-04:00 INFO Standalone reconcile done. 2026-06-10 01:08:56.941488 | controller | ok: Runtime: 0:03:58.776571 2026-06-10 01:08:56.960192 | 2026-06-10 01:08:56.960336 | TASK [health-check/test-backup-restore : Fetch the last build ID reported by Zuul web (after restore)] 2026-06-10 01:08:57.556490 | controller | ok: OK (698 bytes) 2026-06-10 01:08:57.567196 | 2026-06-10 01:08:57.567327 | TASK [health-check/test-backup-restore : Ensure Zuul SQL DB well restored by checking last build ID] 2026-06-10 01:08:57.611728 | controller | ok: All assertions passed 2026-06-10 01:08:57.620330 | 2026-06-10 01:08:57.620451 | TASK [Run a Zuul job workflow in order to validate the restored system] 2026-06-10 01:08:57.655886 | controller | ok 2026-06-10 01:08:57.692745 | 2026-06-10 01:08:57.692829 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 01:08:57.726234 | controller | ok 2026-06-10 01:08:57.757723 | 2026-06-10 01:08:57.757816 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the internal tenant] 2026-06-10 01:08:58.278851 | controller | ok: OK (2 bytes) 2026-06-10 01:08:58.290221 | 2026-06-10 01:08:58.290356 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the internal tenant] 2026-06-10 01:08:58.326358 | controller | skipping: Conditional result was False 2026-06-10 01:08:58.349571 | 2026-06-10 01:08:58.349778 | TASK [health-check/config-update-zuul : Set random value] 2026-06-10 01:08:58.532901 | controller | ok 2026-06-10 01:08:58.541535 | 2026-06-10 01:08:58.541685 | TASK [health-check/config-update-zuul : Create vars to generate and submit resources addition] 2026-06-10 01:08:58.576239 | controller | ok 2026-06-10 01:08:58.585848 | 2026-06-10 01:08:58.585976 | TASK [ansible.builtin.include_role : create-gerrit-repository] 2026-06-10 01:08:58.612523 | controller | ok 2026-06-10 01:08:58.726619 | 2026-06-10 01:08:58.726755 | TASK [create-gerrit-repository : Create gerrit repository named ichiban-config-I8dzMCD9] 2026-06-09 21:09:00.182742 | controller | 2026-06-10 01:09:00,181 INFO managesf.model.yamlbkd.engine - Resource engine is using /tmp/direct-apply as workdir 2026-06-09 21:09:00.183365 | controller | 2026-06-10 01:09:00,182 INFO managesf.model.yamlbkd.engine - Resources engine: direct apply resources requested 2026-06-09 21:09:00.202141 | controller | 2026-06-10 01:09:00,201 INFO managesf.GerritClient - ichiban-config-I8dzMCD9-core: creating group (Team core for the ichiban-config-I8dzMCD9 repo) 2026-06-09 21:09:00.266261 | controller | 2026-06-10 01:09:00,265 INFO managesf.GerritClient - ichiban-config-I8dzMCD9-core: deleting group member admin 2026-06-09 21:09:00.289598 | controller | 2026-06-10 01:09:00,288 INFO managesf.GerritClient - ichiban-config-I8dzMCD9-ptl: creating group (Team lead for the ichiban-config-I8dzMCD9 repo) 2026-06-09 21:09:00.370384 | controller | 2026-06-10 01:09:00,369 INFO managesf.GerritClient - ichiban-config-I8dzMCD9-ptl: deleting group member admin 2026-06-09 21:09:00.465569 | controller | 2026-06-10 01:09:00,464 INFO managesf.GerritClient - ichiban-config-I8dzMCD9-ptl: adding group member admin@sfop.me 2026-06-09 21:09:00.497722 | controller | 2026-06-10 01:09:00,496 INFO managesf.GerritClient - ichiban-config-I8dzMCD9: creating project (ichiban-config-I8dzMCD9 repository) 2026-06-09 21:09:00.597657 | controller | 2026-06-10 01:09:00,596 INFO managesf.services.gerrit.utils - [gerrit] Clone repository ichiban-config-I8dzMCD9 2026-06-09 21:09:00.799825 | controller | 2026-06-10 01:09:00,798 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'clone', 'ssh://admin@gerrit-sshd:29418/ichiban-config-I8dzMCD9', '/tmp/tmpj7xn4hg_/clone-ichiban-config-I8dzMCD9'] output 2026-06-09 21:09:00.799889 | controller | 2026-06-10 01:09:00,799 INFO managesf.services.gerrit.utils - Cloning into '/tmp/tmpj7xn4hg_/clone-ichiban-config-I8dzMCD9'... 2026-06-09 21:09:00.799902 | controller | Total 2 (delta 0), reused 0 (delta 0) 2026-06-09 21:09:00.799912 | controller | 2026-06-09 21:09:01.010144 | controller | 2026-06-10 01:09:01,009 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', '-a'] output 2026-06-09 21:09:01.010207 | controller | 2026-06-10 01:09:01,009 INFO managesf.services.gerrit.utils - 2026-06-09 21:09:01.089046 | controller | 2026-06-10 01:09:01,088 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on config for repository ichiban-config-I8dzMCD9 2026-06-09 21:09:01.274763 | controller | 2026-06-10 01:09:01,273 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', 'origin', 'refs/meta/config:refs/remotes/origin/meta/config'] output 2026-06-09 21:09:01.274822 | controller | 2026-06-10 01:09:01,274 INFO managesf.services.gerrit.utils - Total 4 (delta 0), reused 0 (delta 0) 2026-06-09 21:09:01.274834 | controller | From ssh://gerrit-sshd:29418/ichiban-config-I8dzMCD9 2026-06-09 21:09:01.274843 | controller | * [new ref] refs/meta/config -> origin/meta/config 2026-06-09 21:09:01.274852 | controller | 2026-06-09 21:09:01.285926 | controller | 2026-06-10 01:09:01,285 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'meta/config'] output 2026-06-09 21:09:01.286025 | controller | 2026-06-10 01:09:01,285 INFO managesf.services.gerrit.utils - Switched to a new branch 'meta/config' 2026-06-09 21:09:01.286036 | controller | branch 'meta/config' set up to track 'origin/meta/config'. 2026-06-09 21:09:01.286044 | controller | 2026-06-09 21:09:01.286132 | controller | 2026-06-10 01:09:01,285 INFO managesf.services.gerrit.utils - [gerrit] Add file project.config to index 2026-06-09 21:09:01.296454 | controller | 2026-06-10 01:09:01,295 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'project.config'] output 2026-06-09 21:09:01.296641 | controller | 2026-06-10 01:09:01,295 INFO managesf.services.gerrit.utils - 2026-06-09 21:09:01.296653 | controller | 2026-06-10 01:09:01,296 INFO managesf.services.gerrit.utils - [gerrit] Add file groups to index 2026-06-09 21:09:01.304935 | controller | 2026-06-10 01:09:01,304 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'groups'] output 2026-06-09 21:09:01.305107 | controller | 2026-06-10 01:09:01,304 INFO managesf.services.gerrit.utils - 2026-06-09 21:09:01.311152 | controller | 2026-06-10 01:09:01,310 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-09 21:09:01.311313 | controller | 2026-06-10 01:09:01,310 INFO managesf.services.gerrit.utils - M groups 2026-06-09 21:09:01.311337 | controller | M project.config 2026-06-09 21:09:01.311343 | controller | 2026-06-09 21:09:01.368544 | controller | 2026-06-10 01:09:01,367 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mProvides ACL and Groups'] output 2026-06-09 21:09:01.368585 | controller | 2026-06-10 01:09:01,367 INFO managesf.services.gerrit.utils - [meta/config e626177] Provides ACL and Groups 2026-06-09 21:09:01.368592 | controller | 2 files changed, 31 insertions(+), 9 deletions(-) 2026-06-09 21:09:01.368598 | controller | 2026-06-09 21:09:01.635226 | controller | 2026-06-10 01:09:01,633 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'meta/config:meta/config'] output 2026-06-09 21:09:01.635321 | controller | 2026-06-10 01:09:01,634 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-09 21:09:01.635334 | controller | To ssh://gerrit-sshd:29418/ichiban-config-I8dzMCD9 2026-06-09 21:09:01.635344 | controller | 77b9bbd..e626177 meta/config -> refs/meta/config 2026-06-09 21:09:01.635353 | controller | 2026-06-09 21:09:01.635372 | controller | 2026-06-10 01:09:01,634 INFO managesf.services.gerrit.utils - [gerrit] Push on config for repository ichiban-config-I8dzMCD9 2026-06-09 21:09:01.635385 | controller | 2026-06-10 01:09:01,634 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on master for repository ichiban-config-I8dzMCD9 2026-06-09 21:09:01.642367 | controller | 2026-06-10 01:09:01,641 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'master'] output 2026-06-09 21:09:01.642418 | controller | 2026-06-10 01:09:01,641 INFO managesf.services.gerrit.utils - Switched to branch 'master' 2026-06-09 21:09:01.642432 | controller | Your branch is up to date with 'origin/master'. 2026-06-09 21:09:01.642445 | controller | 2026-06-09 21:09:01.651044 | controller | 2026-06-10 01:09:01,649 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'reset', '--hard', 'origin/master'] output 2026-06-09 21:09:01.651145 | controller | 2026-06-10 01:09:01,650 INFO managesf.services.gerrit.utils - HEAD is now at 22dcf90 Initial empty repository 2026-06-09 21:09:01.651161 | controller | 2026-06-09 21:09:01.651172 | controller | 2026-06-10 01:09:01,650 INFO managesf.services.gerrit.utils - [gerrit] Add file .gitreview to index 2026-06-09 21:09:01.657359 | controller | 2026-06-10 01:09:01,656 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', '.gitreview'] output 2026-06-09 21:09:01.657416 | controller | 2026-06-10 01:09:01,656 INFO managesf.services.gerrit.utils - 2026-06-09 21:09:01.663096 | controller | 2026-06-10 01:09:01,662 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-09 21:09:01.663154 | controller | 2026-06-10 01:09:01,662 INFO managesf.services.gerrit.utils - A .gitreview 2026-06-09 21:09:01.663165 | controller | 2026-06-09 21:09:01.673101 | controller | 2026-06-10 01:09:01,672 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mManageSF commit'] output 2026-06-09 21:09:01.673156 | controller | 2026-06-10 01:09:01,672 INFO managesf.services.gerrit.utils - [master 28d1166] ManageSF commit 2026-06-09 21:09:01.673167 | controller | 1 file changed, 5 insertions(+) 2026-06-09 21:09:01.673178 | controller | create mode 100644 .gitreview 2026-06-09 21:09:01.673188 | controller | 2026-06-09 21:09:01.878974 | controller | 2026-06-10 01:09:01,878 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'master'] output 2026-06-09 21:09:01.879036 | controller | 2026-06-10 01:09:01,878 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-09 21:09:01.879049 | controller | To ssh://gerrit-sshd:29418/ichiban-config-I8dzMCD9 2026-06-09 21:09:01.879058 | controller | 22dcf90..28d1166 master -> master 2026-06-09 21:09:01.879068 | controller | 2026-06-09 21:09:01.879084 | controller | 2026-06-10 01:09:01,878 INFO managesf.services.gerrit.utils - [gerrit] Push on master for repository ichiban-config-I8dzMCD9 2026-06-09 21:09:01.879223 | controller | 2026-06-10 01:09:01,878 INFO managesf.services.gerrit.utils - [gerrit] List remote branches 2026-06-09 21:09:01.885942 | controller | 2026-06-10 01:09:01,885 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'branch', '-rv', '--abbrev=40'] output 2026-06-09 21:09:01.885991 | controller | 2026-06-10 01:09:01,885 INFO managesf.services.gerrit.utils - origin/HEAD -> origin/master 2026-06-09 21:09:01.886001 | controller | origin/master 28d1166847554c9d946da995c4492950d1d1996c ManageSF commit 2026-06-09 21:09:01.886010 | controller | origin/meta/config 77b9bbd88f654b4bd06b12f55c796ca8bb8902e8 Created project 2026-06-09 21:09:01.886018 | controller | 2026-06-09 21:09:01.886198 | controller | 2026-06-10 01:09:01,885 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-I8dzMCD9] is going to be created. 2026-06-09 21:09:01.886215 | controller | 2026-06-10 01:09:01,885 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-I8dzMCD9-core] is going to be created. 2026-06-09 21:09:01.886347 | controller | 2026-06-10 01:09:01,885 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-I8dzMCD9-ptl] is going to be created. 2026-06-09 21:09:01.886584 | controller | 2026-06-10 01:09:01,885 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-I8dzMCD9-acl] is going to be created. 2026-06-09 21:09:01.886606 | controller | 2026-06-10 01:09:01,886 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-I8dzMCD9-core] will be created. 2026-06-09 21:09:01.886614 | controller | 2026-06-10 01:09:01,886 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-I8dzMCD9-core] has been created. 2026-06-09 21:09:01.886622 | controller | 2026-06-10 01:09:01,886 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-I8dzMCD9-ptl] will be created. 2026-06-09 21:09:01.886630 | controller | 2026-06-10 01:09:01,886 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-I8dzMCD9-ptl] has been created. 2026-06-09 21:09:01.886638 | controller | 2026-06-10 01:09:01,886 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-I8dzMCD9-acl] will be created. 2026-06-09 21:09:01.886647 | controller | 2026-06-10 01:09:01,886 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-I8dzMCD9-acl] has been created. 2026-06-09 21:09:01.886656 | controller | 2026-06-10 01:09:01,886 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-I8dzMCD9] will be created. 2026-06-09 21:09:01.886668 | controller | 2026-06-10 01:09:01,886 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-I8dzMCD9] has been created. 2026-06-09 21:09:01.895380 | controller | 2026-06-09 21:09:01.895436 | controller | === Resources actions === 2026-06-09 21:09:01.895446 | controller | 2026-06-09 21:09:01.895455 | controller | Resource [type: repos, ID: ichiban-config-I8dzMCD9] is going to be created. 2026-06-09 21:09:01.895464 | controller | Resource [type: groups, ID: ichiban-config-I8dzMCD9-core] is going to be created. 2026-06-09 21:09:01.895472 | controller | Resource [type: groups, ID: ichiban-config-I8dzMCD9-ptl] is going to be created. 2026-06-09 21:09:01.895480 | controller | Resource [type: acls, ID: ichiban-config-I8dzMCD9-acl] is going to be created. 2026-06-09 21:09:01.895488 | controller | Resource [type: groups, ID: ichiban-config-I8dzMCD9-core] will be created. 2026-06-09 21:09:01.895496 | controller | Resource [type: groups, ID: ichiban-config-I8dzMCD9-core] has been created. 2026-06-09 21:09:01.895504 | controller | Resource [type: groups, ID: ichiban-config-I8dzMCD9-ptl] will be created. 2026-06-09 21:09:01.895513 | controller | Resource [type: groups, ID: ichiban-config-I8dzMCD9-ptl] has been created. 2026-06-09 21:09:01.895520 | controller | Resource [type: acls, ID: ichiban-config-I8dzMCD9-acl] will be created. 2026-06-09 21:09:01.895540 | controller | Resource [type: acls, ID: ichiban-config-I8dzMCD9-acl] has been created. 2026-06-09 21:09:01.895548 | controller | Resource [type: repos, ID: ichiban-config-I8dzMCD9] will be created. 2026-06-09 21:09:01.895556 | controller | Resource [type: repos, ID: ichiban-config-I8dzMCD9] has been created. 2026-06-10 01:09:02.262999 | controller | ok: Runtime: 0:00:03.123779 2026-06-10 01:09:02.276418 | 2026-06-10 01:09:02.276568 | TASK [create-gerrit-repository : Ensure repository exists in Gerrit] 2026-06-10 01:09:02.829471 | controller | ok: OK (752 bytes) 2026-06-10 01:09:02.846121 | 2026-06-10 01:09:02.846356 | TASK [health-check/config-update-zuul : Set bad Zuul tenant config] 2026-06-10 01:09:03.106842 | controller | changed: Block inserted 2026-06-10 01:09:03.118607 | 2026-06-10 01:09:03.118776 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 01:09:03.386701 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.004209 2026-06-10 01:09:03.387210 | controller | changed: All items complete 2026-06-10 01:09:03.387267 | 2026-06-10 01:09:03.653799 | controller | [master 5558a48] Add zuul/main.yaml 2026-06-10 01:09:03.653999 | controller | 1 file changed, 2 insertions(+), 5 deletions(-) 2026-06-10 01:09:03.654067 | controller | ok: Item: git commit -m 'Add zuul/main.yaml' Runtime: 0:00:00.055161 2026-06-10 01:09:03.676363 | 2026-06-10 01:09:03.676694 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 01:09:03.717631 | controller | ok 2026-06-10 01:09:03.775516 | 2026-06-10 01:09:03.775870 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 01:09:03.807257 | controller | ok 2026-06-10 01:09:03.822169 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 01:09:03.837624 | 2026-06-10 01:09:03.837787 | TASK [health-check/repo-submit-change : Submit change] 2026-06-09 21:09:04.601896 | controller | remote: 2026-06-09 21:09:04.601987 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 21:09:04.602024 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 21:09:04.602034 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-09 21:09:04.602045 | controller | remote: 2026-06-09 21:09:04.602054 | controller | remote: SUCCESS 2026-06-09 21:09:04.602065 | controller | remote: 2026-06-09 21:09:04.602076 | controller | remote: https://gerrit.sfop.me/c/config/+/7 Add zuul/main.yaml [NEW] 2026-06-09 21:09:04.602086 | controller | remote: 2026-06-09 21:09:04.602095 | controller | To https://gerrit.sfop.me/a/config 2026-06-09 21:09:04.602104 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 01:09:04.887535 | controller | ok: Runtime: 0:00:00.515167 2026-06-10 01:09:04.897561 | 2026-06-10 01:09:04.897743 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-09 21:09:05.189881 | controller | I6806c7fc337c663853a4a2fa38c1e562c8fdd224 2026-06-10 01:09:05.449835 | controller | ok: Runtime: 0:00:00.008650 2026-06-10 01:09:05.460469 | 2026-06-10 01:09:05.460695 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 01:09:06.025018 | controller | ok: OK (47 bytes) 2026-06-10 01:09:06.034492 | 2026-06-10 01:09:06.034613 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 01:09:06.068669 | controller | ok 2026-06-10 01:09:06.082445 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 01:09:06.092827 | 2026-06-10 01:09:06.092910 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 01:09:06.635553 | controller | ok: OK (7142 bytes) 2026-06-10 01:09:06.647079 | 2026-06-10 01:09:06.647220 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 01:09:06.704697 | controller | ok 2026-06-10 01:09:06.723876 | 2026-06-10 01:09:06.724032 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 01:10:10.306680 | controller | ok: OK (1163 bytes) 2026-06-10 01:10:10.317815 | 2026-06-10 01:10:10.317948 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 01:10:10.355750 | controller | skipping: Conditional result was False 2026-06-10 01:10:10.368980 | 2026-06-10 01:10:10.369120 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 01:10:10.406671 | controller | skipping: Conditional result was False 2026-06-10 01:10:10.419295 | 2026-06-10 01:10:10.419437 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 01:10:10.457467 | controller | skipping: Conditional result was False 2026-06-10 01:10:10.480186 | 2026-06-10 01:10:10.480418 | TASK [health-check/config-update-zuul : Set Zuul tenant config] 2026-06-10 01:10:10.752159 | controller | changed: Block inserted 2026-06-10 01:10:10.761507 | 2026-06-10 01:10:10.761669 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 01:10:11.002831 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.006104 2026-06-10 01:10:11.295393 | controller | [master 4888d4b] Add zuul/main.yaml 2026-06-10 01:10:11.295627 | controller | Date: Tue Jun 9 21:09:03 2026 -0400 2026-06-10 01:10:11.295693 | controller | 1 file changed, 2 insertions(+), 2 deletions(-) 2026-06-10 01:10:11.295750 | controller | ok: Item: git commit --amend --no-edit Runtime: 0:00:00.049322 2026-06-10 01:10:11.313939 | 2026-06-10 01:10:11.314113 | TASK [Submit change and verify Zuul console output] 2026-06-10 01:10:11.353082 | controller | ok 2026-06-10 01:10:11.403697 | 2026-06-10 01:10:11.403816 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 01:10:11.434409 | controller | ok 2026-06-10 01:10:11.442350 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 01:10:11.451717 | 2026-06-10 01:10:11.451802 | TASK [health-check/repo-submit-change : Submit change] 2026-06-09 21:10:12.388986 | controller | remote: 2026-06-09 21:10:12.389070 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-09 21:10:12.389083 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-09 21:10:12.389094 | controller | remote: Processing changes: refs: 1, updated: 1, done 2026-06-09 21:10:12.389104 | controller | remote: 2026-06-09 21:10:12.389113 | controller | remote: SUCCESS 2026-06-09 21:10:12.389122 | controller | remote: 2026-06-09 21:10:12.389132 | controller | remote: https://gerrit.sfop.me/c/config/+/7 Add zuul/main.yaml 2026-06-09 21:10:12.389141 | controller | remote: 2026-06-09 21:10:12.389149 | controller | To https://gerrit.sfop.me/a/config 2026-06-09 21:10:12.389186 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 01:10:12.484895 | controller | ok: Runtime: 0:00:00.697308 2026-06-10 01:10:12.495177 | 2026-06-10 01:10:12.495313 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-09 21:10:12.799125 | controller | I6806c7fc337c663853a4a2fa38c1e562c8fdd224 2026-06-10 01:10:13.040470 | controller | ok: Runtime: 0:00:00.009907 2026-06-10 01:10:13.050171 | 2026-06-10 01:10:13.050305 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 01:10:13.634135 | controller | ok: OK (47 bytes) 2026-06-10 01:10:13.646173 | 2026-06-10 01:10:13.646311 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 01:10:13.683875 | controller | skipping: Conditional result was False 2026-06-10 01:10:13.694793 | 2026-06-10 01:10:13.694932 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 01:10:13.740732 | controller | ok 2026-06-10 01:10:13.753174 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-zuul-console-success.yaml 2026-06-10 01:10:13.772756 | 2026-06-10 01:10:13.772888 | TASK [health-check/repo-submit-change : Wait for config-update job] 2026-06-10 01:11:16.088996 | controller | ok: Runtime: 0:00:00.049582 2026-06-10 01:11:16.100813 | 2026-06-10 01:11:16.100994 | TASK [health-check/repo-submit-change : Connect and validate console stream] 2026-06-09 21:11:16.496117 | controller | 2026-06-10 01:11:07.270551 | Job console starting 2026-06-09 21:11:16.496170 | controller | 2026-06-10 01:11:07.271092 | Updating git repos 2026-06-09 21:11:16.496177 | controller | 2026-06-10 01:11:07.286064 | Cloning repos into workspace 2026-06-09 21:11:16.496182 | controller | 2026-06-10 01:11:08.503536 | Restoring repo states 2026-06-09 21:11:16.496188 | controller | 2026-06-10 01:11:08.510585 | Merging changes 2026-06-09 21:11:16.496193 | controller | 2026-06-10 01:11:08.510606 | Checking out repos 2026-06-09 21:11:16.496198 | controller | 2026-06-10 01:11:08.577049 | Preparing playbooks 2026-06-09 21:11:16.496202 | controller | 2026-06-10 01:11:09.320707 | Running Ansible setup 2026-06-09 21:11:16.496207 | controller | 2026-06-10 01:11:12.305146 | PRE-RUN START: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-09 21:11:16.496212 | controller | 2026-06-10 01:11:14.184416 | 2026-06-09 21:11:16.496216 | controller | 2026-06-10 01:11:14.184585 | PLAY [localhost] 2026-06-09 21:11:16.496220 | controller | 2026-06-10 01:11:14.193869 | 2026-06-09 21:11:16.496224 | controller | 2026-06-10 01:11:14.193966 | TASK [Gathering Facts] 2026-06-09 21:11:16.496242 | controller | 2026-06-10 01:11:16.206538 | localhost | ok 2026-06-09 21:11:16.496248 | controller | 2026-06-10 01:11:16.272912 | 2026-06-09 21:11:16.496255 | controller | 2026-06-10 01:11:16.273114 | TASK [Setup log path fact] 2026-06-09 21:11:16.496261 | controller | 2026-06-10 01:11:16.292639 | localhost | ok 2026-06-09 21:11:16.496266 | controller | 2026-06-10 01:11:16.366801 | 2026-06-09 21:11:16.496272 | controller | 2026-06-10 01:11:16.367007 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-09 21:11:16.496277 | controller | 2026-06-10 01:11:16.399691 | localhost | ok 2026-06-09 21:11:16.496282 | controller | 2026-06-10 01:11:16. 2026-06-09 21:11:16.496291 | controller | 463744 | 2026-06-09 21:11:16.496299 | controller | 2026-06-10 01:11:16.463933 | TASK [emit-job-header : Print job information] 2026-06-09 21:11:16.496303 | controller | 2026-06-09 21:11:16.997899 | controller | 2026-06-10 01:11:16.504346 | # Job Information 2026-06-09 21:11:16.997972 | controller | 2026-06-10 01:11:16.504597 | Ansible Version: 2.16.18 2026-06-09 21:11:16.997985 | controller | 2026-06-10 01:11:16.504660 | Job: config-update 2026-06-09 21:11:16.997994 | controller | 2026-06-10 01:11:16.504701 | Pipeline: post 2026-06-09 21:11:16.998003 | controller | 2026-06-10 01:11:16.504737 | Executor: zuul-executor-0.zuul-executor-headless.sf.svc.cluster.local 2026-06-09 21:11:16.998012 | controller | 2026-06-10 01:11:16.504773 | Triggered by: https://gerrit.sfop.me/plugins/gitiles/config/+/4888d4b0cd22e76b63f86f13f5302d2abad81c24^!/ 2026-06-09 21:11:16.998021 | controller | 2026-06-10 01:11:16.504815 | Log URL (when completed): https://sfop.me/logs/d9f/internal/d9f26b3a6fa84ef3833eea4fed54eae1/ 2026-06-09 21:11:16.998030 | controller | 2026-06-10 01:11:16.504852 | Event ID: d7e9ed02fe2c4b47a399ed3d184dd65e 2026-06-09 21:11:16.998038 | controller | 2026-06-10 01:11:16.563896 | 2026-06-09 21:11:16.998046 | controller | 2026-06-10 01:11:16.564102 | LOOP [emit-job-header : Print node information] 2026-06-09 21:11:16.998054 | controller | 2026-06-10 01:11:16.600795 | 2026-06-09 21:11:16.998062 | controller | 2026-06-10 01:11:16.601058 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-09 21:11:16.998071 | controller | 2026-06-09 21:11:17.997635 | controller | 2026-06-10 01:11:17.510482 | localhost -> localhost | changed 2026-06-09 21:11:17.997697 | controller | 2026-06-10 01:11:17.570305 | 2026-06-09 21:11:17.997708 | controller | 2026-06-10 01:11:17.570471 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-06-09 21:11:17.997717 | controller | 2026-06-09 21:11:20.498153 | controller | 2026-06-10 01:11:20.105213 | localhost -> localhost | changed 2026-06-09 21:11:20.498213 | controller | 2026-06-10 01:11:20.167564 | 2026-06-09 21:11:20.498219 | controller | 2026-06-10 01:11:20.167724 | PLAY [all] 2026-06-09 21:11:20.498225 | controller | 2026-06-10 01:11:20.167930 | 2026-06-09 21:11:20.498243 | controller | 2026-06-10 01:11:20.167985 | PLAY RECAP 2026-06-09 21:11:20.498251 | controller | 2026-06-10 01:11:20.168049 | localhost | ok: 5 changed: 2 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-06-09 21:11:20.498256 | controller | 2026-06-10 01:11:20.168093 | 2026-06-09 21:11:20.498261 | controller | 2026-06-10 01:11:20.476193 | PRE-RUN END RESULT_NORMAL: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-09 21:11:20.498267 | controller | 2026-06-10 01:11:20.477205 | RUN START: [trusted : git-server/system-config/playbooks/config/update.yaml@master] 2026-06-09 21:11:20.498286 | controller | 2026-06-10 01:11:21.651945 | controller | ok: Runtime: 0:00:05.024313 2026-06-10 01:11:21.665505 | 2026-06-10 01:11:21.665681 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 01:11:21.712745 | controller | ok 2026-06-10 01:11:21.725941 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 01:11:21.746247 | 2026-06-10 01:11:21.746336 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-09 21:11:22.024194 | controller | 4888d4b0cd22e76b63f86f13f5302d2abad81c24 2026-06-10 01:11:22.285749 | controller | ok: Runtime: 0:00:00.004540 2026-06-10 01:11:22.298184 | 2026-06-10 01:11:22.298486 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 01:12:46.995113 | controller | ok: OK (1143 bytes) 2026-06-10 01:12:47.008985 | 2026-06-10 01:12:47.009121 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 01:12:47.048144 | controller | skipping: Conditional result was False 2026-06-10 01:12:47.069114 | 2026-06-10 01:12:47.069295 | TASK [health-check/config-update-zuul : ansible.builtin.set_fact] 2026-06-10 01:12:47.116447 | controller | ok 2026-06-10 01:12:47.127400 | 2026-06-10 01:12:47.127552 | TASK [Check that Zuul has pushed the job console on the logserver] 2026-06-10 01:12:47.164275 | controller | ok 2026-06-10 01:12:47.222567 | 2026-06-10 01:12:47.222736 | TASK [health-check/ensure-job-result-artifacts : Ensure job console (job-output.txt.gz) exists in Logserver] 2026-06-10 01:12:47.754294 | controller | ok: OK (4748 bytes) 2026-06-10 01:12:47.765945 | 2026-06-10 01:12:47.766087 | TASK [health-check/ensure-job-result-artifacts : Ensure internal logserver rewrite from job-output.txt to job-output.txt.gz] 2026-06-10 01:12:48.289561 | controller | ok: OK (4748 bytes) 2026-06-10 01:12:48.308083 | 2026-06-10 01:12:48.308283 | TASK [health-check/config-update-zuul : Get Zuul projects list] 2026-06-10 01:12:48.792931 | controller | ok: OK (144 bytes) 2026-06-10 01:12:48.809974 | 2026-06-10 01:12:48.810132 | TASK [health-check/config-update-zuul : Ensure project exists in Zuul (fail if not)] 2026-06-10 01:12:48.862470 | controller | skipping: Conditional result was False 2026-06-10 01:12:48.893229 | 2026-06-10 01:12:48.893414 | LOOP [health-check/test-nodepool-launcher-pod : Reset config dir to HEAD] 2026-06-10 01:12:49.350483 | controller | Fetching origin 2026-06-10 01:12:49.351000 | controller | From https://gerrit.sfop.me/a/demo-project 2026-06-10 01:12:49.351065 | controller | 4ab7d04..bb037cc master -> origin/master 2026-06-10 01:12:49.351462 | controller | Fetching gerrit 2026-06-10 01:12:49.351644 | controller | ok: Item: git remote update Runtime: 0:00:00.149329 2026-06-10 01:12:49.351800 | controller | changed: All items complete 2026-06-10 01:12:49.351868 | 2026-06-10 01:12:49.568712 | controller | HEAD is now at bb037cc Update demo-job to sleep 1 second 2026-06-10 01:12:49.568927 | controller | ok: Item: git reset --hard origin/master Runtime: 0:00:00.008436 2026-06-10 01:12:49.586899 | 2026-06-10 01:12:49.587120 | LOOP [health-check/test-nodepool-launcher-pod : Ensure playbooks and zuul.d dir exists] 2026-06-10 01:12:49.875061 | controller | ok: "playbooks" 2026-06-10 01:12:50.100966 | controller | changed: "zuul.d" 2026-06-10 01:12:50.120477 | 2026-06-10 01:12:50.120747 | TASK [health-check/test-nodepool-launcher-pod : Create new zuul config] 2026-06-10 01:12:50.969826 | controller | changed 2026-06-10 01:12:50.979786 | 2026-06-10 01:12:50.979920 | TASK [health-check/test-nodepool-launcher-pod : Create simple playbook] 2026-06-10 01:12:51.769511 | controller | changed 2026-06-10 01:12:51.778673 | 2026-06-10 01:12:51.778861 | LOOP [health-check/test-nodepool-launcher-pod : Create git commit message] 2026-06-10 01:12:52.081971 | controller | ok: Item: git add . Runtime: 0:00:00.007140 2026-06-10 01:12:52.082467 | controller | changed: All items complete 2026-06-10 01:12:52.082528 | 2026-06-10 01:12:52.373215 | controller | [test_short_job 07902da] Provide a demo job for testing the local pod spawning 2026-06-10 01:12:52.373401 | controller | 2 files changed, 18 insertions(+) 2026-06-10 01:12:52.373453 | controller | create mode 100644 playbooks/microshift-pod.yml 2026-06-10 01:12:52.373494 | controller | create mode 100644 zuul.d/config.yaml 2026-06-10 01:12:52.373548 | controller | ok: Item: git commit -m 'Provide a demo job for testing the local pod spawning' Runtime: 0:00:00.056748 2026-06-10 01:12:52.391385 | 2026-06-10 01:12:52.391520 | TASK [Submit change and validate job success] 2026-06-10 01:12:52.429024 | controller | ok 2026-06-10 01:12:52.468063 | 2026-06-10 01:12:52.468227 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 01:12:52.500767 | controller | ok 2026-06-10 01:12:52.509402 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 01:12:52.516889 | 2026-06-10 01:12:52.516971 | TASK [health-check/repo-submit-change : Submit change] 2026-06-09 21:12:53.402423 | controller | remote: 2026-06-09 21:12:53.402529 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 21:12:53.402542 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-09 21:12:53.402551 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-09 21:12:53.402561 | controller | remote: commit 07902da: warning: subject >50 characters; use shorter first paragraph 2026-06-09 21:12:53.402573 | controller | remote: 2026-06-09 21:12:53.402581 | controller | remote: SUCCESS 2026-06-09 21:12:53.402590 | controller | remote: 2026-06-09 21:12:53.402598 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/8 Provide a demo job for testing the local pod spawning [NEW] 2026-06-09 21:12:53.402609 | controller | remote: 2026-06-09 21:12:53.402617 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-09 21:12:53.402628 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 01:12:53.556955 | controller | ok: Runtime: 0:00:00.633133 2026-06-10 01:12:53.561935 | 2026-06-10 01:12:53.562003 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-09 21:12:53.839280 | controller | Id43772397971bec72680d74c0f9ec12c53e34f26 2026-06-10 01:12:54.090862 | controller | ok: Runtime: 0:00:00.009710 2026-06-10 01:12:54.097439 | 2026-06-10 01:12:54.097512 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 01:12:54.124370 | controller | skipping: Conditional result was False 2026-06-10 01:12:54.133819 | 2026-06-10 01:12:54.133944 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 01:12:54.158161 | controller | skipping: Conditional result was False 2026-06-10 01:12:54.171215 | 2026-06-10 01:12:54.171404 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 01:12:54.196951 | controller | skipping: Conditional result was False 2026-06-10 01:12:54.206417 | 2026-06-10 01:12:54.206519 | TASK [health-check/repo-submit-change : Ensure a success for microshift-pod post job] 2026-06-10 01:12:54.240639 | controller | skipping: Conditional result was False 2026-06-10 01:12:54.254566 | 2026-06-10 01:12:54.254760 | TASK [health-check/repo-submit-change : Ensure a success for microshift-pod job] 2026-06-10 01:12:54.299688 | controller | ok 2026-06-10 01:12:54.314446 | controller | included: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-success.yaml 2026-06-10 01:12:54.328740 | 2026-06-10 01:12:54.328872 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 01:12:54.850551 | controller | ok: OK (2986 bytes) 2026-06-10 01:12:54.865327 | 2026-06-10 01:12:54.865464 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 01:12:54.923352 | controller | ok 2026-06-10 01:12:54.936897 | 2026-06-10 01:12:54.937047 | TASK [health-check/repo-submit-change : Wait microshift-pod SUCCESS result using zuul-web API] 2026-06-10 01:14:50.991740 | controller | ok: OK (798 bytes) 2026-06-10 01:14:51.009689 | 2026-06-10 01:14:51.009786 | TASK [health-check/test-volumestats-sidecar : ensure logserver deployment is ready] 2026-06-10 01:14:51.624215 | controller | ok: Runtime: 0:00:00.101172 2026-06-10 01:14:51.639346 | 2026-06-10 01:14:51.639527 | TASK [health-check/test-volumestats-sidecar : Start port forwarding to logserver sidecar] 2026-06-10 01:14:52.355764 | controller | ok: ERROR (ignored) 2026-06-10 01:14:52.356170 | controller | { 2026-06-10 01:14:52.356229 | controller | "ansible_job_id": "j641233627347.187865", 2026-06-10 01:14:52.356270 | controller | "finished": 0, 2026-06-10 01:14:52.356307 | controller | "results_file": "/home/zuul-worker/.ansible_async/j641233627347.187865", 2026-06-10 01:14:52.356343 | controller | "started": 1 2026-06-10 01:14:52.356377 | controller | } 2026-06-10 01:14:52.365213 | 2026-06-10 01:14:52.365348 | TASK [health-check/test-volumestats-sidecar : Pause to ensure the port forwarding is up] 2026-06-10 01:15:12.415053 | controller | ok 2026-06-10 01:15:12.424158 | 2026-06-10 01:15:12.424296 | TASK [health-check/test-volumestats-sidecar : Ensure volume info is exposed by metrics] 2026-06-10 01:15:12.997138 | controller | ok: OK (unknown bytes) 2026-06-10 01:15:13.017142 | 2026-06-10 01:15:13.017276 | TASK [Register CLI version] 2026-06-09 21:15:13.642086 | controller | v0.0.67-9-g0f780bbe 2026-06-10 01:15:14.059698 | controller | ok: Runtime: 0:00:00.346238 2026-06-10 01:15:14.068943 | 2026-06-10 01:15:14.069076 | TASK [ensure annotations are present on sf-standalone-owner] 2026-06-10 01:15:14.616699 | controller | ok: Runtime: 0:00:00.098764 2026-06-10 01:15:14.620265 | 2026-06-10 01:15:14.620368 | PLAY RECAP 2026-06-10 01:15:14.620460 | controller | ok: 367 changed: 219 unreachable: 0 failed: 0 skipped: 57 rescued: 0 ignored: 0 2026-06-10 01:15:14.620503 | 2026-06-10 01:15:14.894413 | RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-tests.yaml@master] 2026-06-10 01:15:14.897138 | POST-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/get-minikube-logs.yaml@master] 2026-06-10 01:15:15.522930 | 2026-06-10 01:15:15.523046 | PLAY [controller] 2026-06-10 01:15:15.532614 | 2026-06-10 01:15:15.532701 | TASK [Grab minikube logs] 2026-06-10 01:15:18.564561 | controller | ok: Runtime: 0:00:02.437672 2026-06-10 01:15:18.573993 | 2026-06-10 01:15:18.574154 | TASK [ansible.builtin.include_role : post/get-kubectl-logs] 2026-06-10 01:15:18.600466 | controller | ok 2026-06-10 01:15:18.630851 | 2026-06-10 01:15:18.630948 | TASK [post/get-kubectl-logs : Set proper facts] 2026-06-10 01:15:18.660914 | controller | ok 2026-06-10 01:15:18.665638 | 2026-06-10 01:15:18.665721 | TASK [post/get-kubectl-logs : Get pod logs] 2026-06-10 01:15:28.705538 | controller | ok: Runtime: 0:00:09.376416 2026-06-10 01:15:28.709215 | 2026-06-10 01:15:28.709279 | LOOP [post/get-kubectl-logs : Describe namespace resources] 2026-06-10 01:15:33.665744 | controller | ok: Item: pods Runtime: 0:00:04.627959 2026-06-10 01:15:35.610359 | controller | ok: Item: deployments Runtime: 0:00:01.668478 2026-06-10 01:15:37.904348 | controller | ok: Item: statefulsets Runtime: 0:00:02.044846 2026-06-10 01:15:41.583022 | controller | ok: Item: services Runtime: 0:00:03.395408 2026-06-10 01:15:44.463678 | controller | ok: Item: secrets Runtime: 0:00:02.622969 2026-06-10 01:15:49.076200 | controller | ok: Item: configmaps Runtime: 0:00:04.378302 2026-06-10 01:15:50.549517 | controller | ok: Item: jobs Runtime: 0:00:01.231802 2026-06-10 01:15:52.990172 | controller | ok: Item: pvc Runtime: 0:00:02.227827 2026-06-10 01:15:54.024820 | controller | ok: Item: ingress Runtime: 0:00:00.803175 2026-06-10 01:15:54.039242 | 2026-06-10 01:15:54.039401 | TASK [post/get-kubectl-logs : Get events] 2026-06-10 01:15:56.085974 | controller | ok: Runtime: 0:00:01.473031 2026-06-10 01:15:56.094033 | 2026-06-10 01:15:56.094168 | TASK [post/get-kubectl-logs : Fetch decoded secrets] 2026-06-10 01:15:59.140705 | controller | ok: Runtime: 0:00:02.648908 2026-06-10 01:15:59.155370 | 2026-06-10 01:15:59.155511 | TASK [ansible.builtin.include_role : post/get-logserver-content] 2026-06-10 01:15:59.181371 | controller | ok 2026-06-10 01:15:59.215223 | 2026-06-10 01:15:59.215355 | TASK [post/get-logserver-content : Ensure removal of ~/zuul-output/logs/logserver-content] 2026-06-10 01:15:59.593565 | controller | ok 2026-06-10 01:15:59.601345 | 2026-06-10 01:15:59.601473 | TASK [post/get-logserver-content : Get logserver pod name] 2026-06-09 21:15:59.991699 | controller | logserver-0 2026-06-10 01:16:00.146828 | controller | ok: Runtime: 0:00:00.102748 2026-06-10 01:16:00.158526 | 2026-06-10 01:16:00.158720 | TASK [post/get-logserver-content : Fetch logserver content on locally in ~/zuul-output/logs/logserver-content] 2026-06-09 21:16:00.668432 | controller | tar: Removing leading `/' from member names 2026-06-10 01:16:00.727346 | controller | ok: Runtime: 0:00:00.237305 2026-06-10 01:16:00.741916 | 2026-06-10 01:16:00.742045 | TASK [ansible.builtin.include_role : post/get-repos] 2026-06-10 01:16:00.767809 | controller | ok 2026-06-10 01:16:00.801564 | 2026-06-10 01:16:00.801671 | TASK [post/get-repos : Set proper facts] 2026-06-10 01:16:00.832758 | controller | ok 2026-06-10 01:16:00.838758 | 2026-06-10 01:16:00.838843 | TASK [post/get-repos : Ensure ~/zuul-output/logs/repos/] 2026-06-10 01:16:01.124195 | controller | changed 2026-06-10 01:16:01.133230 | 2026-06-10 01:16:01.133359 | TASK [post/get-repos : Copy repositories] 2026-06-10 01:16:01.937078 | controller | changed 2026-06-10 01:16:01.956113 | 2026-06-10 01:16:01.956245 | TASK [ansible.builtin.include_role : post/get-loki-logs] 2026-06-10 01:16:01.990652 | controller | ok 2026-06-10 01:16:02.028030 | 2026-06-10 01:16:02.028112 | TASK [post/get-loki-logs : Ensure output dir exists for loki logs] 2026-06-10 01:16:02.306149 | controller | changed 2026-06-10 01:16:02.314547 | 2026-06-10 01:16:02.314711 | TASK [post/get-loki-logs : Set realpath for the log directory] 2026-06-09 21:16:02.598179 | controller | /home/zuul-worker/zuul-output/logs/loki-logs 2026-06-10 01:16:02.858554 | controller | ok: Runtime: 0:00:00.005280 2026-06-10 01:16:02.867164 | 2026-06-10 01:16:02.867297 | TASK [post/get-loki-logs : Start port forwarding to loki-http if needed] 2026-06-10 01:16:03.665500 | controller | ok: ERROR (ignored) 2026-06-10 01:16:03.666103 | controller | { 2026-06-10 01:16:03.666194 | controller | "ansible_job_id": "j166610042768.194310", 2026-06-10 01:16:03.666254 | controller | "finished": 0, 2026-06-10 01:16:03.666309 | controller | "results_file": "/home/zuul-worker/.ansible_async/j166610042768.194310", 2026-06-10 01:16:03.666365 | controller | "started": 1 2026-06-10 01:16:03.666419 | controller | } 2026-06-10 01:16:03.679869 | 2026-06-10 01:16:03.680136 | TASK [post/get-loki-logs : Ensure loki is reachable] 2026-06-10 01:16:04.479317 | controller | ok: OK (141 bytes) 2026-06-10 01:16:04.489388 | 2026-06-10 01:16:04.489522 | TASK [post/get-loki-logs : Get aggregated logs prefixed by service] 2026-06-10 01:16:06.540088 | controller | ok: Runtime: 0:00:01.449489 2026-06-10 01:16:06.548445 | 2026-06-10 01:16:06.548621 | TASK [post/get-loki-logs : Change owner and group for the log dir] 2026-06-10 01:16:07.097862 | controller | ok: Runtime: 0:00:00.008650 2026-06-10 01:16:07.108759 | 2026-06-10 01:16:07.108857 | PLAY RECAP 2026-06-10 01:16:07.108965 | controller | ok: 18 changed: 14 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 01:16:07.109013 | 2026-06-10 01:16:07.288855 | POST-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/get-minikube-logs.yaml@master] 2026-06-10 01:16:07.291045 | POST-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/post.yaml@main] 2026-06-10 01:16:07.977894 | 2026-06-10 01:16:07.978038 | PLAY [all] 2026-06-10 01:16:07.988224 | 2026-06-10 01:16:07.988304 | TASK [include_role : fetch-output] 2026-06-10 01:16:08.018322 | controller | ok 2026-06-10 01:16:08.030826 | 2026-06-10 01:16:08.030902 | TASK [fetch-output : Set log path for multiple nodes] 2026-06-10 01:16:08.117353 | controller | skipping: Conditional result was False 2026-06-10 01:16:08.129235 | 2026-06-10 01:16:08.129475 | TASK [fetch-output : Set log path for single node] 2026-06-10 01:16:08.196704 | controller | ok 2026-06-10 01:16:08.204200 | 2026-06-10 01:16:08.204334 | LOOP [fetch-output : Ensure local output dirs] 2026-06-10 01:16:08.666884 | controller -> localhost | changed: "/var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/work/logs" 2026-06-10 01:16:08.930388 | controller -> localhost | changed: "/var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/work/artifacts" 2026-06-10 01:16:09.246450 | controller -> localhost | changed: "/var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/work/docs" 2026-06-10 01:16:09.261243 | 2026-06-10 01:16:09.261474 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-06-10 01:16:10.154217 | controller | changed: 2026-06-10 01:16:10.154717 | controller | .d..t...... ./ 2026-06-10 01:16:10.154824 | controller | >f+++++++++ dmesg.txt 2026-06-10 01:16:10.154900 | controller | >f+++++++++ minikube.logs 2026-06-10 01:16:10.154968 | controller | >f+++++++++ sf-operator-dry-run-lifecycle.log 2026-06-10 01:16:10.155062 | controller | >f+++++++++ sf-operator-dry-run-update.log 2026-06-10 01:16:10.155138 | controller | >f+++++++++ sf-operator.log 2026-06-10 01:16:10.155209 | controller | cd+++++++++ decoded-secrets/ 2026-06-10 01:16:10.155280 | controller | >f+++++++++ decoded-secrets/ingress-nginx--ingress-nginx-admission 2026-06-10 01:16:10.155351 | controller | >f+++++++++ decoded-secrets/kube-system--bootstrap-token-3qyetz 2026-06-10 01:16:10.155418 | controller | >f+++++++++ decoded-secrets/nodepool--nodepool-token 2026-06-10 01:16:10.155486 | controller | >f+++++++++ decoded-secrets/sf--admin-ssh-key 2026-06-10 01:16:10.155552 | controller | >f+++++++++ decoded-secrets/sf--config-update-secrets 2026-06-10 01:16:10.155714 | controller | >f+++++++++ decoded-secrets/sf--dummygerritsecret 2026-06-10 01:16:10.155787 | controller | >f+++++++++ decoded-secrets/sf--es-basicauth 2026-06-10 01:16:10.155850 | controller | >f+++++++++ decoded-secrets/sf--gerrit-admin-api-key 2026-06-10 01:16:10.155913 | controller | >f+++++++++ decoded-secrets/sf--logserver-keys 2026-06-10 01:16:10.155973 | controller | >f+++++++++ decoded-secrets/sf--logserver-uploader-keys 2026-06-10 01:16:10.156032 | controller | >f+++++++++ decoded-secrets/sf--logserver-uploader-spare-keys 2026-06-10 01:16:10.156092 | controller | >f+++++++++ decoded-secrets/sf--mariadb-config-secrets 2026-06-10 01:16:10.156147 | controller | >f+++++++++ decoded-secrets/sf--mariadb-initdb-secrets 2026-06-10 01:16:10.156202 | controller | >f+++++++++ decoded-secrets/sf--mariadb-root-password 2026-06-10 01:16:10.156263 | controller | >f+++++++++ decoded-secrets/sf--nodepool-builder-ssh-key 2026-06-10 01:16:10.156331 | controller | >f+++++++++ decoded-secrets/sf--nodepool-providers-secrets 2026-06-10 01:16:10.156401 | controller | >f+++++++++ decoded-secrets/sf--self-signed-cert 2026-06-10 01:16:10.156469 | controller | >f+++++++++ decoded-secrets/sf--smtp-secret 2026-06-10 01:16:10.156537 | controller | >f+++++++++ decoded-secrets/sf--zookeeper-client-tls 2026-06-10 01:16:10.156666 | controller | >f+++++++++ decoded-secrets/sf--zookeeper-server-tls 2026-06-10 01:16:10.156741 | controller | >f+++++++++ decoded-secrets/sf--zuul-auth-secret 2026-06-10 01:16:10.156804 | controller | >f+++++++++ decoded-secrets/sf--zuul-config 2026-06-10 01:16:10.156868 | controller | >f+++++++++ decoded-secrets/sf--zuul-db-connection 2026-06-10 01:16:10.156933 | controller | >f+++++++++ decoded-secrets/sf--zuul-gerrit-api-key 2026-06-10 01:16:10.156998 | controller | >f+++++++++ decoded-secrets/sf--zuul-keystore-password 2026-06-10 01:16:10.157061 | controller | >f+++++++++ decoded-secrets/sf--zuul-spare-ssh-key 2026-06-10 01:16:10.157125 | controller | >f+++++++++ decoded-secrets/sf--zuul-ssh-key 2026-06-10 01:16:10.157189 | controller | cd+++++++++ events/ 2026-06-10 01:16:10.157253 | controller | >f+++++++++ events/default.txt 2026-06-10 01:16:10.157317 | controller | >f+++++++++ events/ingress-nginx.txt 2026-06-10 01:16:10.157379 | controller | >f+++++++++ events/kube-node-lease.txt 2026-06-10 01:16:10.157452 | controller | >f+++++++++ events/kube-public.txt 2026-06-10 01:16:10.157519 | controller | >f+++++++++ events/kube-system.txt 2026-06-10 01:16:10.157642 | controller | >f+++++++++ events/nodepool.txt 2026-06-10 01:16:10.157736 | controller | >f+++++++++ events/sf.txt 2026-06-10 01:16:10.157844 | controller | cd+++++++++ items-describe/ 2026-06-10 01:16:10.157918 | controller | >f+++++++++ items-describe/default--configmaps--kube-root-ca.crt.txt 2026-06-10 01:16:10.157988 | controller | >f+++++++++ items-describe/default--services--kubernetes.txt 2026-06-10 01:16:10.158052 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--ingress-nginx-controller.txt 2026-06-10 01:16:10.158123 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--kube-root-ca.crt.txt 2026-06-10 01:16:10.158190 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--tcp-services.txt 2026-06-10 01:16:10.158276 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--udp-services.txt 2026-06-10 01:16:10.158348 | controller | >f+++++++++ items-describe/ingress-nginx--deployments--ingress-nginx-controller.txt 2026-06-10 01:16:10.158411 | controller | >f+++++++++ items-describe/ingress-nginx--jobs--ingress-nginx-admission-create.txt 2026-06-10 01:16:10.158471 | controller | >f+++++++++ items-describe/ingress-nginx--jobs--ingress-nginx-admission-patch.txt 2026-06-10 01:16:10.158530 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-admission-create-gqrrr.txt 2026-06-10 01:16:10.158643 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-admission-patch-ftshk.txt 2026-06-10 01:16:10.158716 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-controller-86649c845b-wnd44.txt 2026-06-10 01:16:10.158779 | controller | >f+++++++++ items-describe/ingress-nginx--secrets--ingress-nginx-admission.txt 2026-06-10 01:16:10.158844 | controller | >f+++++++++ items-describe/ingress-nginx--services--ingress-nginx-controller-admission.txt 2026-06-10 01:16:10.158913 | controller | >f+++++++++ items-describe/ingress-nginx--services--ingress-nginx-controller.txt 2026-06-10 01:16:10.158978 | controller | >f+++++++++ items-describe/kube-node-lease--configmaps--kube-root-ca.crt.txt 2026-06-10 01:16:10.159049 | controller | >f+++++++++ items-describe/kube-public--configmaps--cluster-info.txt 2026-06-10 01:16:10.159121 | controller | >f+++++++++ items-describe/kube-public--configmaps--kube-root-ca.crt.txt 2026-06-10 01:16:10.159193 | controller | >f+++++++++ items-describe/kube-system--configmaps--coredns.txt 2026-06-10 01:16:10.159264 | controller | >f+++++++++ items-describe/kube-system--configmaps--extension-apiserver-authentication.txt 2026-06-10 01:16:10.159335 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-apiserver-legacy-service-account-token-tracking.txt 2026-06-10 01:16:10.159425 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-proxy.txt 2026-06-10 01:16:10.159507 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-root-ca.crt.txt 2026-06-10 01:16:10.159625 | controller | >f+++++++++ items-describe/kube-system--configmaps--kubeadm-config.txt 2026-06-10 01:16:10.159731 | controller | >f+++++++++ items-describe/kube-system--configmaps--kubelet-config.txt 2026-06-10 01:16:10.159818 | controller | >f+++++++++ items-describe/kube-system--deployments--coredns.txt 2026-06-10 01:16:10.159905 | controller | >f+++++++++ items-describe/kube-system--pods--coredns-668d6bf9bc-5m5hh.txt 2026-06-10 01:16:10.159986 | controller | >f+++++++++ items-describe/kube-system--pods--etcd-minikube.txt 2026-06-10 01:16:10.160061 | controller | >f+++++++++ items-describe/kube-system--pods--kindnet-twq7n.txt 2026-06-10 01:16:10.160133 | controller | >f+++++++++ items-describe/kube-system--pods--kube-apiserver-minikube.txt 2026-06-10 01:16:10.160205 | controller | >f+++++++++ items-describe/kube-system--pods--kube-controller-manager-minikube.txt 2026-06-10 01:16:10.160276 | controller | >f+++++++++ items-describe/kube-system--pods--kube-ingress-dns-minikube.txt 2026-06-10 01:16:10.160346 | controller | >f+++++++++ items-describe/kube-system--pods--kube-proxy-92vnl.txt 2026-06-10 01:16:10.160415 | controller | >f+++++++++ items-describe/kube-system--pods--kube-scheduler-minikube.txt 2026-06-10 01:16:10.160492 | controller | >f+++++++++ items-describe/kube-system--pods--storage-provisioner.txt 2026-06-10 01:16:10.160619 | controller | >f+++++++++ items-describe/kube-system--secrets--bootstrap-token-3qyetz.txt 2026-06-10 01:16:10.160819 | controller | >f+++++++++ items-describe/kube-system--services--kube-dns.txt 2026-06-10 01:16:10.160910 | controller | >f+++++++++ items-describe/nodepool--configmaps--kube-root-ca.crt.txt 2026-06-10 01:16:10.160981 | controller | >f+++++++++ items-describe/nodepool--secrets--nodepool-token.txt 2026-06-10 01:16:10.161052 | controller | >f+++++++++ items-describe/sf--configmaps--gateway-config-map.txt 2026-06-10 01:16:10.161135 | controller | >f+++++++++ items-describe/sf--configmaps--git-server-pi-config-map.txt 2026-06-10 01:16:10.161199 | controller | >f+++++++++ items-describe/sf--configmaps--kube-root-ca.crt.txt 2026-06-10 01:16:10.161259 | controller | >f+++++++++ items-describe/sf--configmaps--logserver-config-map.txt 2026-06-10 01:16:10.161317 | controller | >f+++++++++ items-describe/sf--configmaps--loki-config.txt 2026-06-10 01:16:10.161373 | controller | >f+++++++++ items-describe/sf--configmaps--managesf-resources-config-map.txt 2026-06-10 01:16:10.161431 | controller | >f+++++++++ items-describe/sf--configmaps--managesf-resources-tooling-config-map.txt 2026-06-10 01:16:10.161493 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-builder-extra-config-config-map.txt 2026-06-10 01:16:10.161561 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-launcher-extra-config-config-map.txt 2026-06-10 01:16:10.161700 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-tooling-config-map.txt 2026-06-10 01:16:10.161776 | controller | >f+++++++++ items-describe/sf--configmaps--np-statsd-config-map.txt 2026-06-10 01:16:10.161997 | controller | >f+++++++++ items-describe/sf--configmaps--pymod-sf-operator-config-map.txt 2026-06-10 01:16:10.162057 | controller | >f+++++++++ items-describe/sf--configmaps--sf-standalone-owner.txt 2026-06-10 01:16:10.162100 | controller | >f+++++++++ items-describe/sf--configmaps--test-fluentbit-config.txt 2026-06-10 01:16:10.162137 | controller | >f+++++++++ items-describe/sf--configmaps--zookeeper-pi-config-map.txt 2026-06-10 01:16:10.162174 | controller | >f+++++++++ items-describe/sf--configmaps--zs-internal-tenant-reconfigure.txt 2026-06-10 01:16:10.162210 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-extra-config-map.txt 2026-06-10 01:16:10.162245 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-logging-config-map.txt 2026-06-10 01:16:10.162281 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-scheduler-tooling-config-map.txt 2026-06-10 01:16:10.162317 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-statsd-config-map.txt 2026-06-10 01:16:10.162351 | controller | >f+++++++++ items-describe/sf--deployments--gateway.txt 2026-06-10 01:16:10.162386 | controller | >f+++++++++ items-describe/sf--deployments--logjuicer.txt 2026-06-10 01:16:10.162422 | controller | >f+++++++++ items-describe/sf--deployments--nodepool-launcher.txt 2026-06-10 01:16:10.162457 | controller | >f+++++++++ items-describe/sf--deployments--test-loki.txt 2026-06-10 01:16:10.162493 | controller | >f+++++++++ items-describe/sf--deployments--zuul-web.txt 2026-06-10 01:16:10.162529 | controller | >f+++++++++ items-describe/sf--deployments--zuul-weeder.txt 2026-06-10 01:16:10.162565 | controller | >f+++++++++ items-describe/sf--ingress--gerrit-ingress.txt 2026-06-10 01:16:10.162656 | controller | >f+++++++++ items-describe/sf--ingress--sf-ingress.txt 2026-06-10 01:16:10.162697 | controller | >f+++++++++ items-describe/sf--jobs--config-base-secret-1-8715-ns.txt 2026-06-10 01:16:10.162734 | controller | >f+++++++++ items-describe/sf--jobs--post-init.txt 2026-06-10 01:16:10.162769 | controller | >f+++++++++ items-describe/sf--jobs--zuul-db-provision.txt 2026-06-10 01:16:10.162805 | controller | >f+++++++++ items-describe/sf--pods--config-base-secret-1-8715-ns-v267j.txt 2026-06-10 01:16:10.162841 | controller | >f+++++++++ items-describe/sf--pods--config-base-secret-1-945-ns-85xbq.txt 2026-06-10 01:16:10.162876 | controller | >f+++++++++ items-describe/sf--pods--gateway-6bbbd758d9-fhwt6.txt 2026-06-10 01:16:10.162911 | controller | >f+++++++++ items-describe/sf--pods--gerrit-0.txt 2026-06-10 01:16:10.162945 | controller | >f+++++++++ items-describe/sf--pods--git-server-0.txt 2026-06-10 01:16:10.162979 | controller | >f+++++++++ items-describe/sf--pods--hound-search-0.txt 2026-06-10 01:16:10.163013 | controller | >f+++++++++ items-describe/sf--pods--logjuicer-6ff84764b7-z8rn9.txt 2026-06-10 01:16:10.163047 | controller | >f+++++++++ items-describe/sf--pods--logserver-0.txt 2026-06-10 01:16:10.163093 | controller | >f+++++++++ items-describe/sf--pods--mariadb-0.txt 2026-06-10 01:16:10.163129 | controller | >f+++++++++ items-describe/sf--pods--nodepool-builder-0.txt 2026-06-10 01:16:10.163164 | controller | >f+++++++++ items-describe/sf--pods--nodepool-launcher-6cffc7496d-bmpgw.txt 2026-06-10 01:16:10.163200 | controller | >f+++++++++ items-describe/sf--pods--post-init-wjpvj.txt 2026-06-10 01:16:10.163235 | controller | >f+++++++++ items-describe/sf--pods--test-fluentbit.txt 2026-06-10 01:16:10.163270 | controller | >f+++++++++ items-describe/sf--pods--test-loki-8cd4fd56-sbxfw.txt 2026-06-10 01:16:10.163305 | controller | >f+++++++++ items-describe/sf--pods--zookeeper-0.txt 2026-06-10 01:16:10.163340 | controller | >f+++++++++ items-describe/sf--pods--zuul-db-provision-52vwg.txt 2026-06-10 01:16:10.163375 | controller | >f+++++++++ items-describe/sf--pods--zuul-db-provision-m98vf.txt 2026-06-10 01:16:10.163410 | controller | >f+++++++++ items-describe/sf--pods--zuul-executor-0.txt 2026-06-10 01:16:10.163444 | controller | >f+++++++++ items-describe/sf--pods--zuul-merger-0.txt 2026-06-10 01:16:10.163479 | controller | >f+++++++++ items-describe/sf--pods--zuul-scheduler-0.txt 2026-06-10 01:16:10.163514 | controller | >f+++++++++ items-describe/sf--pods--zuul-web-8445ff8bc6-jp9hd.txt 2026-06-10 01:16:10.163548 | controller | >f+++++++++ items-describe/sf--pods--zuul-weeder-57c6567f5f-glkrq.txt 2026-06-10 01:16:10.163619 | controller | >f+++++++++ items-describe/sf--pvc--gerrit-gerrit-0.txt 2026-06-10 01:16:10.163659 | controller | >f+++++++++ items-describe/sf--pvc--git-server-git-server-0.txt 2026-06-10 01:16:10.163694 | controller | >f+++++++++ items-describe/sf--pvc--hound-search-data-hound-search-0.txt 2026-06-10 01:16:10.163730 | controller | >f+++++++++ items-describe/sf--pvc--logjuicer-pvc.txt 2026-06-10 01:16:10.163764 | controller | >f+++++++++ items-describe/sf--pvc--logserver-logserver-0.txt 2026-06-10 01:16:10.163799 | controller | >f+++++++++ items-describe/sf--pvc--mariadb-logs-mariadb-0.txt 2026-06-10 01:16:10.163833 | controller | >f+++++++++ items-describe/sf--pvc--mariadb-mariadb-0.txt 2026-06-10 01:16:10.163868 | controller | >f+++++++++ items-describe/sf--pvc--nodepool-builder-nodepool-builder-0.txt 2026-06-10 01:16:10.163903 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-data-zookeeper-0.txt 2026-06-10 01:16:10.163938 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-logs-zookeeper-0.txt 2026-06-10 01:16:10.163974 | controller | >f+++++++++ items-describe/sf--pvc--zuul-executor-zuul-executor-0.txt 2026-06-10 01:16:10.164008 | controller | >f+++++++++ items-describe/sf--pvc--zuul-merger-zuul-merger-0.txt 2026-06-10 01:16:10.164043 | controller | >f+++++++++ items-describe/sf--pvc--zuul-scheduler-zuul-scheduler-0.txt 2026-06-10 01:16:10.164078 | controller | >f+++++++++ items-describe/sf--secrets--admin-ssh-key.txt 2026-06-10 01:16:10.164112 | controller | >f+++++++++ items-describe/sf--secrets--config-update-secrets.txt 2026-06-10 01:16:10.164147 | controller | >f+++++++++ items-describe/sf--secrets--dummygerritsecret.txt 2026-06-10 01:16:10.164182 | controller | >f+++++++++ items-describe/sf--secrets--es-basicauth.txt 2026-06-10 01:16:10.164217 | controller | >f+++++++++ items-describe/sf--secrets--gerrit-admin-api-key.txt 2026-06-10 01:16:10.164251 | controller | >f+++++++++ items-describe/sf--secrets--logserver-keys.txt 2026-06-10 01:16:10.164287 | controller | >f+++++++++ items-describe/sf--secrets--logserver-uploader-keys.txt 2026-06-10 01:16:10.164322 | controller | >f+++++++++ items-describe/sf--secrets--logserver-uploader-spare-keys.txt 2026-06-10 01:16:10.164358 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-config-secrets.txt 2026-06-10 01:16:10.164393 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-initdb-secrets.txt 2026-06-10 01:16:10.164429 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-root-password.txt 2026-06-10 01:16:10.164465 | controller | >f+++++++++ items-describe/sf--secrets--nodepool-builder-ssh-key.txt 2026-06-10 01:16:10.164512 | controller | >f+++++++++ items-describe/sf--secrets--nodepool-providers-secrets.txt 2026-06-10 01:16:10.164549 | controller | >f+++++++++ items-describe/sf--secrets--self-signed-cert.txt 2026-06-10 01:16:10.164619 | controller | >f+++++++++ items-describe/sf--secrets--smtp-secret.txt 2026-06-10 01:16:10.164659 | controller | >f+++++++++ items-describe/sf--secrets--zookeeper-client-tls.txt 2026-06-10 01:16:10.164695 | controller | >f+++++++++ items-describe/sf--secrets--zookeeper-server-tls.txt 2026-06-10 01:16:10.164731 | controller | >f+++++++++ items-describe/sf--secrets--zuul-auth-secret.txt 2026-06-10 01:16:10.164765 | controller | >f+++++++++ items-describe/sf--secrets--zuul-config.txt 2026-06-10 01:16:10.164800 | controller | >f+++++++++ items-describe/sf--secrets--zuul-db-connection.txt 2026-06-10 01:16:10.164834 | controller | >f+++++++++ items-describe/sf--secrets--zuul-gerrit-api-key.txt 2026-06-10 01:16:10.164870 | controller | >f+++++++++ items-describe/sf--secrets--zuul-keystore-password.txt 2026-06-10 01:16:10.164905 | controller | >f+++++++++ items-describe/sf--secrets--zuul-spare-ssh-key.txt 2026-06-10 01:16:10.164939 | controller | >f+++++++++ items-describe/sf--secrets--zuul-ssh-key.txt 2026-06-10 01:16:10.164974 | controller | >f+++++++++ items-describe/sf--services--fluent-bit-forward-input.txt 2026-06-10 01:16:10.165009 | controller | >f+++++++++ items-describe/sf--services--gateway.txt 2026-06-10 01:16:10.165044 | controller | >f+++++++++ items-describe/sf--services--gerrit-httpd.txt 2026-06-10 01:16:10.165080 | controller | >f+++++++++ items-describe/sf--services--gerrit-sshd.txt 2026-06-10 01:16:10.165115 | controller | >f+++++++++ items-describe/sf--services--git-server-rw.txt 2026-06-10 01:16:10.165149 | controller | >f+++++++++ items-describe/sf--services--git-server.txt 2026-06-10 01:16:10.165184 | controller | >f+++++++++ items-describe/sf--services--hound-search.txt 2026-06-10 01:16:10.165219 | controller | >f+++++++++ items-describe/sf--services--logjuicer.txt 2026-06-10 01:16:10.165254 | controller | >f+++++++++ items-describe/sf--services--logserver.txt 2026-06-10 01:16:10.165289 | controller | >f+++++++++ items-describe/sf--services--loki-http.txt 2026-06-10 01:16:10.165324 | controller | >f+++++++++ items-describe/sf--services--mariadb.txt 2026-06-10 01:16:10.165358 | controller | >f+++++++++ items-describe/sf--services--nodepool-builder.txt 2026-06-10 01:16:10.165394 | controller | >f+++++++++ items-describe/sf--services--nodepool-launcher.txt 2026-06-10 01:16:10.165428 | controller | >f+++++++++ items-describe/sf--services--zookeeper-headless.txt 2026-06-10 01:16:10.165464 | controller | >f+++++++++ items-describe/sf--services--zookeeper.txt 2026-06-10 01:16:10.165499 | controller | >f+++++++++ items-describe/sf--services--zuul-capacity.txt 2026-06-10 01:16:10.165533 | controller | >f+++++++++ items-describe/sf--services--zuul-executor-headless.txt 2026-06-10 01:16:10.165568 | controller | >f+++++++++ items-describe/sf--services--zuul-web.txt 2026-06-10 01:16:10.165644 | controller | >f+++++++++ items-describe/sf--services--zuul-weeder.txt 2026-06-10 01:16:10.165682 | controller | >f+++++++++ items-describe/sf--statefulsets--gerrit.txt 2026-06-10 01:16:10.165718 | controller | >f+++++++++ items-describe/sf--statefulsets--git-server.txt 2026-06-10 01:16:10.165753 | controller | >f+++++++++ items-describe/sf--statefulsets--hound-search.txt 2026-06-10 01:16:10.165815 | controller | >f+++++++++ items-describe/sf--statefulsets--logserver.txt 2026-06-10 01:16:10.165855 | controller | >f+++++++++ items-describe/sf--statefulsets--mariadb.txt 2026-06-10 01:16:10.165891 | controller | >f+++++++++ items-describe/sf--statefulsets--nodepool-builder.txt 2026-06-10 01:16:10.165927 | controller | >f+++++++++ items-describe/sf--statefulsets--zookeeper.txt 2026-06-10 01:16:10.165961 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-executor.txt 2026-06-10 01:16:10.165996 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-merger.txt 2026-06-10 01:16:10.166042 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-scheduler.txt 2026-06-10 01:16:10.166078 | controller | cd+++++++++ k8s-resources/ 2026-06-10 01:16:10.166113 | controller | cd+++++++++ k8s-resources/sf/ 2026-06-10 01:16:10.166149 | controller | cd+++++++++ k8s-resources/sf/pod/ 2026-06-10 01:16:10.166184 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-executor-0-zuul-executor-logs0.txt 2026-06-10 01:16:10.166219 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-merger-0-zuul-merger-logs0.txt 2026-06-10 01:16:10.166255 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-scheduler-0-zuul-scheduler-logs0.txt 2026-06-10 01:16:10.166291 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-web-0-zuul-web-logs0.txt 2026-06-10 01:16:10.166326 | controller | cd+++++++++ logserver-content/ 2026-06-10 01:16:10.166360 | controller | cd+++++++++ logserver-content/a16/ 2026-06-10 01:16:10.166395 | controller | cd+++++++++ logserver-content/a16/internal/ 2026-06-10 01:16:10.166429 | controller | cd+++++++++ logserver-content/a16/internal/a16399828161487babbe9eb882fc1679/ 2026-06-10 01:16:10.166463 | controller | >f+++++++++ logserver-content/a16/internal/a16399828161487babbe9eb882fc1679/job-output.json.gz 2026-06-10 01:16:10.166497 | controller | >f+++++++++ logserver-content/a16/internal/a16399828161487babbe9eb882fc1679/job-output.txt.gz 2026-06-10 01:16:10.166531 | controller | >f+++++++++ logserver-content/a16/internal/a16399828161487babbe9eb882fc1679/workspace-repos.json 2026-06-10 01:16:10.166565 | controller | >f+++++++++ logserver-content/a16/internal/a16399828161487babbe9eb882fc1679/zuul-manifest.json 2026-06-10 01:16:10.166638 | controller | cd+++++++++ logserver-content/a16/internal/a16399828161487babbe9eb882fc1679/zuul-info/ 2026-06-10 01:16:10.166676 | controller | >f+++++++++ logserver-content/a16/internal/a16399828161487babbe9eb882fc1679/zuul-info/inventory.yaml 2026-06-10 01:16:10.166710 | controller | cd+++++++++ logserver-content/d9f/ 2026-06-10 01:16:10.166743 | controller | cd+++++++++ logserver-content/d9f/internal/ 2026-06-10 01:16:10.166777 | controller | cd+++++++++ logserver-content/d9f/internal/d9f26b3a6fa84ef3833eea4fed54eae1/ 2026-06-10 01:16:10.166811 | controller | >f+++++++++ logserver-content/d9f/internal/d9f26b3a6fa84ef3833eea4fed54eae1/job-output.json.gz 2026-06-10 01:16:10.166845 | controller | >f+++++++++ logserver-content/d9f/internal/d9f26b3a6fa84ef3833eea4fed54eae1/job-output.txt.gz 2026-06-10 01:16:10.166879 | controller | >f+++++++++ logserver-content/d9f/internal/d9f26b3a6fa84ef3833eea4fed54eae1/workspace-repos.json 2026-06-10 01:16:10.166913 | controller | >f+++++++++ logserver-content/d9f/internal/d9f26b3a6fa84ef3833eea4fed54eae1/zuul-manifest.json 2026-06-10 01:16:10.166946 | controller | cd+++++++++ logserver-content/d9f/internal/d9f26b3a6fa84ef3833eea4fed54eae1/zuul-info/ 2026-06-10 01:16:10.166979 | controller | >f+++++++++ logserver-content/d9f/internal/d9f26b3a6fa84ef3833eea4fed54eae1/zuul-info/inventory.yaml 2026-06-10 01:16:10.167013 | controller | cd+++++++++ logserver-content/ef1/ 2026-06-10 01:16:10.167046 | controller | cd+++++++++ logserver-content/ef1/internal/ 2026-06-10 01:16:10.167080 | controller | cd+++++++++ logserver-content/ef1/internal/ef11b34b0a474e718baebb74ee97fd04/ 2026-06-10 01:16:10.167114 | controller | >f+++++++++ logserver-content/ef1/internal/ef11b34b0a474e718baebb74ee97fd04/job-output.json.gz 2026-06-10 01:16:10.167147 | controller | >f+++++++++ logserver-content/ef1/internal/ef11b34b0a474e718baebb74ee97fd04/job-output.txt.gz 2026-06-10 01:16:10.167180 | controller | >f+++++++++ logserver-content/ef1/internal/ef11b34b0a474e718baebb74ee97fd04/workspace-repos.json 2026-06-10 01:16:10.167213 | controller | >f+++++++++ logserver-content/ef1/internal/ef11b34b0a474e718baebb74ee97fd04/zuul-manifest.json 2026-06-10 01:16:10.167246 | controller | cd+++++++++ logserver-content/ef1/internal/ef11b34b0a474e718baebb74ee97fd04/zuul-info/ 2026-06-10 01:16:10.167290 | controller | >f+++++++++ logserver-content/ef1/internal/ef11b34b0a474e718baebb74ee97fd04/zuul-info/inventory.yaml 2026-06-10 01:16:10.167329 | controller | cd+++++++++ loki-logs/ 2026-06-10 01:16:10.167363 | controller | >f+++++++++ loki-logs/all.log 2026-06-10 01:16:10.167398 | controller | cd+++++++++ pods-logs/ 2026-06-10 01:16:10.167432 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-admission-create-gqrrr--create.txt 2026-06-10 01:16:10.167465 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-admission-patch-ftshk--patch.txt 2026-06-10 01:16:10.167500 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-controller-86649c845b-wnd44--controller.txt 2026-06-10 01:16:10.167534 | controller | >f+++++++++ pods-logs/kube-system--coredns-668d6bf9bc-5m5hh--coredns.txt 2026-06-10 01:16:10.167569 | controller | >f+++++++++ pods-logs/kube-system--etcd-minikube--etcd.txt 2026-06-10 01:16:10.167643 | controller | >f+++++++++ pods-logs/kube-system--kindnet-twq7n--kindnet-cni.txt 2026-06-10 01:16:10.167680 | controller | >f+++++++++ pods-logs/kube-system--kube-apiserver-minikube--kube-apiserver.txt 2026-06-10 01:16:10.167715 | controller | >f+++++++++ pods-logs/kube-system--kube-controller-manager-minikube--kube-controller-manager.txt 2026-06-10 01:16:10.167749 | controller | >f+++++++++ pods-logs/kube-system--kube-ingress-dns-minikube--minikube-ingress-dns.txt 2026-06-10 01:16:10.167783 | controller | >f+++++++++ pods-logs/kube-system--kube-proxy-92vnl--kube-proxy.txt 2026-06-10 01:16:10.167816 | controller | >f+++++++++ pods-logs/kube-system--kube-scheduler-minikube--kube-scheduler.txt 2026-06-10 01:16:10.167849 | controller | >f+++++++++ pods-logs/kube-system--storage-provisioner--storage-provisioner.txt 2026-06-10 01:16:10.167882 | controller | >f+++++++++ pods-logs/sf--config-base-secret-1-8715-ns-v267j--sf-operator.txt 2026-06-10 01:16:10.167915 | controller | >f+++++++++ pods-logs/sf--config-base-secret-1-945-ns-85xbq--sf-operator.txt 2026-06-10 01:16:10.167948 | controller | >f+++++++++ pods-logs/sf--gateway-6bbbd758d9-fhwt6--gateway.txt 2026-06-10 01:16:10.167981 | controller | >f+++++++++ pods-logs/sf--gerrit-0--gerrit.txt 2026-06-10 01:16:10.168014 | controller | >f+++++++++ pods-logs/sf--gerrit-0--managesf-resources.txt 2026-06-10 01:16:10.168048 | controller | >f+++++++++ pods-logs/sf--git-server-0--git-server-rw.txt 2026-06-10 01:16:10.168081 | controller | >f+++++++++ pods-logs/sf--git-server-0--git-server.txt 2026-06-10 01:16:10.168114 | controller | >f+++++++++ pods-logs/sf--hound-search-0--hound-search.txt 2026-06-10 01:16:10.168148 | controller | >f+++++++++ pods-logs/sf--logjuicer-6ff84764b7-z8rn9--logjuicer.txt 2026-06-10 01:16:10.168183 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver-nodeexporter.txt 2026-06-10 01:16:10.168217 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver-sshd.txt 2026-06-10 01:16:10.168250 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver.txt 2026-06-10 01:16:10.168283 | controller | >f+++++++++ pods-logs/sf--logserver-0--purgelogs.txt 2026-06-10 01:16:10.168315 | controller | >f+++++++++ pods-logs/sf--mariadb-0--mariadb-nodeexporter.txt 2026-06-10 01:16:10.168348 | controller | >f+++++++++ pods-logs/sf--mariadb-0--mariadb.txt 2026-06-10 01:16:10.168381 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--build-logs-httpd.txt 2026-06-10 01:16:10.168414 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--nodepool-builder-nodeexporter.txt 2026-06-10 01:16:10.168448 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--nodepool-builder.txt 2026-06-10 01:16:10.168481 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--np-statsd.txt 2026-06-10 01:16:10.168514 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-6cffc7496d-bmpgw--launcher.txt 2026-06-10 01:16:10.168547 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-6cffc7496d-bmpgw--np-statsd.txt 2026-06-10 01:16:10.168609 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-6cffc7496d-bmpgw--zuul-capacity.txt 2026-06-10 01:16:10.168662 | controller | >f+++++++++ pods-logs/sf--post-init-wjpvj--post-init-container.txt 2026-06-10 01:16:10.168698 | controller | >f+++++++++ pods-logs/sf--test-fluentbit--test-fluentbit.txt 2026-06-10 01:16:10.168732 | controller | >f+++++++++ pods-logs/sf--test-loki-8cd4fd56-sbxfw--loki.txt 2026-06-10 01:16:10.168765 | controller | >f+++++++++ pods-logs/sf--zookeeper-0--zookeeper-nodeexporter.txt 2026-06-10 01:16:10.168798 | controller | >f+++++++++ pods-logs/sf--zookeeper-0--zookeeper.txt 2026-06-10 01:16:10.168832 | controller | >f+++++++++ pods-logs/sf--zuul-db-provision-52vwg--mariadb-client.txt 2026-06-10 01:16:10.168865 | controller | >f+++++++++ pods-logs/sf--zuul-db-provision-m98vf--mariadb-client.txt 2026-06-10 01:16:10.168899 | controller | >f+++++++++ pods-logs/sf--zuul-executor-0--zuul-executor-nodeexporter.txt 2026-06-10 01:16:10.168934 | controller | >f+++++++++ pods-logs/sf--zuul-executor-0--zuul-executor.txt 2026-06-10 01:16:10.168967 | controller | >f+++++++++ pods-logs/sf--zuul-merger-0--zuul-merger-nodeexporter.txt 2026-06-10 01:16:10.169001 | controller | >f+++++++++ pods-logs/sf--zuul-merger-0--zuul-merger.txt 2026-06-10 01:16:10.169035 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-scheduler-nodeexporter.txt 2026-06-10 01:16:10.169069 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-scheduler.txt 2026-06-10 01:16:10.169103 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-statsd.txt 2026-06-10 01:16:10.169137 | controller | >f+++++++++ pods-logs/sf--zuul-web-8445ff8bc6-jp9hd--zuul-web.txt 2026-06-10 01:16:10.169170 | controller | >f+++++++++ pods-logs/sf--zuul-weeder-57c6567f5f-glkrq--zuul-weeder.txt 2026-06-10 01:16:10.169203 | controller | cd+++++++++ repos/ 2026-06-10 01:16:10.169237 | controller | >f+++++++++ repos/test-ssh-key 2026-06-10 01:16:10.169272 | controller | >f+++++++++ repos/test-ssh-key.pub 2026-06-10 01:16:10.169305 | controller | >f+++++++++ repos/zuul-ssh-key 2026-06-10 01:16:10.169339 | controller | cd+++++++++ repos/config/ 2026-06-10 01:16:10.169372 | controller | >f+++++++++ repos/config/.gitreview 2026-06-10 01:16:10.169405 | controller | cd+++++++++ repos/config/.git/ 2026-06-10 01:16:10.169438 | controller | >f+++++++++ repos/config/.git/COMMIT_EDITMSG 2026-06-10 01:16:10.169471 | controller | >f+++++++++ repos/config/.git/FETCH_HEAD 2026-06-10 01:16:10.169505 | controller | >f+++++++++ repos/config/.git/HEAD 2026-06-10 01:16:10.169537 | controller | >f+++++++++ repos/config/.git/ORIG_HEAD 2026-06-10 01:16:10.169570 | controller | >f+++++++++ repos/config/.git/config 2026-06-10 01:16:10.169639 | controller | >f+++++++++ repos/config/.git/description 2026-06-10 01:16:10.169675 | controller | >f+++++++++ repos/config/.git/index 2026-06-10 01:16:10.169707 | controller | >f+++++++++ repos/config/.git/packed-refs 2026-06-10 01:16:10.169740 | controller | cd+++++++++ repos/config/.git/hooks/ 2026-06-10 01:16:10.169772 | controller | >f+++++++++ repos/config/.git/hooks/applypatch-msg.sample 2026-06-10 01:16:10.169825 | controller | >f+++++++++ repos/config/.git/hooks/commit-msg 2026-06-10 01:16:10.169860 | controller | >f+++++++++ repos/config/.git/hooks/commit-msg.sample 2026-06-10 01:16:10.169894 | controller | >f+++++++++ repos/config/.git/hooks/fsmonitor-watchman.sample 2026-06-10 01:16:10.169927 | controller | >f+++++++++ repos/config/.git/hooks/post-update.sample 2026-06-10 01:16:10.169960 | controller | >f+++++++++ repos/config/.git/hooks/pre-applypatch.sample 2026-06-10 01:16:10.169993 | controller | >f+++++++++ repos/config/.git/hooks/pre-commit.sample 2026-06-10 01:16:10.170026 | controller | >f+++++++++ repos/config/.git/hooks/pre-merge-commit.sample 2026-06-10 01:16:10.170059 | controller | >f+++++++++ repos/config/.git/hooks/pre-push.sample 2026-06-10 01:16:10.170093 | controller | >f+++++++++ repos/config/.git/hooks/pre-rebase.sample 2026-06-10 01:16:10.170126 | controller | >f+++++++++ repos/config/.git/hooks/pre-receive.sample 2026-06-10 01:16:10.170158 | controller | >f+++++++++ repos/config/.git/hooks/prepare-commit-msg.sample 2026-06-10 01:16:10.170199 | controller | >f+++++++++ repos/config/.git/hooks/push-to-checkout.sample 2026-06-10 01:16:10.170234 | controller | >f+++++++++ repos/config/.git/hooks/sendemail-validate.sample 2026-06-10 01:16:10.170266 | controller | >f+++++++++ repos/config/.git/hooks/update.sample 2026-06-10 01:16:10.170299 | controller | cd+++++++++ repos/config/.git/info/ 2026-06-10 01:16:10.170331 | controller | >f+++++++++ repos/config/.git/info/exclude 2026-06-10 01:16:10.170364 | controller | cd+++++++++ repos/config/.git/logs/ 2026-06-10 01:16:10.170398 | controller | >f+++++++++ repos/config/.git/logs/HEAD 2026-06-10 01:16:10.170430 | controller | cd+++++++++ repos/config/.git/logs/refs/ 2026-06-10 01:16:10.170464 | controller | cd+++++++++ repos/config/.git/logs/refs/heads/ 2026-06-10 01:16:10.170497 | controller | >f+++++++++ repos/config/.git/logs/refs/heads/master 2026-06-10 01:16:10.170530 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/ 2026-06-10 01:16:10.170563 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/gerrit/ 2026-06-10 01:16:10.170635 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/gerrit/HEAD 2026-06-10 01:16:10.170672 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/gerrit/master 2026-06-10 01:16:10.170705 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/origin/ 2026-06-10 01:16:10.170738 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/origin/HEAD 2026-06-10 01:16:10.170772 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/origin/master 2026-06-10 01:16:10.170805 | controller | cd+++++++++ repos/config/.git/objects/ 2026-06-10 01:16:10.170839 | controller | cd+++++++++ repos/config/.git/objects/01/ 2026-06-10 01:16:10.170938 | controller | >f+++++++++ repos/config/.git/objects/01/c21257bd9dfb4afa3d88589353f39231438948 2026-06-10 01:16:10.170983 | controller | cd+++++++++ repos/config/.git/objects/06/ 2026-06-10 01:16:10.171017 | controller | >f+++++++++ repos/config/.git/objects/06/5460e3e4fba2934d333152681c02f5bafe09c9 2026-06-10 01:16:10.171052 | controller | cd+++++++++ repos/config/.git/objects/35/ 2026-06-10 01:16:10.171086 | controller | >f+++++++++ repos/config/.git/objects/35/5afe5148178206dc7d0f3ac3dbce99057f5c51 2026-06-10 01:16:10.171119 | controller | >f+++++++++ repos/config/.git/objects/35/a1b88bc0691f57e70f6b5a851197f2ec48a1d3 2026-06-10 01:16:10.171153 | controller | cd+++++++++ repos/config/.git/objects/38/ 2026-06-10 01:16:10.171187 | controller | >f+++++++++ repos/config/.git/objects/38/5c83c9e303c30a6517d9c47d074b7da488f620 2026-06-10 01:16:10.171221 | controller | cd+++++++++ repos/config/.git/objects/48/ 2026-06-10 01:16:10.171255 | controller | >f+++++++++ repos/config/.git/objects/48/88d4b0cd22e76b63f86f13f5302d2abad81c24 2026-06-10 01:16:10.171289 | controller | cd+++++++++ repos/config/.git/objects/4a/ 2026-06-10 01:16:10.171322 | controller | >f+++++++++ repos/config/.git/objects/4a/62e97789919fe106203e16bec87d8c240a3fa2 2026-06-10 01:16:10.171356 | controller | cd+++++++++ repos/config/.git/objects/4f/ 2026-06-10 01:16:10.171390 | controller | >f+++++++++ repos/config/.git/objects/4f/0de5c58658d7121b6541e017d2244e5ff70f06 2026-06-10 01:16:10.171423 | controller | cd+++++++++ repos/config/.git/objects/55/ 2026-06-10 01:16:10.171456 | controller | >f+++++++++ repos/config/.git/objects/55/58a488456a09ccc2debde4fb319cc8701749b3 2026-06-10 01:16:10.171489 | controller | cd+++++++++ repos/config/.git/objects/6a/ 2026-06-10 01:16:10.171523 | controller | >f+++++++++ repos/config/.git/objects/6a/906645df0c95e3c0f28d78f55392f0c0ef8c10 2026-06-10 01:16:10.171557 | controller | cd+++++++++ repos/config/.git/objects/6d/ 2026-06-10 01:16:10.171647 | controller | >f+++++++++ repos/config/.git/objects/6d/9f13b3be7af02d0fd7c94187a0c6a0aa996489 2026-06-10 01:16:10.171687 | controller | cd+++++++++ repos/config/.git/objects/75/ 2026-06-10 01:16:10.171721 | controller | >f+++++++++ repos/config/.git/objects/75/026a20fdbeb3f69d8d51c8357b94cab9b33ae7 2026-06-10 01:16:10.171766 | controller | cd+++++++++ repos/config/.git/objects/78/ 2026-06-10 01:16:10.171802 | controller | >f+++++++++ repos/config/.git/objects/78/0a8faef73e8eeb346f19b926fa78364f27e082 2026-06-10 01:16:10.171836 | controller | cd+++++++++ repos/config/.git/objects/81/ 2026-06-10 01:16:10.171869 | controller | >f+++++++++ repos/config/.git/objects/81/320c4786550199ff699a83539dd7c1f0a1e8af 2026-06-10 01:16:10.171903 | controller | cd+++++++++ repos/config/.git/objects/89/ 2026-06-10 01:16:10.171937 | controller | >f+++++++++ repos/config/.git/objects/89/acef0650c7adef430cd0de35ea18b179affaa6 2026-06-10 01:16:10.171971 | controller | cd+++++++++ repos/config/.git/objects/8e/ 2026-06-10 01:16:10.172004 | controller | >f+++++++++ repos/config/.git/objects/8e/068bef39faf6d6756c872a565548c6307c28c3 2026-06-10 01:16:10.172038 | controller | cd+++++++++ repos/config/.git/objects/91/ 2026-06-10 01:16:10.172072 | controller | >f+++++++++ repos/config/.git/objects/91/74b11542bb50ba3526aa696c49bde326c85f33 2026-06-10 01:16:10.172107 | controller | cd+++++++++ repos/config/.git/objects/9c/ 2026-06-10 01:16:10.172140 | controller | >f+++++++++ repos/config/.git/objects/9c/d86e4acdae80b3d3ae1d5f3986010fb1e7a0cf 2026-06-10 01:16:10.172174 | controller | cd+++++++++ repos/config/.git/objects/a0/ 2026-06-10 01:16:10.172207 | controller | >f+++++++++ repos/config/.git/objects/a0/4bf6aaeddb8d742e78b754cf83dab90a6d247b 2026-06-10 01:16:10.172241 | controller | cd+++++++++ repos/config/.git/objects/a2/ 2026-06-10 01:16:10.172274 | controller | >f+++++++++ repos/config/.git/objects/a2/35d5f8afd4665e50073f1e25f9a9dd99d7fe14 2026-06-10 01:16:10.172308 | controller | cd+++++++++ repos/config/.git/objects/a3/ 2026-06-10 01:16:10.172341 | controller | >f+++++++++ repos/config/.git/objects/a3/3bd7322e7353b8f3689ca9f790d3a67f155250 2026-06-10 01:16:10.172374 | controller | cd+++++++++ repos/config/.git/objects/af/ 2026-06-10 01:16:10.172408 | controller | >f+++++++++ repos/config/.git/objects/af/da7ce4b960c2cc78c55c05af10a1336e04e285 2026-06-10 01:16:10.172442 | controller | cd+++++++++ repos/config/.git/objects/b4/ 2026-06-10 01:16:10.172475 | controller | >f+++++++++ repos/config/.git/objects/b4/8c64172c9d244e7dea5a851a55fa075c667c27 2026-06-10 01:16:10.172509 | controller | cd+++++++++ repos/config/.git/objects/ba/ 2026-06-10 01:16:10.172542 | controller | >f+++++++++ repos/config/.git/objects/ba/6052f3dbec6156d05e03561a1c34045537b8bd 2026-06-10 01:16:10.172606 | controller | cd+++++++++ repos/config/.git/objects/c8/ 2026-06-10 01:16:10.172650 | controller | >f+++++++++ repos/config/.git/objects/c8/c413e5c78147040ceac56880fba7c990edf4af 2026-06-10 01:16:10.172686 | controller | cd+++++++++ repos/config/.git/objects/cc/ 2026-06-10 01:16:10.172719 | controller | >f+++++++++ repos/config/.git/objects/cc/6450d887904229c514a2a58f48d4f6231d6d67 2026-06-10 01:16:10.172753 | controller | cd+++++++++ repos/config/.git/objects/d9/ 2026-06-10 01:16:10.172786 | controller | >f+++++++++ repos/config/.git/objects/d9/e1f68f671f696eab26c31a9350ce1601a18a1b 2026-06-10 01:16:10.172818 | controller | cd+++++++++ repos/config/.git/objects/ee/ 2026-06-10 01:16:10.172851 | controller | >f+++++++++ repos/config/.git/objects/ee/0534b3a10626df463c19036b252db7ab804d0b 2026-06-10 01:16:10.172884 | controller | cd+++++++++ repos/config/.git/objects/fd/ 2026-06-10 01:16:10.172917 | controller | >f+++++++++ repos/config/.git/objects/fd/c2b710ab4e21c135a952ddc7fc44d3510caae6 2026-06-10 01:16:10.172950 | controller | cd+++++++++ repos/config/.git/objects/info/ 2026-06-10 01:16:10.172983 | controller | cd+++++++++ repos/config/.git/objects/pack/ 2026-06-10 01:16:10.173017 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-4e98c8a74fa019f8b7d9d0a7a67a743b8bf3bb53.idx 2026-06-10 01:16:10.173051 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-4e98c8a74fa019f8b7d9d0a7a67a743b8bf3bb53.pack 2026-06-10 01:16:10.173085 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-4e98c8a74fa019f8b7d9d0a7a67a743b8bf3bb53.rev 2026-06-10 01:16:10.173128 | controller | cd+++++++++ repos/config/.git/refs/ 2026-06-10 01:16:10.173162 | controller | cd+++++++++ repos/config/.git/refs/heads/ 2026-06-10 01:16:10.173194 | controller | >f+++++++++ repos/config/.git/refs/heads/master 2026-06-10 01:16:10.173227 | controller | cd+++++++++ repos/config/.git/refs/remotes/ 2026-06-10 01:16:10.173260 | controller | cd+++++++++ repos/config/.git/refs/remotes/gerrit/ 2026-06-10 01:16:10.173293 | controller | >f+++++++++ repos/config/.git/refs/remotes/gerrit/HEAD 2026-06-10 01:16:10.173327 | controller | >f+++++++++ repos/config/.git/refs/remotes/gerrit/master 2026-06-10 01:16:10.173359 | controller | cd+++++++++ repos/config/.git/refs/remotes/origin/ 2026-06-10 01:16:10.173392 | controller | >f+++++++++ repos/config/.git/refs/remotes/origin/HEAD 2026-06-10 01:16:10.173424 | controller | >f+++++++++ repos/config/.git/refs/remotes/origin/master 2026-06-10 01:16:10.173457 | controller | cd+++++++++ repos/config/.git/refs/rewritten/ 2026-06-10 01:16:10.173490 | controller | cd+++++++++ repos/config/.git/refs/tags/ 2026-06-10 01:16:10.173524 | controller | cd+++++++++ repos/config/nodepool/ 2026-06-10 01:16:10.173557 | controller | >f+++++++++ repos/config/nodepool/nodepool.yaml 2026-06-10 01:16:10.173625 | controller | cd+++++++++ repos/config/zuul/ 2026-06-10 01:16:10.173662 | controller | >f+++++++++ repos/config/zuul/main.yaml 2026-06-10 01:16:10.173695 | controller | cd+++++++++ repos/demo-project/ 2026-06-10 01:16:10.173728 | controller | >f+++++++++ repos/demo-project/.gitreview 2026-06-10 01:16:10.173760 | controller | cd+++++++++ repos/demo-project/.git/ 2026-06-10 01:16:10.173793 | controller | >f+++++++++ repos/demo-project/.git/COMMIT_EDITMSG 2026-06-10 01:16:10.173844 | controller | >f+++++++++ repos/demo-project/.git/FETCH_HEAD 2026-06-10 01:16:10.173878 | controller | >f+++++++++ repos/demo-project/.git/HEAD 2026-06-10 01:16:10.173911 | controller | >f+++++++++ repos/demo-project/.git/ORIG_HEAD 2026-06-10 01:16:10.173943 | controller | >f+++++++++ repos/demo-project/.git/config 2026-06-10 01:16:10.173976 | controller | >f+++++++++ repos/demo-project/.git/description 2026-06-10 01:16:10.174008 | controller | >f+++++++++ repos/demo-project/.git/index 2026-06-10 01:16:10.174041 | controller | >f+++++++++ repos/demo-project/.git/packed-refs 2026-06-10 01:16:10.174073 | controller | cd+++++++++ repos/demo-project/.git/hooks/ 2026-06-10 01:16:10.174106 | controller | >f+++++++++ repos/demo-project/.git/hooks/applypatch-msg.sample 2026-06-10 01:16:10.174138 | controller | >f+++++++++ repos/demo-project/.git/hooks/commit-msg 2026-06-10 01:16:10.174171 | controller | >f+++++++++ repos/demo-project/.git/hooks/commit-msg.sample 2026-06-10 01:16:10.174204 | controller | >f+++++++++ repos/demo-project/.git/hooks/fsmonitor-watchman.sample 2026-06-10 01:16:10.174237 | controller | >f+++++++++ repos/demo-project/.git/hooks/post-update.sample 2026-06-10 01:16:10.174270 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-applypatch.sample 2026-06-10 01:16:10.174302 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-commit.sample 2026-06-10 01:16:10.174335 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-merge-commit.sample 2026-06-10 01:16:10.174368 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-push.sample 2026-06-10 01:16:10.174401 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-rebase.sample 2026-06-10 01:16:10.174434 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-receive.sample 2026-06-10 01:16:10.174467 | controller | >f+++++++++ repos/demo-project/.git/hooks/prepare-commit-msg.sample 2026-06-10 01:16:10.174500 | controller | >f+++++++++ repos/demo-project/.git/hooks/push-to-checkout.sample 2026-06-10 01:16:10.174533 | controller | >f+++++++++ repos/demo-project/.git/hooks/sendemail-validate.sample 2026-06-10 01:16:10.174566 | controller | >f+++++++++ repos/demo-project/.git/hooks/update.sample 2026-06-10 01:16:10.174636 | controller | cd+++++++++ repos/demo-project/.git/info/ 2026-06-10 01:16:10.174688 | controller | >f+++++++++ repos/demo-project/.git/info/exclude 2026-06-10 01:16:10.174723 | controller | cd+++++++++ repos/demo-project/.git/logs/ 2026-06-10 01:16:10.174755 | controller | >f+++++++++ repos/demo-project/.git/logs/HEAD 2026-06-10 01:16:10.174788 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/ 2026-06-10 01:16:10.174821 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/heads/ 2026-06-10 01:16:10.174854 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/master 2026-06-10 01:16:10.174886 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/test_check_url 2026-06-10 01:16:10.174920 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/test_short_job 2026-06-10 01:16:10.174953 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/ 2026-06-10 01:16:10.174986 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/ 2026-06-10 01:16:10.175019 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/HEAD 2026-06-10 01:16:10.175052 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/master 2026-06-10 01:16:10.175085 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/ 2026-06-10 01:16:10.175118 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/HEAD 2026-06-10 01:16:10.175151 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/master 2026-06-10 01:16:10.175183 | controller | cd+++++++++ repos/demo-project/.git/objects/ 2026-06-10 01:16:10.175216 | controller | cd+++++++++ repos/demo-project/.git/objects/00/ 2026-06-10 01:16:10.175249 | controller | >f+++++++++ repos/demo-project/.git/objects/00/7c4957494f22626bb76384e0a77eb9ec53fdcb 2026-06-10 01:16:10.175281 | controller | cd+++++++++ repos/demo-project/.git/objects/07/ 2026-06-10 01:16:10.175314 | controller | >f+++++++++ repos/demo-project/.git/objects/07/902da752a3d0648afb857e375184b13de13084 2026-06-10 01:16:10.175346 | controller | cd+++++++++ repos/demo-project/.git/objects/15/ 2026-06-10 01:16:10.175378 | controller | >f+++++++++ repos/demo-project/.git/objects/15/9aa84c7b3593a5aef25dfc1efc6355e8943655 2026-06-10 01:16:10.175410 | controller | cd+++++++++ repos/demo-project/.git/objects/24/ 2026-06-10 01:16:10.175442 | controller | >f+++++++++ repos/demo-project/.git/objects/24/b588ea643039d084dbf42ad74e56b159656690 2026-06-10 01:16:10.175475 | controller | cd+++++++++ repos/demo-project/.git/objects/26/ 2026-06-10 01:16:10.175507 | controller | >f+++++++++ repos/demo-project/.git/objects/26/a3816fbd763ed648023f02849f8c3b0db3966c 2026-06-10 01:16:10.175540 | controller | cd+++++++++ repos/demo-project/.git/objects/33/ 2026-06-10 01:16:10.175600 | controller | >f+++++++++ repos/demo-project/.git/objects/33/f6891e6efb4109b0f6ed1dd09260b7ef5e9cf1 2026-06-10 01:16:10.175644 | controller | cd+++++++++ repos/demo-project/.git/objects/34/ 2026-06-10 01:16:10.175679 | controller | >f+++++++++ repos/demo-project/.git/objects/34/b5843439d922385c7cb8c9794699263b573c66 2026-06-10 01:16:10.175712 | controller | cd+++++++++ repos/demo-project/.git/objects/3f/ 2026-06-10 01:16:10.175746 | controller | >f+++++++++ repos/demo-project/.git/objects/3f/d135bd9b6f99f568fbcc367ed9bf0f6f95c3e3 2026-06-10 01:16:10.175779 | controller | cd+++++++++ repos/demo-project/.git/objects/46/ 2026-06-10 01:16:10.175812 | controller | >f+++++++++ repos/demo-project/.git/objects/46/ce7acecf042191db7a61ff969587684559fd50 2026-06-10 01:16:10.175845 | controller | cd+++++++++ repos/demo-project/.git/objects/48/ 2026-06-10 01:16:10.175878 | controller | >f+++++++++ repos/demo-project/.git/objects/48/c7e8a6339bdda2526991aa9fb3707cb253e2c4 2026-06-10 01:16:10.175911 | controller | cd+++++++++ repos/demo-project/.git/objects/4a/ 2026-06-10 01:16:10.175944 | controller | >f+++++++++ repos/demo-project/.git/objects/4a/b7d04115f2ba7de861fe640c9b9450da2132a2 2026-06-10 01:16:10.175977 | controller | cd+++++++++ repos/demo-project/.git/objects/4d/ 2026-06-10 01:16:10.176010 | controller | >f+++++++++ repos/demo-project/.git/objects/4d/d05ea0a58950ee887bfe1b67cfd7f44b8bba35 2026-06-10 01:16:10.176051 | controller | cd+++++++++ repos/demo-project/.git/objects/51/ 2026-06-10 01:16:10.176084 | controller | >f+++++++++ repos/demo-project/.git/objects/51/febfc882d29214e8ddb275b148abb77a18bac4 2026-06-10 01:16:10.176117 | controller | cd+++++++++ repos/demo-project/.git/objects/5f/ 2026-06-10 01:16:10.176150 | controller | >f+++++++++ repos/demo-project/.git/objects/5f/a965ccd5f8194e7865f9e02143de7481c0ea8c 2026-06-10 01:16:10.176182 | controller | cd+++++++++ repos/demo-project/.git/objects/60/ 2026-06-10 01:16:10.176215 | controller | >f+++++++++ repos/demo-project/.git/objects/60/4a228bad7e3a0aeb33c04c5f610ca1774ff81c 2026-06-10 01:16:10.176248 | controller | cd+++++++++ repos/demo-project/.git/objects/6b/ 2026-06-10 01:16:10.176280 | controller | >f+++++++++ repos/demo-project/.git/objects/6b/ad82d5a4960cb77c1069b288b57314cc4c3e09 2026-06-10 01:16:10.176313 | controller | cd+++++++++ repos/demo-project/.git/objects/71/ 2026-06-10 01:16:10.176346 | controller | >f+++++++++ repos/demo-project/.git/objects/71/cfda7043eeeb26af2c21e2e75965327b0966aa 2026-06-10 01:16:10.176382 | controller | cd+++++++++ repos/demo-project/.git/objects/72/ 2026-06-10 01:16:10.176416 | controller | >f+++++++++ repos/demo-project/.git/objects/72/dfc9c9553ebbd3d3f399036f015fd433cea17c 2026-06-10 01:16:10.176452 | controller | cd+++++++++ repos/demo-project/.git/objects/74/ 2026-06-10 01:16:10.176485 | controller | >f+++++++++ repos/demo-project/.git/objects/74/32b55d7db19f8c788f8fe355819a630f28906d 2026-06-10 01:16:10.176519 | controller | cd+++++++++ repos/demo-project/.git/objects/76/ 2026-06-10 01:16:10.176552 | controller | >f+++++++++ repos/demo-project/.git/objects/76/329e0d45f54dea4bdf76328e8597e999225689 2026-06-10 01:16:10.176621 | controller | cd+++++++++ repos/demo-project/.git/objects/78/ 2026-06-10 01:16:10.176659 | controller | >f+++++++++ repos/demo-project/.git/objects/78/27a04c00e02d0195342740c1b6154592d4cac0 2026-06-10 01:16:10.176694 | controller | cd+++++++++ repos/demo-project/.git/objects/89/ 2026-06-10 01:16:10.176726 | controller | >f+++++++++ repos/demo-project/.git/objects/89/d21bb11d68c711ade1530fb3a3b4cdfd7d72ee 2026-06-10 01:16:10.176759 | controller | cd+++++++++ repos/demo-project/.git/objects/9d/ 2026-06-10 01:16:10.176793 | controller | >f+++++++++ repos/demo-project/.git/objects/9d/139d568fb3c661b5a0c54ac6e96a1c87231bbc 2026-06-10 01:16:10.176826 | controller | cd+++++++++ repos/demo-project/.git/objects/a7/ 2026-06-10 01:16:10.176859 | controller | >f+++++++++ repos/demo-project/.git/objects/a7/69aa02a4833e1a9ba49c0c3a9374b2adbecd95 2026-06-10 01:16:10.176892 | controller | cd+++++++++ repos/demo-project/.git/objects/aa/ 2026-06-10 01:16:10.176925 | controller | >f+++++++++ repos/demo-project/.git/objects/aa/2bf83c4d594393624a3707f355d7a7cbcacf22 2026-06-10 01:16:10.176959 | controller | cd+++++++++ repos/demo-project/.git/objects/af/ 2026-06-10 01:16:10.176991 | controller | >f+++++++++ repos/demo-project/.git/objects/af/b9c51ff287f6408185ec86d9f4781cbe40472e 2026-06-10 01:16:10.177024 | controller | cd+++++++++ repos/demo-project/.git/objects/bb/ 2026-06-10 01:16:10.177057 | controller | >f+++++++++ repos/demo-project/.git/objects/bb/037cc610ee38bde70c3eeeb925fe0206daeccb 2026-06-10 01:16:10.177091 | controller | cd+++++++++ repos/demo-project/.git/objects/cc/ 2026-06-10 01:16:10.177124 | controller | >f+++++++++ repos/demo-project/.git/objects/cc/0db6e4fb0786eae97a6c53baac8a060e4901e6 2026-06-10 01:16:10.177158 | controller | cd+++++++++ repos/demo-project/.git/objects/d0/ 2026-06-10 01:16:10.177191 | controller | >f+++++++++ repos/demo-project/.git/objects/d0/05b22e4415ca5ced07782c79053ba35c56eb95 2026-06-10 01:16:10.177223 | controller | >f+++++++++ repos/demo-project/.git/objects/d0/2db411aaa63f2a85daf1c6c871d8ca080778ba 2026-06-10 01:16:10.177257 | controller | >f+++++++++ repos/demo-project/.git/objects/d0/e66ab409ee4347449c43a1e3773363c72488b5 2026-06-10 01:16:10.177290 | controller | cd+++++++++ repos/demo-project/.git/objects/d4/ 2026-06-10 01:16:10.177332 | controller | >f+++++++++ repos/demo-project/.git/objects/d4/ad8d0fd75f2c12b671a0a497958c9d366b672c 2026-06-10 01:16:10.177367 | controller | cd+++++++++ repos/demo-project/.git/objects/db/ 2026-06-10 01:16:10.177401 | controller | >f+++++++++ repos/demo-project/.git/objects/db/eb08405aec76c24906ac7dfe1f13fed7fa2908 2026-06-10 01:16:10.177434 | controller | cd+++++++++ repos/demo-project/.git/objects/f5/ 2026-06-10 01:16:10.177466 | controller | >f+++++++++ repos/demo-project/.git/objects/f5/1aa562330b0e9d2a7156e0bfe794287dc5b976 2026-06-10 01:16:10.177500 | controller | cd+++++++++ repos/demo-project/.git/objects/fe/ 2026-06-10 01:16:10.177533 | controller | >f+++++++++ repos/demo-project/.git/objects/fe/888323db29e0dfec803c565befd8ec95e6b796 2026-06-10 01:16:10.177566 | controller | cd+++++++++ repos/demo-project/.git/objects/info/ 2026-06-10 01:16:10.177635 | controller | cd+++++++++ repos/demo-project/.git/objects/pack/ 2026-06-10 01:16:10.177670 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-beeab3813e7488fa65e202c930c948d9a72d3410.idx 2026-06-10 01:16:10.177703 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-beeab3813e7488fa65e202c930c948d9a72d3410.pack 2026-06-10 01:16:10.177736 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-beeab3813e7488fa65e202c930c948d9a72d3410.rev 2026-06-10 01:16:10.177769 | controller | cd+++++++++ repos/demo-project/.git/refs/ 2026-06-10 01:16:10.177802 | controller | cd+++++++++ repos/demo-project/.git/refs/heads/ 2026-06-10 01:16:10.177854 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/master 2026-06-10 01:16:10.177889 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/test_check_url 2026-06-10 01:16:10.177922 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/test_short_job 2026-06-10 01:16:10.177955 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/ 2026-06-10 01:16:10.177987 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/gerrit/ 2026-06-10 01:16:10.178020 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/gerrit/HEAD 2026-06-10 01:16:10.178053 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/gerrit/master 2026-06-10 01:16:10.178085 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/origin/ 2026-06-10 01:16:10.178117 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/origin/HEAD 2026-06-10 01:16:10.178150 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/origin/master 2026-06-10 01:16:10.178182 | controller | cd+++++++++ repos/demo-project/.git/refs/rewritten/ 2026-06-10 01:16:10.178214 | controller | cd+++++++++ repos/demo-project/.git/refs/tags/ 2026-06-10 01:16:10.178247 | controller | cd+++++++++ repos/demo-project/.zuul.d/ 2026-06-10 01:16:10.178280 | controller | >f+++++++++ repos/demo-project/.zuul.d/jobs.yaml 2026-06-10 01:16:10.178313 | controller | >f+++++++++ repos/demo-project/.zuul.d/project.yaml 2026-06-10 01:16:10.178346 | controller | >f+++++++++ repos/demo-project/.zuul.d/secrets.yaml 2026-06-10 01:16:10.178379 | controller | cd+++++++++ repos/demo-project/playbooks/ 2026-06-10 01:16:10.178412 | controller | >f+++++++++ repos/demo-project/playbooks/microshift-pod.yml 2026-06-10 01:16:10.178444 | controller | >f+++++++++ repos/demo-project/playbooks/run-check-url.yaml 2026-06-10 01:16:10.178477 | controller | >f+++++++++ repos/demo-project/playbooks/run.yaml 2026-06-10 01:16:10.178509 | controller | cd+++++++++ repos/demo-project/zuul.d/ 2026-06-10 01:16:10.178541 | controller | >f+++++++++ repos/demo-project/zuul.d/config.yaml 2026-06-10 01:16:10.178626 | controller | cd+++++++++ repos/demo-tenant-config/ 2026-06-10 01:16:10.178728 | controller | >f+++++++++ repos/demo-tenant-config/.gitreview 2026-06-10 01:16:10.178765 | controller | cd+++++++++ repos/demo-tenant-config/.git/ 2026-06-10 01:16:10.178798 | controller | >f+++++++++ repos/demo-tenant-config/.git/COMMIT_EDITMSG 2026-06-10 01:16:10.178840 | controller | >f+++++++++ repos/demo-tenant-config/.git/HEAD 2026-06-10 01:16:10.178874 | controller | >f+++++++++ repos/demo-tenant-config/.git/ORIG_HEAD 2026-06-10 01:16:10.178906 | controller | >f+++++++++ repos/demo-tenant-config/.git/config 2026-06-10 01:16:10.178939 | controller | >f+++++++++ repos/demo-tenant-config/.git/description 2026-06-10 01:16:10.178971 | controller | >f+++++++++ repos/demo-tenant-config/.git/index 2026-06-10 01:16:10.179003 | controller | >f+++++++++ repos/demo-tenant-config/.git/packed-refs 2026-06-10 01:16:10.179035 | controller | cd+++++++++ repos/demo-tenant-config/.git/hooks/ 2026-06-10 01:16:10.179068 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/applypatch-msg.sample 2026-06-10 01:16:10.179101 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/commit-msg.sample 2026-06-10 01:16:10.179135 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/fsmonitor-watchman.sample 2026-06-10 01:16:10.179168 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/post-update.sample 2026-06-10 01:16:10.179201 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-applypatch.sample 2026-06-10 01:16:10.179233 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-commit.sample 2026-06-10 01:16:10.179266 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-merge-commit.sample 2026-06-10 01:16:10.179298 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-push.sample 2026-06-10 01:16:10.179330 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-rebase.sample 2026-06-10 01:16:10.179362 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-receive.sample 2026-06-10 01:16:10.179394 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/prepare-commit-msg.sample 2026-06-10 01:16:10.179426 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/push-to-checkout.sample 2026-06-10 01:16:10.179459 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/sendemail-validate.sample 2026-06-10 01:16:10.179492 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/update.sample 2026-06-10 01:16:10.179525 | controller | cd+++++++++ repos/demo-tenant-config/.git/info/ 2026-06-10 01:16:10.179557 | controller | >f+++++++++ repos/demo-tenant-config/.git/info/exclude 2026-06-10 01:16:10.179627 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/ 2026-06-10 01:16:10.179664 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/HEAD 2026-06-10 01:16:10.179697 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/ 2026-06-10 01:16:10.179730 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/heads/ 2026-06-10 01:16:10.179763 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/heads/master 2026-06-10 01:16:10.179796 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/ 2026-06-10 01:16:10.179829 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/ 2026-06-10 01:16:10.179862 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/HEAD 2026-06-10 01:16:10.179895 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/master 2026-06-10 01:16:10.179928 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/ 2026-06-10 01:16:10.179961 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/35/ 2026-06-10 01:16:10.179993 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/35/b27848a551ba9940a7d29be8ac540ae6bf76c6 2026-06-10 01:16:10.180026 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/50/ 2026-06-10 01:16:10.180058 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/50/fde5ebd6d390f207bd0b354430c39ad4ef0cd8 2026-06-10 01:16:10.180091 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/71/ 2026-06-10 01:16:10.180123 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/71/766da1ed79ae81b8510541bc42a17123e9f4c7 2026-06-10 01:16:10.180164 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/7e/ 2026-06-10 01:16:10.180200 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/7e/56626d48d97eee76893616729b62b50ce1d6aa 2026-06-10 01:16:10.180235 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/82/ 2026-06-10 01:16:10.180271 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/82/7fb999b752e98c420a55a6844053be01b085ed 2026-06-10 01:16:10.180307 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/9e/ 2026-06-10 01:16:10.180343 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/9e/d5bc9a591ecf02f8b90e37ef0395c13113ad9d 2026-06-10 01:16:10.180379 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/b2/ 2026-06-10 01:16:10.180416 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/b2/9ffbd8fb2947c43ddcd388735096ad3451acaa 2026-06-10 01:16:10.180452 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/e4/ 2026-06-10 01:16:10.180488 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/e4/641b65f1174ec39331eff28a4e1223a9b05c89 2026-06-10 01:16:10.180525 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/info/ 2026-06-10 01:16:10.180560 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/pack/ 2026-06-10 01:16:10.180634 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-3748ffd6a863b8aee0cfb5fac53fe99557b69ede.idx 2026-06-10 01:16:10.180673 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-3748ffd6a863b8aee0cfb5fac53fe99557b69ede.pack 2026-06-10 01:16:10.180709 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-3748ffd6a863b8aee0cfb5fac53fe99557b69ede.rev 2026-06-10 01:16:10.180745 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/ 2026-06-10 01:16:10.180781 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/heads/ 2026-06-10 01:16:10.180817 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/heads/master 2026-06-10 01:16:10.180852 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/remotes/ 2026-06-10 01:16:10.180887 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/ 2026-06-10 01:16:10.180922 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/HEAD 2026-06-10 01:16:10.180958 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/master 2026-06-10 01:16:10.180994 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/tags/ 2026-06-10 01:16:10.181097 | controller | cd+++++++++ repos/demo-tenant-config/playbooks/ 2026-06-10 01:16:10.181140 | controller | >f+++++++++ repos/demo-tenant-config/playbooks/gerrit-pre.yaml 2026-06-10 01:16:10.181177 | controller | cd+++++++++ repos/demo-tenant-config/zuul.d/ 2026-06-10 01:16:10.181213 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-base-jobs.yaml 2026-06-10 01:16:10.181250 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-pipeline.yaml 2026-06-10 01:16:10.181285 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-project-pipeline.yaml 2026-06-10 01:16:10.181401 | controller | changed: All items complete 2026-06-10 01:16:10.181449 | 2026-06-10 01:16:10.729597 | controller | changed: .d..t...... ./ 2026-06-10 01:16:11.298996 | controller | changed: .d..t...... ./ 2026-06-10 01:16:11.333572 | 2026-06-10 01:16:11.333883 | TASK [include_role : fetch-output-openshift] 2026-06-10 01:16:11.361275 | controller | skipping: Conditional result was False 2026-06-10 01:16:11.373741 | 2026-06-10 01:16:11.373897 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-06-10 01:16:11.423835 | controller | skipping: Conditional result was False 2026-06-10 01:16:11.433044 | controller | skipping: Conditional result was False 2026-06-10 01:16:11.458023 | 2026-06-10 01:16:11.458227 | PLAY [localhost] 2026-06-10 01:16:11.470904 | 2026-06-10 01:16:11.471036 | TASK [add-fileserver : Create SSH private key tempfile] 2026-06-10 01:16:11.932884 | localhost | changed 2026-06-10 01:16:11.983273 | 2026-06-10 01:16:11.983455 | TASK [add-fileserver : Create SSH private key from secret] 2026-06-10 01:16:12.820495 | localhost | changed 2026-06-10 01:16:12.829058 | 2026-06-10 01:16:12.829253 | TASK [add-fileserver : Add fileserver ssh key] 2026-06-10 01:16:13.282491 | localhost | Identity added: /var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/work/tmp/ansible.ep1iag9g (/var/lib/zuul/builds/5d71f650632045e78ef50fac9860d220/work/tmp/ansible.ep1iag9g) 2026-06-10 01:16:13.282884 | localhost | ok: Runtime: 0:00:00.020700 2026-06-10 01:16:13.289986 | 2026-06-10 01:16:13.290116 | TASK [add-fileserver : Remove SSH private key from disk] 2026-06-10 01:16:13.638659 | localhost | ok: Runtime: 0:00:00.010813 2026-06-10 01:16:13.646846 | 2026-06-10 01:16:13.646981 | TASK [add-fileserver : Add fileserver to inventory] 2026-06-10 01:16:13.722498 | localhost | changed 2026-06-10 01:16:13.729192 | 2026-06-10 01:16:13.729320 | TASK [add-fileserver : Add fileserver server to known hosts] 2026-06-10 01:16:14.199231 | localhost | changed 2026-06-10 01:16:14.212239 | 2026-06-10 01:16:14.212368 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-06-10 01:16:14.682034 | localhost | changed 2026-06-10 01:16:14.701209 | 2026-06-10 01:16:14.701351 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-06-10 01:16:14.732362 | localhost | ok 2026-06-10 01:16:14.742394 | 2026-06-10 01:16:14.742487 | TASK [zuul_return] 2026-06-10 01:16:14.789071 | localhost | skipping: Conditional result was False 2026-06-10 01:16:14.794486 | 2026-06-10 01:16:14.794640 | PLAY [a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com] 2026-06-10 01:16:14.801842 | 2026-06-10 01:16:14.801971 | TASK [Set zuul-log-path fact] 2026-06-10 01:16:14.831708 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | ok 2026-06-10 01:16:14.849815 | 2026-06-10 01:16:14.849962 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 01:16:14.890429 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | ok 2026-06-10 01:16:14.896246 | 2026-06-10 01:16:14.896376 | TASK [upload-logs : Create log directories] 2026-06-10 01:16:15.751963 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | changed 2026-06-10 01:16:15.755556 | 2026-06-10 01:16:15.755719 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-06-10 01:16:16.130887 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com -> localhost | ok: Runtime: 0:00:00.017963 2026-06-10 01:16:16.135406 | 2026-06-10 01:16:16.135542 | TASK [upload-logs : Upload logs to log server] 2026-06-10 01:16:37.546661 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | changed: 2026-06-10 01:16:37.546974 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | .d..tp..... ./ 2026-06-10 01:16:37.547030 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com |