2026-06-10 10:45:00.062395 | Job console starting 2026-06-10 10:45:00.201770 | Updating git repos 2026-06-10 10:45:00.246079 | Cloning repos into workspace 2026-06-10 10:45:00.800306 | Restoring repo states 2026-06-10 10:45:00.832017 | Merging changes 2026-06-10 10:45:01.474992 | Checking out repos 2026-06-10 10:45:01.609854 | Preparing playbooks 2026-06-10 10:45:04.065660 | Running Ansible setup 2026-06-10 10:45:08.911714 | PRE-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/pre.yaml@main] 2026-06-10 10:45:09.554730 | 2026-06-10 10:45:09.554843 | PLAY [localhost] 2026-06-10 10:45:09.561645 | 2026-06-10 10:45:09.561712 | TASK [Gathering Facts] 2026-06-10 10:45:10.590380 | localhost | ok 2026-06-10 10:45:10.603488 | 2026-06-10 10:45:10.603657 | TASK [Setup log path fact] 2026-06-10 10:45:10.627796 | localhost | ok 2026-06-10 10:45:10.649043 | 2026-06-10 10:45:10.649179 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 10:45:10.677941 | localhost | ok 2026-06-10 10:45:10.684946 | 2026-06-10 10:45:10.685037 | TASK [emit-job-header : Print job information] 2026-06-10 10:45:10.724380 | # Job Information 2026-06-10 10:45:10.724554 | Ansible Version: 2.16.18 2026-06-10 10:45:10.724594 | Job: sf-operator-minikube-integration 2026-06-10 10:45:10.724622 | Pipeline: check 2026-06-10 10:45:10.724647 | Executor: 38.102.83.51 2026-06-10 10:45:10.724669 | Triggered by: https://softwarefactory-project.io/r/c/software-factory/sf-operator/+/34868 2026-06-10 10:45:10.724696 | Log URL (when completed): https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/logs/742/sf/74204cee34be4cddbbd3b400f2a79c3d/ 2026-06-10 10:45:10.724721 | Event ID: df3f693c7b06489081990b4e104019d5 2026-06-10 10:45:10.729039 | 2026-06-10 10:45:10.729129 | LOOP [emit-job-header : Print node information] 2026-06-10 10:45:10.843469 | localhost | ok: 2026-06-10 10:45:10.843805 | localhost | # Node Information 2026-06-10 10:45:10.843863 | localhost | Inventory Hostname: controller 2026-06-10 10:45:10.843905 | localhost | Hostname: np0000069616 2026-06-10 10:45:10.843944 | localhost | Username: zuul-worker 2026-06-10 10:45:10.843991 | localhost | Distro: CentOS 9 2026-06-10 10:45:10.844034 | localhost | Provider: vexxhost-nodepool-sf 2026-06-10 10:45:10.844070 | localhost | Region: RegionOne 2026-06-10 10:45:10.844105 | localhost | Label: cloud-centos-9-big 2026-06-10 10:45:10.844138 | localhost | Product Name: OpenStack Nova 2026-06-10 10:45:10.844173 | localhost | Interface IP: 38.102.83.66 2026-06-10 10:45:10.851465 | 2026-06-10 10:45:10.851624 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-10 10:45:11.303514 | localhost -> localhost | changed 2026-06-10 10:45:11.312666 | 2026-06-10 10:45:11.312778 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-06-10 10:45:12.380303 | localhost -> localhost | changed 2026-06-10 10:45:12.387043 | 2026-06-10 10:45:12.387101 | PLAY [all] 2026-06-10 10:45:12.393371 | 2026-06-10 10:45:12.393439 | TASK [include_role : start-zuul-console] 2026-06-10 10:45:12.412657 | controller | ok 2026-06-10 10:45:12.422007 | 2026-06-10 10:45:12.422069 | TASK [start-zuul-console : Start zuul_console daemon.] 2026-06-10 10:45:12.828683 | controller | ok 2026-06-10 10:45:12.843633 | 2026-06-10 10:45:12.843771 | TASK [include_role : validate-host] 2026-06-10 10:45:12.877697 | controller | ok 2026-06-10 10:45:12.911845 | 2026-06-10 10:45:12.912013 | TASK [validate-host : Define zuul_info_dir fact] 2026-06-10 10:45:12.951800 | controller | ok 2026-06-10 10:45:12.957020 | 2026-06-10 10:45:12.957108 | TASK [validate-host : Ensure Zuul Ansible directory exists] 2026-06-10 10:45:13.307253 | controller -> localhost | ok 2026-06-10 10:45:13.317890 | 2026-06-10 10:45:13.318022 | TASK [validate-host : Collect information about the host] 2026-06-10 10:45:14.086971 | controller | ok 2026-06-10 10:45:14.144500 | 2026-06-10 10:45:14.144667 | TASK [validate-host : Sanitize hostname] 2026-06-10 10:45:14.197572 | controller | ok 2026-06-10 10:45:14.205395 | 2026-06-10 10:45:14.205573 | TASK [validate-host : Write out all ansible variables/facts known for each host] 2026-06-10 10:45:14.738551 | controller -> localhost | changed 2026-06-10 10:45:14.748309 | 2026-06-10 10:45:14.748498 | TASK [validate-host : Collect information about zuul worker] 2026-06-10 10:45:15.119550 | controller | ok 2026-06-10 10:45:15.126810 | 2026-06-10 10:45:15.126963 | TASK [validate-host : Write out all zuul information for each host] 2026-06-10 10:45:15.690982 | controller -> localhost | changed 2026-06-10 10:45:15.705804 | 2026-06-10 10:45:15.706128 | TASK [include_role : prepare-workspace] 2026-06-10 10:45:15.738310 | controller | ok 2026-06-10 10:45:15.774004 | 2026-06-10 10:45:15.774091 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-06-10 10:45:16.038616 | controller | ok 2026-06-10 10:45:16.050501 | 2026-06-10 10:45:16.050656 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-06-10 10:45:17.410867 | controller | Output suppressed because no_log was given 2026-06-10 10:45:17.418778 | 2026-06-10 10:45:17.418907 | TASK [include_role : add-build-sshkey] 2026-06-10 10:45:17.454920 | controller | ok 2026-06-10 10:45:17.482621 | 2026-06-10 10:45:17.482724 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-06-10 10:45:17.781396 | controller -> localhost | ok 2026-06-10 10:45:17.787484 | 2026-06-10 10:45:17.787581 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-06-10 10:45:17.833863 | controller | ok 2026-06-10 10:45:17.864752 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-06-10 10:45:17.872823 | 2026-06-10 10:45:17.872914 | TASK [add-build-sshkey : Create Temp SSH key] 2026-06-10 10:45:18.521717 | controller -> localhost | Generating public/private rsa key pair. 2026-06-10 10:45:18.521916 | controller -> localhost | Your identification has been saved in /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/work/74204cee34be4cddbbd3b400f2a79c3d_id_rsa 2026-06-10 10:45:18.521946 | controller -> localhost | Your public key has been saved in /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/work/74204cee34be4cddbbd3b400f2a79c3d_id_rsa.pub 2026-06-10 10:45:18.521967 | controller -> localhost | The key fingerprint is: 2026-06-10 10:45:18.521987 | controller -> localhost | SHA256:NXclAaaZK6O7IThyNpq3EuBKsXIqhUFfEEzj1m25I0Q zuul-build-sshkey 2026-06-10 10:45:18.522006 | controller -> localhost | The key's randomart image is: 2026-06-10 10:45:18.522023 | controller -> localhost | +---[RSA 3072]----+ 2026-06-10 10:45:18.522040 | controller -> localhost | | o=oE o.o..| 2026-06-10 10:45:18.522059 | controller -> localhost | | ...+.. . = o | 2026-06-10 10:45:18.522076 | controller -> localhost | |. .o.o + * . . | 2026-06-10 10:45:18.522093 | controller -> localhost | |o.... . .. + . | 2026-06-10 10:45:18.522110 | controller -> localhost | |ooo . oS . | 2026-06-10 10:45:18.522126 | controller -> localhost | |++o. ...o | 2026-06-10 10:45:18.522144 | controller -> localhost | |==* . o | 2026-06-10 10:45:18.522163 | controller -> localhost | |=*.o . o | 2026-06-10 10:45:18.522181 | controller -> localhost | |+o.. o. | 2026-06-10 10:45:18.522198 | controller -> localhost | +----[SHA256]-----+ 2026-06-10 10:45:18.522245 | controller -> localhost | ok: Runtime: 0:00:00.149912 2026-06-10 10:45:18.526171 | 2026-06-10 10:45:18.526236 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-06-10 10:45:18.566937 | controller | ok 2026-06-10 10:45:18.574722 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-06-10 10:45:18.581304 | 2026-06-10 10:45:18.581381 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-06-10 10:45:18.616925 | controller | skipping: Conditional result was False 2026-06-10 10:45:18.621702 | 2026-06-10 10:45:18.621772 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-06-10 10:45:19.082225 | controller | changed 2026-06-10 10:45:19.090420 | 2026-06-10 10:45:19.090597 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-06-10 10:45:19.353463 | controller | ok 2026-06-10 10:45:19.361038 | 2026-06-10 10:45:19.361182 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-06-10 10:45:20.366844 | controller | changed 2026-06-10 10:45:20.374405 | 2026-06-10 10:45:20.374550 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-06-10 10:45:21.173618 | controller | changed 2026-06-10 10:45:21.178612 | 2026-06-10 10:45:21.178698 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-06-10 10:45:21.215057 | controller | skipping: Conditional result was False 2026-06-10 10:45:21.223886 | 2026-06-10 10:45:21.224056 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-06-10 10:45:21.646841 | controller -> localhost | changed 2026-06-10 10:45:21.662847 | 2026-06-10 10:45:21.662985 | TASK [add-build-sshkey : Add back temp key] 2026-06-10 10:45:22.043521 | controller -> localhost | Identity added: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/work/74204cee34be4cddbbd3b400f2a79c3d_id_rsa (zuul-build-sshkey) 2026-06-10 10:45:22.043872 | controller -> localhost | ok: Runtime: 0:00:00.019373 2026-06-10 10:45:22.051696 | 2026-06-10 10:45:22.051830 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-06-10 10:45:22.406715 | controller | ok 2026-06-10 10:45:22.413656 | 2026-06-10 10:45:22.413802 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-06-10 10:45:22.452294 | controller | skipping: Conditional result was False 2026-06-10 10:45:22.467069 | 2026-06-10 10:45:22.467210 | TASK [include_role : prepare-workspace-openshift] 2026-06-10 10:45:22.495659 | controller | skipping: Conditional result was False 2026-06-10 10:45:22.505103 | 2026-06-10 10:45:22.505240 | TASK [include_role : remove-zuul-sshkey] 2026-06-10 10:45:22.532714 | controller | skipping: Conditional result was False 2026-06-10 10:45:22.542101 | 2026-06-10 10:45:22.542254 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-06-10 10:45:22.822504 | controller | ok: "logs" 2026-06-10 10:45:22.822938 | controller | ok: All items complete 2026-06-10 10:45:22.822992 | 2026-06-10 10:45:23.034126 | controller | ok: "artifacts" 2026-06-10 10:45:23.306035 | controller | ok: "docs" 2026-06-10 10:45:23.316694 | 2026-06-10 10:45:23.317048 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-06-10 10:45:23.595079 | controller | changed: "logs" 2026-06-10 10:45:23.846095 | controller | changed: "artifacts" 2026-06-10 10:45:24.062177 | controller | changed: "docs" 2026-06-10 10:45:24.080573 | 2026-06-10 10:45:24.080804 | LOOP [Tune dnf options] 2026-06-10 10:45:24.558867 | controller | changed: option added 2026-06-10 10:45:24.559374 | controller | changed: All items complete 2026-06-10 10:45:24.559447 | 2026-06-10 10:45:24.845640 | controller | changed: option added 2026-06-10 10:45:25.114657 | controller | changed: option added 2026-06-10 10:45:25.131015 | 2026-06-10 10:45:25.131194 | PLAY RECAP 2026-06-10 10:45:25.131286 | controller | ok: 24 changed: 11 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-06-10 10:45:25.131399 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 10:45:25.131455 | 2026-06-10 10:45:25.319423 | PRE-RUN END RESULT_NORMAL: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/pre.yaml@main] 2026-06-10 10:45:25.323110 | PRE-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/rhel_subscription/register.yaml@main] 2026-06-10 10:45:26.007613 | 2026-06-10 10:45:26.007743 | PLAY [all] 2026-06-10 10:45:26.017287 | 2026-06-10 10:45:26.017395 | TASK [Register system] 2026-06-10 10:45:26.103980 | 2026-06-10 10:45:26.104246 | TASK [Get minor release for EUS activation] 2026-06-10 10:45:26.152675 | controller | skipping: Conditional result was False 2026-06-10 10:45:26.159329 | 2026-06-10 10:45:26.159497 | LOOP [Manage Extended Update Support repos] 2026-06-10 10:45:26.221803 | controller | skipping: Conditional result was False 2026-06-10 10:45:26.226789 | controller | skipping: Conditional result was False 2026-06-10 10:45:26.237990 | 2026-06-10 10:45:26.238213 | TASK [Upgrade all packages] 2026-06-10 10:45:26.297580 | controller | skipping: Conditional result was False 2026-06-10 10:45:26.308778 | 2026-06-10 10:45:26.308964 | TASK [Reboot to use new kernel] 2026-06-10 10:45:26.388788 | controller | skipping: Conditional result was False 2026-06-10 10:45:26.391339 | 2026-06-10 10:45:26.391465 | PLAY RECAP 2026-06-10 10:45:26.391617 | controller | ok: 0 changed: 0 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-06-10 10:45:26.391674 | 2026-06-10 10:45:26.562262 | PRE-RUN END RESULT_NORMAL: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/rhel_subscription/register.yaml@main] 2026-06-10 10:45:26.563430 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-minikube.yaml@master] 2026-06-10 10:45:27.289960 | 2026-06-10 10:45:27.290105 | PLAY [controller] 2026-06-10 10:45:27.299899 | 2026-06-10 10:45:27.299968 | TASK [Create ramdisk folder] 2026-06-10 10:45:27.780114 | controller | changed 2026-06-10 10:45:27.792107 | 2026-06-10 10:45:27.792227 | TASK [Installing packages] 2026-06-10 10:46:24.098949 | controller | changed 2026-06-10 10:46:24.110754 | 2026-06-10 10:46:24.110933 | TASK [Set pids_limit in containers.conf] 2026-06-10 10:46:25.260626 | controller | changed 2026-06-10 10:46:25.270905 | 2026-06-10 10:46:25.271089 | TASK [Installing Minikube] 2026-06-10 10:46:26.992575 | controller | changed: OK (132766301 bytes) 2026-06-10 10:46:27.004554 | 2026-06-10 10:46:27.004691 | TASK [Start minikube] 2026-06-10 06:46:27.504180 | controller | * minikube v1.36.0 on Centos 9 (kvm/amd64) 2026-06-10 06:46:27.513894 | controller | Using Kubernetes 1.32.5 since patch version was unspecified 2026-06-10 06:46:28.073418 | controller | * minikube 1.38.1 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.38.1 2026-06-10 06:46:28.075070 | controller | * To disable this notice, run: 'minikube config set WantUpdateNotification false' 2026-06-10 06:46:28.075108 | controller | 2026-06-10 06:46:28.127628 | controller | * Using the podman driver based on user configuration 2026-06-10 06:46:28.734222 | controller | * Using Podman driver with root privileges 2026-06-10 06:46:28.774427 | controller | * Starting "minikube" primary control-plane node in "minikube" cluster 2026-06-10 06:46:28.779929 | controller | * Pulling base image v0.0.47 ... 2026-06-10 06:46:28.910654 | controller | * Downloading Kubernetes v1.32.5 preload ... 2026-06-10 06:46:49.912306 | controller | > gcr.io/k8s-minikube/kicbase...: 0 B [_______________________] ?% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 0 B [_______________________] ?% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB [>_] 0.00% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB 0.00% 2.68 KiB p/ > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB 0.00% 2.68 KiB p/ > gcr.io/k8s-minikube/kicbase...: 177.73 KiB / 502.26 MiB 0.03% 2.68 KiB > gcr.io/k8s-minikube/kicbase...: 1.42 MiB / 502.26 MiB 0.28% 158.38 KiB > gcr.io/k8s-minikube/kicbase...: 6.93 MiB / 502.26 MiB 1.38% 158.38 KiB > gcr.io/k8s-minikube/kicbase...: 12.97 MiB / 502.26 MiB 2.58% 158.38 KiB > gcr.io/k8s-minikube/kicbase...: 19.04 MiB / 502.26 MiB 3.79% 2.04 MiB p > gcr.io/k8s-minikube/kicbase...: 25.07 MiB / 502.26 MiB 4.99% 2.04 MiB p > gcr.io/k8s-minikube/kicbase...: 31.16 MiB / 502.26 MiB 6.20% 2.04 MiB p > gcr.io/k8s-minikube/kicbase...: 37.16 MiB / 502.26 MiB 7.40% 3.86 MiB p > gcr.io/k8s-minikube/kicbase...: 43.19 MiB / 502.26 MiB 8.60% 3.86 MiB p > gcr.io/k8s-minikube/kicbase...: 49.24 MiB / 502.26 MiB 9.80% 3.86 MiB p > gcr.io/k8s-minikube/kicbase...: 55.28 MiB / 502.26 MiB 11.01% 5.55 MiB > gcr.io/k8s-minikube/kicbase...: 61.34 MiB / 502.26 MiB 12.21% 5.55 MiB > gcr.io/k8s-minikube/kicbase...: 63.62 MiB / 502.26 MiB 12.67% 5.55 MiB > gcr.io/k8s-minikube/kicbase...: 63.68 MiB / 502.26 MiB 12.68% 6.10 MiB > gcr.io/k8s-minikube/kicbase...: 63.68 MiB / 502.26 MiB 12.68% 5.71 MiB > gcr.io/k8s-minikube/kicbase...: 63.68 MiB / 502.26 MiB 12.68% 5.71 MiB > gcr.io/k8s-minikube/kicbase...: 65.65 MiB / 502.26 MiB 13.07% 5.71 MiB > gcr.io/k8s-minikube/kicbase...: 71.51 MiB / 502.26 MiB 14.24% 5.71 MiB > gcr.io/k8s-minikube/kicbase...: 77.51 MiB / 502.26 MiB 15.43% 6.62 MiB > gcr.io/k8s-minikube/kicbase...: 83.56 MiB / 502.26 MiB 16.64% 6.62 MiB > gcr.io/k8s-minikube/kicbase...: 89.61 MiB / 502.26 MiB 17.84% 6.62 MiB > gcr.io/k8s-minikube/kicbase...: 95.65 MiB / 502.26 MiB 19.04% 8.15 MiB > gcr.io/k8s-minikube/kicbase...: 101.71 MiB / 502.26 MiB 20.25% 8.15 MiB > gcr.io/k8s-minikube/kicbase...: 107.75 MiB / 502.26 MiB 21.45% 8.15 MiB > gcr.io/k8s-minikube/kicbase...: 113.82 MiB / 502.26 MiB 22.66% 9.58 MiB > gcr.io/k8s-minikube/kicbase...: 119.91 MiB / 502.26 MiB 23.87% 9.58 MiB > gcr.io/k8s-minikube/kicbase...: 125.97 MiB / 502.26 MiB 25.08% 9.58 MiB > gcr.io/k8s-minikube/kicbase...: 132.01 MiB / 502.26 MiB 26.28% 10.91 Mi > gcr.io/k8s-minikube/kicbase...: 138.07 MiB / 502.26 MiB 27.49% 10.91 Mi > gcr.io/k8s-minikube/kicbase...: 144.11 MiB / 502.26 MiB 28.69% 10.91 Mi > gcr.io/k8s-minikube/kicbase...: 150.17 MiB / 502.26 MiB 29.90% 12.16 Mi > gcr.io/k8s-minikube/kicbase...: 156.23 MiB / 502.26 MiB 31.11% 12.16 Mi > gcr.io/k8s-minikube/kicbase...: 162.26 MiB / 502.26 MiB 32.31% 12.16 Mi > gcr.io/k8s-minikube/kicbase...: 168.33 MiB / 502.26 MiB 33.51% 13.33 Mi > gcr.io/k8s-minikube/kicbase...: 174.41 MiB / 502.26 MiB 34.73% 13.33 Mi > gcr.io/k8s-minikube/kicbase...: 175.27 MiB / 502.26 MiB 34.90% 13.33 Mi > gcr.io/k8s-minikube/kicbase...: 177.11 MiB / 502.26 MiB 35.26% 13.41 Mi > gcr.io/k8s-minikube/kicbase...: 183.17 MiB / 502.26 MiB 36.47% 13.41 Mi > gcr.io/k8s-minikube/kicbase...: 189.18 MiB / 502.26 MiB 37.66% 13.41 Mi > gcr.io/k8s-minikube/kicbase...: 195.25 MiB / 502.26 MiB 38.87% 14.50 Mi > gcr.io/k8s-minikube/kicbase...: 201.30 MiB / 502.26 MiB 40.08% 14.50 Mi > gcr.io/k8s-minikube/kicbase...: 207.34 MiB / 502.26 MiB 41.28% 14.50 Mi > gcr.io/k8s-minikube/kicbase...: 213.34 MiB / 502.26 MiB 42.48% 15.51 Mi > gcr.io/k8s-minikube/kicbase...: 219.40 MiB / 502.26 MiB 43.68% 15.51 Mi > gcr.io/k8s-minikube/kicbase...: 225.42 MiB / 502.26 MiB 44.88% 15.51 Mi > gcr.io/k8s-minikube/kicbase...: 231.51 MiB / 502.26 MiB 46.09% 16.46 Mi > gcr.io/k8s-minikube/kicbase...: 237.57 MiB / 502.26 MiB 47.30% 16.46 Mi > gcr.io/k8s-minikube/kicbase...: 243.63 MiB / 502.26 MiB 48.51% 16.46 Mi > gcr.io/k8s-minikube/kicbase...: 249.70 MiB / 502.26 MiB 49.72% 17.36 Mi > gcr.io/k8s-minikube/kicbase...: 255.76 MiB / 502.26 MiB 50.92% 17.36 Mi > gcr.io/k8s-minikube/kicbase...: 261.39 MiB / 502.26 MiB 52.04% 17.36 Mi > gcr.io/k8s-minikube/kicbase...: 264.36 MiB / 502.26 MiB 52.63% 17.81 Mi > gcr.io/k8s-minikube/kicbase...: 269.98 MiB / 502.26 MiB 53.75% 17.81 Mi > gcr.io/k8s-minikube/kicbase...: 276.05 MiB / 502.26 MiB 54.96% 17.81 Mi > gcr.io/k8s-minikube/kicbase...: 282.10 MiB / 502.26 MiB 56.17% 18.57 Mi > gcr.io/k8s-minikube/kicbase...: 288.17 MiB / 502.26 MiB 57.37% 18.57 Mi > gcr.io/k8s-minikube/kicbase...: 294.22 MiB / 502.26 MiB 58.58% 18.57 Mi > gcr.io/k8s-minikube/kicbase...: 300.28 MiB / 502.26 MiB 59.79% 19.33 Mi > gcr.io/k8s-minikube/kicbase...: 306.39 MiB / 502.26 MiB 61.00% 19.33 Mi > gcr.io/k8s-minikube/kicbase...: 312.43 MiB / 502.26 MiB 62.20% 19.33 Mi > gcr.io/k8s-minikube/kicbase...: 318.53 MiB / 502.26 MiB 63.42% 20.04 Mi > gcr.io/k8s-minikube/kicbase...: 324.56 MiB / 502.26 MiB 64.62% 20.04 Mi > gcr.io/k8s-minikube/kicbase...: 330.64 MiB / 502.26 MiB 65.83% 20.04 Mi > gcr.io/k8s-minikube/kicbase...: 336.70 MiB / 502.26 MiB 67.04% 20.70 Mi > gcr.io/k8s-minikube/kicbase...: 342.77 MiB / 502.26 MiB 68.25% 20.70 Mi > gcr.io/k8s-minikube/kicbase...: 348.82 MiB / 502.26 MiB 69.45% 20.70 Mi > gcr.io/k8s-minikube/kicbase...: 354.88 MiB / 502.26 MiB 70.66% 21.32 Mi > gcr.io/k8s-minikube/kicbase...: 360.95 MiB / 502.26 MiB 71.86% 21.32 Mi > gcr.io/k8s-minikube/kicbase...: 367.02 MiB / 502.26 MiB 73.07% 21.32 Mi > gcr.io/k8s-minikube/kicbase...: 373.11 MiB / 502.26 MiB 74.29% 21.91 Mi > gcr.io/k8s-minikube/kicbase...: 379.13 MiB / 502.26 MiB 75.49% 21.91 Mi > gcr.io/k8s-minikube/kicbase...: 385.18 MiB / 502.26 MiB 76.69% 21.91 Mi > gcr.io/k8s-minikube/kicbase...: 391.24 MiB / 502.26 MiB 77.90% 22.44 Mi > gcr.io/k8s-minikube/kicbase...: 397.32 MiB / 502.26 MiB 79.11% 22.44 Mi > gcr.io/k8s-minikube/kicbase...: 403.38 MiB / 502.26 MiB 80.31% 22.44 Mi > gcr.io/k8s-minikube/kicbase...: 405.23 MiB / 502.26 MiB 80.68% 22.50 Mi > gcr.io/k8s-minikube/kicbase...: 405.76 MiB / 502.26 MiB 80.79% 22.50 Mi > gcr.io/k8s-minikube/kicbase...: 411.41 MiB / 502.26 MiB 81.91% 22.50 Mi > gcr.io/k8s-minikube/kicbase...: 415.56 MiB / 502.26 MiB 82.74% 22.16 Mi > gcr.io/k8s-minikube/kicbase...: 419.66 MiB / 502.26 MiB 83.55% 22.16 Mi > gcr.io/k8s-minikube/kicbase...: 425.78 MiB / 502.26 MiB 84.77% 22.16 Mi > gcr.io/k8s-minikube/kicbase...: 431.88 MiB / 502.26 MiB 85.99% 22.48 Mi > gcr.io/k8s-minikube/kicbase...: 437.91 MiB / 502.26 MiB 87.19% 22.48 Mi > gcr.io/k8s-minikube/kicbase...: 444.00 MiB / 502.26 MiB 88.40% 22.48 Mi > gcr.io/k8s-minikube/kicbase...: 450.03 MiB / 502.26 MiB 89.60% 22.98 Mi > gcr.io/k8s-minikube/kicbase...: 456.10 MiB / 502.26 MiB 90.81% 22.98 Mi > gcr.io/k8s-minikube/kicbase...: 462.17 MiB / 502.26 MiB 92.02% 22.98 Mi > gcr.io/k8s-minikube/kicbase...: 468.24 MiB / 502.26 MiB 93.23% 23.46 Mi > gcr.io/k8s-minikube/kicbase...: 474.26 MiB / 502.26 MiB 94.42% 23.46 Mi > gcr.io/k8s-minikube/kicbase...: 480.32 MiB / 502.26 MiB 95.63% 23.46 Mi > gcr.io/k8s-minikube/kicbase...: 486.41 MiB / 502.26 MiB 96.84% 23.90 Mi > gcr.io/k8s-minikube/kicbase...: 492.45 MiB / 502.26 MiB 98.05% 23.90 Mi > gcr.io/k8s-minikube/kicbase...: 498.52 MiB / 502.26 MiB 99.25% 23.90 Mi > gcr.io/k8s-minikube/kicbase...: 502.26 MiB / 502.26 MiB 100.00% 25.47 ME0610 06:46:49.912222 11063 cache.go:225] Error downloading kic artifacts: not yet implemented, see issue #8426 2026-06-10 06:46:49.924100 | controller | * Creating podman container (CPUs=no-limit, Memory=no-limit) ... 2026-06-10 06:47:43.753262 | controller | * Preparing Kubernetes v1.32.5 on CRI-O 1.24.6 ... 2026-06-10 06:47:43.930744 | controller | - kubelet.housekeeping-interval=5m 2026-06-10 06:47:57.262805 | controller | - Generating certificates and keys ... 2026-06-10 06:47:57.283474 | controller | - Booting up control plane ... 2026-06-10 06:47:57.295399 | controller | - Configuring RBAC rules ... 2026-06-10 06:47:57.315658 | controller | * Configuring CNI (Container Networking Interface) ... 2026-06-10 06:47:57.700178 | controller | * Verifying Kubernetes components... 2026-06-10 06:47:57.804285 | controller | - Using image gcr.io/k8s-minikube/storage-provisioner:v5 2026-06-10 06:47:57.804331 | controller | - Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3 2026-06-10 06:47:57.804494 | controller | - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.3 2026-06-10 06:47:57.843677 | controller | - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.3 2026-06-10 06:47:57.848938 | controller | - Using image registry.k8s.io/ingress-nginx/controller:v1.12.2 2026-06-10 06:47:58.906318 | controller | * Verifying ingress addon... 2026-06-10 06:48:56.467863 | controller | * Enabled addons: ingress-dns, default-storageclass, storage-provisioner, ingress 2026-06-10 06:48:56.582808 | controller | * kubectl not found. If you need it, try: 'minikube kubectl -- get pods -A' 2026-06-10 06:48:56.588513 | controller | * Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default 2026-06-10 10:48:56.999863 | controller | ok: Runtime: 0:02:29.187581 2026-06-10 10:48:57.008469 | 2026-06-10 10:48:57.008609 | TASK [Get minikube deployment ip] 2026-06-10 06:48:57.749372 | controller | 192.168.49.2 2026-06-10 10:48:58.055180 | controller | ok: Runtime: 0:00:00.462669 2026-06-10 10:48:58.063325 | 2026-06-10 10:48:58.063485 | TASK [Get default interface] 2026-06-10 06:48:58.364954 | controller | eth0 2026-06-10 10:48:58.610742 | controller | ok: Runtime: 0:00:00.008752 2026-06-10 10:48:58.618336 | 2026-06-10 10:48:58.618492 | TASK [Test minikube] 2026-06-10 06:48:59.452565 | controller | NAMESPACE NAME READY STATUS RESTARTS AGE 2026-06-10 06:48:59.452813 | controller | ingress-nginx ingress-nginx-admission-create-58vqp 0/1 Completed 0 59s 2026-06-10 06:48:59.452990 | controller | ingress-nginx ingress-nginx-admission-patch-zkxtk 0/1 Completed 2 59s 2026-06-10 06:48:59.453148 | controller | ingress-nginx ingress-nginx-controller-86649c845b-c9s5l 0/1 Running 0 59s 2026-06-10 06:48:59.453350 | controller | kube-system coredns-668d6bf9bc-4rn2t 1/1 Running 0 59s 2026-06-10 06:48:59.453542 | controller | kube-system etcd-minikube 1/1 Running 0 64s 2026-06-10 06:48:59.453727 | controller | kube-system kindnet-ngpf2 1/1 Running 0 59s 2026-06-10 06:48:59.453904 | controller | kube-system kube-apiserver-minikube 1/1 Running 0 64s 2026-06-10 06:48:59.454067 | controller | kube-system kube-controller-manager-minikube 1/1 Running 0 64s 2026-06-10 06:48:59.454268 | controller | kube-system kube-ingress-dns-minikube 0/1 ImagePullBackOff 0 61s 2026-06-10 06:48:59.454457 | controller | kube-system kube-proxy-952fl 1/1 Running 0 59s 2026-06-10 06:48:59.454630 | controller | kube-system kube-scheduler-minikube 1/1 Running 0 66s 2026-06-10 06:48:59.454812 | controller | kube-system storage-provisioner 1/1 Running 0 61s 2026-06-10 10:49:00.166443 | controller | ok: Runtime: 0:00:00.844955 2026-06-10 10:49:00.169608 | 2026-06-10 10:49:00.169701 | PLAY RECAP 2026-06-10 10:49:00.169789 | controller | ok: 8 changed: 8 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 10:49:00.169831 | 2026-06-10 10:49:00.335482 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-minikube.yaml@master] 2026-06-10 10:49:00.336852 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/prepare-minikube.yaml@master] 2026-06-10 10:49:00.951762 | 2026-06-10 10:49:00.951889 | PLAY [controller] 2026-06-10 10:49:00.962169 | 2026-06-10 10:49:00.962239 | TASK [Crete sf namespace] 2026-06-10 06:49:01.635607 | controller | namespace/sf created 2026-06-10 10:49:01.991436 | controller | ok: Runtime: 0:00:00.195412 2026-06-10 10:49:01.997601 | 2026-06-10 10:49:01.997708 | TASK [Set sf as the default namespace] 2026-06-10 06:49:02.837063 | controller | Context "minikube" modified. 2026-06-10 10:49:03.030115 | controller | ok: Runtime: 0:00:00.150475 2026-06-10 10:49:03.032806 | 2026-06-10 10:49:03.032906 | PLAY RECAP 2026-06-10 10:49:03.033020 | controller | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 10:49:03.033067 | 2026-06-10 10:49:03.197972 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/prepare-minikube.yaml@master] 2026-06-10 10:49:03.199210 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-test-dependencies.yaml@master] 2026-06-10 10:49:03.913778 | 2026-06-10 10:49:03.913909 | PLAY [controller] 2026-06-10 10:49:03.925064 | 2026-06-10 10:49:03.925147 | TASK [install-golang : Check Go version] 2026-06-10 10:49:04.453490 | controller | ok: ERROR (ignored) 2026-06-10 10:49:04.454030 | controller | { 2026-06-10 10:49:04.454117 | controller | "failed_when_result": false, 2026-06-10 10:49:04.454176 | controller | "msg": "[Errno 2] No such file or directory: b'go'", 2026-06-10 10:49:04.454272 | controller | "rc": 2 2026-06-10 10:49:04.454320 | controller | } 2026-06-10 10:49:04.466966 | 2026-06-10 10:49:04.467110 | TASK [install-golang : Ensure a clean install] 2026-06-10 10:49:04.977427 | controller | ok 2026-06-10 10:49:04.983615 | 2026-06-10 10:49:04.983711 | TASK [install-golang : Extract Go archive] 2026-06-10 10:49:20.339683 | controller | changed 2026-06-10 10:49:20.343851 | 2026-06-10 10:49:20.343920 | LOOP [install-golang : Create symlinks for Go binary] 2026-06-10 10:49:20.576566 | controller | changed: "go" 2026-06-10 10:49:20.866670 | controller | changed: "gofmt" 2026-06-10 10:49:20.885838 | 2026-06-10 10:49:20.885981 | TASK [Installing packages] 2026-06-10 10:49:22.005958 | controller | ok: Nothing to do 2026-06-10 10:49:22.014396 | 2026-06-10 10:49:22.014526 | TASK [Install pip packages] 2026-06-10 10:49:25.520046 | controller | changed 2026-06-10 10:49:25.532899 | 2026-06-10 10:49:25.533161 | PLAY RECAP 2026-06-10 10:49:25.533294 | controller | ok: 6 changed: 3 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 10:49:25.533382 | 2026-06-10 10:49:25.700886 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-test-dependencies.yaml@master] 2026-06-10 10:49:25.703859 | RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-deploy.yaml@master] 2026-06-10 10:49:26.338502 | 2026-06-10 10:49:26.338636 | PLAY [controller] 2026-06-10 10:49:26.349081 | 2026-06-10 10:49:26.349172 | TASK [Run ./hack/deploy.sh] 2026-06-10 06:49:26.748143 | controller | [+] Installing kubectl 2026-06-10 06:49:26.800100 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 06:49:26.800235 | controller | Dload Upload Total Spent Left Speed 2026-06-10 06:49:27.128874 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 56.7M 100 56.7M 0 0 173M 0 --:--:-- --:--:-- --:--:-- 173M 2026-06-10 06:49:27.226144 | controller | go: downloading go1.25.11 (linux/amd64) 2026-06-10 06:49:34.921684 | controller | Error from server (NotFound): pods "gerrit-0" not found 2026-06-10 06:49:34.924118 | controller | [+] Setting up gerrit 2026-06-10 06:49:34.959965 | controller | go: downloading k8s.io/client-go v0.30.2 2026-06-10 06:49:34.960470 | controller | go: downloading github.com/spf13/cobra v1.8.1 2026-06-10 06:49:34.985241 | controller | go: downloading github.com/openshift/api v0.0.0-20240715171821-e9f09d21bcb5 2026-06-10 06:49:34.985782 | controller | go: downloading go.uber.org/zap v1.27.0 2026-06-10 06:49:34.986829 | controller | go: downloading k8s.io/api v0.30.2 2026-06-10 06:49:34.988118 | controller | go: downloading k8s.io/apimachinery v0.30.2 2026-06-10 06:49:36.134491 | controller | go: downloading k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 2026-06-10 06:49:36.139797 | controller | go: downloading sigs.k8s.io/controller-runtime v0.18.4 2026-06-10 06:49:36.429511 | controller | go: downloading gopkg.in/yaml.v3 v3.0.1 2026-06-10 06:49:36.501887 | controller | go: downloading sigs.k8s.io/yaml v1.4.0 2026-06-10 06:49:36.528186 | controller | go: downloading github.com/fatih/color v1.17.0 2026-06-10 06:49:36.529151 | controller | go: downloading github.com/go-sql-driver/mysql v1.8.1 2026-06-10 06:49:36.593110 | controller | go: downloading gopkg.in/ini.v1 v1.67.0 2026-06-10 06:49:36.594099 | controller | go: downloading k8s.io/kubectl v0.30.2 2026-06-10 06:49:36.623684 | controller | go: downloading github.com/spf13/pflag v1.0.5 2026-06-10 06:49:36.625887 | controller | go: downloading go.uber.org/multierr v1.11.0 2026-06-10 06:49:36.630652 | controller | go: downloading github.com/gogo/protobuf v1.3.2 2026-06-10 06:49:36.676788 | controller | go: downloading github.com/google/gofuzz v1.2.0 2026-06-10 06:49:36.693587 | controller | go: downloading k8s.io/klog/v2 v2.130.1 2026-06-10 06:49:36.741904 | controller | go: downloading github.com/go-logr/logr v1.4.2 2026-06-10 06:49:36.765711 | controller | go: downloading github.com/go-logr/zapr v1.3.0 2026-06-10 06:49:36.811521 | controller | go: downloading golang.org/x/exp v0.0.0-20240716175740-e3f259677ff7 2026-06-10 06:49:36.813114 | controller | go: downloading gopkg.in/yaml.v2 v2.4.0 2026-06-10 06:49:36.854716 | controller | go: downloading github.com/google/uuid v1.6.0 2026-06-10 06:49:36.903089 | controller | go: downloading golang.org/x/crypto v0.52.0 2026-06-10 06:49:36.904562 | controller | go: downloading k8s.io/apiextensions-apiserver v0.30.2 2026-06-10 06:49:37.111112 | controller | go: downloading gopkg.in/inf.v0 v0.9.1 2026-06-10 06:49:37.135351 | controller | go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.4.1 2026-06-10 06:49:37.212692 | controller | go: downloading github.com/imdario/mergo v0.3.16 2026-06-10 06:49:37.212907 | controller | go: downloading golang.org/x/term v0.43.0 2026-06-10 06:49:37.248474 | controller | go: downloading github.com/gorilla/websocket v1.5.1 2026-06-10 06:49:37.248913 | controller | go: downloading golang.org/x/net v0.55.0 2026-06-10 06:49:37.250858 | controller | go: downloading github.com/evanphx/json-patch/v5 v5.9.0 2026-06-10 06:49:37.320356 | controller | go: downloading github.com/mattn/go-colorable v0.1.13 2026-06-10 06:49:37.320983 | controller | go: downloading github.com/mattn/go-isatty v0.0.20 2026-06-10 06:49:37.334146 | controller | go: downloading github.com/evanphx/json-patch v5.9.0+incompatible 2026-06-10 06:49:37.363437 | controller | go: downloading filippo.io/edwards25519 v1.1.1 2026-06-10 06:49:37.408902 | controller | go: downloading golang.org/x/sys v0.45.0 2026-06-10 06:49:37.413700 | controller | go: downloading github.com/golang/protobuf v1.5.4 2026-06-10 06:49:37.459083 | controller | go: downloading github.com/google/gnostic-models v0.6.8 2026-06-10 06:49:37.463206 | controller | go: downloading golang.org/x/oauth2 v0.27.0 2026-06-10 06:49:37.468674 | controller | go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd 2026-06-10 06:49:37.480088 | controller | go: downloading golang.org/x/time v0.5.0 2026-06-10 06:49:37.508004 | controller | go: downloading k8s.io/kube-openapi v0.0.0-20240430033511-f0e62f92d13f 2026-06-10 06:49:37.619029 | controller | go: downloading github.com/json-iterator/go v1.1.12 2026-06-10 06:49:37.621365 | controller | go: downloading gomodules.xyz/jsonpatch/v2 v2.4.0 2026-06-10 06:49:37.622023 | controller | go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da 2026-06-10 06:49:37.639211 | controller | go: downloading github.com/prometheus/client_golang v1.18.0 2026-06-10 06:49:37.760935 | controller | go: downloading github.com/pkg/errors v0.9.1 2026-06-10 06:49:37.772099 | controller | go: downloading github.com/moby/spdystream v0.5.1 2026-06-10 06:49:37.796047 | controller | go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc 2026-06-10 06:49:37.841098 | controller | go: downloading google.golang.org/protobuf v1.34.1 2026-06-10 06:49:38.235969 | controller | go: downloading github.com/google/go-cmp v0.6.0 2026-06-10 06:49:38.236568 | controller | go: downloading github.com/fsnotify/fsnotify v1.7.0 2026-06-10 06:49:38.238379 | controller | go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd 2026-06-10 06:49:38.289161 | controller | go: downloading github.com/modern-go/reflect2 v1.0.2 2026-06-10 06:49:38.289451 | controller | go: downloading github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f 2026-06-10 06:49:38.289688 | controller | go: downloading github.com/prometheus/client_model v0.6.1 2026-06-10 06:49:38.292357 | controller | go: downloading github.com/prometheus/common v0.47.0 2026-06-10 06:49:38.294638 | controller | go: downloading golang.org/x/text v0.37.0 2026-06-10 06:49:38.429871 | controller | go: downloading github.com/beorn7/perks v1.0.1 2026-06-10 06:49:38.430885 | controller | go: downloading github.com/cespare/xxhash/v2 v2.3.0 2026-06-10 06:49:38.433673 | controller | go: downloading github.com/prometheus/procfs v0.15.0 2026-06-10 06:49:38.461938 | controller | go: downloading github.com/go-openapi/jsonreference v0.21.0 2026-06-10 06:49:38.463626 | controller | go: downloading github.com/go-openapi/swag v0.23.0 2026-06-10 06:49:38.573330 | controller | go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 2026-06-10 06:49:38.578018 | controller | go: downloading github.com/emicklei/go-restful/v3 v3.12.0 2026-06-10 06:49:38.603524 | controller | go: downloading github.com/go-openapi/jsonpointer v0.21.0 2026-06-10 06:49:38.613745 | controller | go: downloading github.com/mailru/easyjson v0.7.7 2026-06-10 06:49:38.763806 | controller | go: downloading github.com/josharian/intern v1.0.0 2026-06-10 06:50:17.881000 | controller | 2026-06-10T06:50:17-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:50:17.881118 | controller | 2026-06-10 06:50:17.892993 | controller | 2026-06-10T06:50:17-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:50:17.893053 | controller | 2026-06-10 06:50:18.112955 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "admin-ssh-key" in sf 2026-06-10 06:50:18.117664 | controller | 2026-06-10T06:50:18-04:00 INFO "admin-ssh-key" created 2026-06-10 06:50:18.500570 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "zuul-ssh-key" in sf 2026-06-10 06:50:18.504129 | controller | 2026-06-10T06:50:18-04:00 INFO "zuul-ssh-key" created 2026-06-10 06:50:18.506273 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "gerrit-admin-api-key" in sf 2026-06-10 06:50:18.509154 | controller | 2026-06-10T06:50:18-04:00 INFO "gerrit-admin-api-key" created 2026-06-10 06:50:18.513112 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "zuul-gerrit-api-key" in sf 2026-06-10 06:50:18.515160 | controller | 2026-06-10T06:50:18-04:00 INFO "zuul-gerrit-api-key" created 2026-06-10 06:50:18.516596 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "gerrit-httpd" in sf 2026-06-10 06:50:18.523073 | controller | 2026-06-10T06:50:18-04:00 INFO "gerrit-httpd" created 2026-06-10 06:50:18.524851 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "gerrit-sshd" in sf 2026-06-10 06:50:18.531686 | controller | 2026-06-10T06:50:18-04:00 INFO "gerrit-sshd" created 2026-06-10 06:50:18.534054 | controller | 2026-06-10T06:50:18-04:00 INFO managesf-resources 2026-06-10 06:50:18.534087 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "managesf-resources-config-map" in sf 2026-06-10 06:50:18.536717 | controller | 2026-06-10T06:50:18-04:00 INFO "managesf-resources-config-map" created 2026-06-10 06:50:18.541924 | controller | 2026-06-10T06:50:18-04:00 INFO managesf-resources-tooling 2026-06-10 06:50:18.541946 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "managesf-resources-tooling-config-map" in sf 2026-06-10 06:50:18.541972 | controller | 2026-06-10T06:50:18-04:00 INFO "managesf-resources-tooling-config-map" created 2026-06-10 06:50:18.543962 | controller | 2026-06-10T06:50:18-04:00 INFO Creating "gerrit" in sf 2026-06-10 06:50:18.549492 | controller | 2026-06-10T06:50:18-04:00 INFO "gerrit" created 2026-06-10 06:50:18.551432 | controller | 2026-06-10T06:50:18-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 06:50:28.557142 | controller | 2026-06-10T06:50:28-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 06:50:38.569762 | controller | 2026-06-10T06:50:38-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 06:50:48.575143 | controller | 2026-06-10T06:50:48-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 06:50:58.589714 | controller | 2026-06-10T06:50:58-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 06:51:08.595885 | controller | 2026-06-10T06:51:08-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 06:51:18.600196 | controller | 2026-06-10T06:51:18-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 06:51:28.619836 | controller | 2026-06-10T06:51:28-04:00 INFO Creating "post-init" in sf 2026-06-10 06:51:28.624280 | controller | 2026-06-10T06:51:28-04:00 INFO "post-init" created 2026-06-10 06:51:48.749322 | controller | 2026-06-10T06:51:48-04:00 INFO Creating "self-signed-cert" in sf 2026-06-10 06:51:48.757066 | controller | 2026-06-10T06:51:48-04:00 INFO "self-signed-cert" created 2026-06-10 06:51:48.762429 | controller | 2026-06-10T06:51:48-04:00 INFO Creating "gerrit-ingress" in sf 2026-06-10 06:51:48.780237 | controller | 2026-06-10T06:51:48-04:00 INFO "gerrit-ingress" created 2026-06-10 06:51:48.785496 | controller | 2026-06-10T06:51:48-04:00 INFO Creating "sf-ingress" in sf 2026-06-10 06:51:48.794369 | controller | 2026-06-10T06:51:48-04:00 INFO "sf-ingress" created 2026-06-10 06:51:48.796496 | controller | 2026-06-10T06:51:48-04:00 INFO Waiting for gerrit-ingress ... [attempt 1/25] 2026-06-10 06:51:55.805872 | controller | 2026-06-10T06:51:55-04:00 INFO Waiting for gerrit-ingress ... [attempt 2/25] 2026-06-10 06:52:02.811728 | controller | 2026-06-10T06:52:02-04:00 INFO Updating /etc/hosts 2026-06-10 06:52:02.811899 | controller | 2026-06-10T06:52:02-04:00 INFO Making sure Gerrit is up and ready... 2026-06-10 06:52:02.811912 | controller | 2026-06-10T06:52:02-04:00 INFO Querying Gerrit projects endpoint... [attempt 1/10] 2026-06-10 06:52:02.832209 | controller | 2026-06-10T06:52:02-04:00 INFO Gerrit is up and available 2026-06-10 06:52:02.832244 | controller | 2026-06-10T06:52:02-04:00 INFO Cloning config... 2026-06-10 06:52:02.835382 | controller | 2026-06-10T06:52:02-04:00 INFO Cloning repo https://admin:25bb43de-ff3a-49eb-9ba6-b4c388db94de@gerrit.sfop.me/a/config in deploy/config 2026-06-10 06:52:03.071505 | controller | 2026-06-10T06:52:03-04:00 INFO Configuring local repository for commits... 2026-06-10 06:52:03.080300 | controller | 2026-06-10T06:52:03-04:00 INFO Cloning demo-tenant-config... 2026-06-10 06:52:03.082887 | controller | 2026-06-10T06:52:03-04:00 INFO Cloning repo https://admin:25bb43de-ff3a-49eb-9ba6-b4c388db94de@gerrit.sfop.me/a/demo-tenant-config in deploy/demo-tenant-config 2026-06-10 06:52:03.185412 | controller | 2026-06-10T06:52:03-04:00 INFO Configuring local repository for commits... 2026-06-10 06:52:03.199398 | controller | 2026-06-10T06:52:03-04:00 INFO Cloning demo-project... 2026-06-10 06:52:03.202885 | controller | 2026-06-10T06:52:03-04:00 INFO Cloning repo https://admin:25bb43de-ff3a-49eb-9ba6-b4c388db94de@gerrit.sfop.me/a/demo-project in deploy/demo-project 2026-06-10 06:52:03.285807 | controller | 2026-06-10T06:52:03-04:00 INFO Configuring local repository for commits... 2026-06-10 06:52:03.295010 | controller | 2026-06-10T06:52:03-04:00 INFO Initialize demo-tenant's pipelines and jobs... 2026-06-10 06:52:03.301470 | controller | 2026-06-10T06:52:03-04:00 INFO Pushing local repo state to origin... 2026-06-10 06:52:03.458840 | controller | 2026-06-10T06:52:03-04:00 INFO Creating or updating tenant demo-tenant... 2026-06-10 06:52:03.466206 | controller | 2026-06-10T06:52:03-04:00 INFO Pushing local repo state to origin... 2026-06-10 06:52:03.557345 | controller | 2026-06-10T06:52:03-04:00 INFO Initialize demo-project's pipelines and jobs... 2026-06-10 06:52:03.565109 | controller | 2026-06-10T06:52:03-04:00 INFO Pushing local repo state to origin... 2026-06-10 06:52:03.679245 | controller | 2026-06-10T06:52:03-04:00 INFO Applying CRDs (did you run "make manifests" first?) 2026-06-10 06:52:04.021857 | controller | [+] Deploying sf-operator 2026-06-10 06:52:04.436695 | controller | 2026-06-10T06:52:04-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:52:04.436759 | controller | 2026-06-10 06:52:04.450957 | controller | 2026-06-10T06:52:04-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:52:04.451004 | controller | 2026-06-10 06:52:04.466715 | controller | 2026-06-10T06:52:04-04:00 INFO Restarting zookeeper prior to client cert generation... 2026-06-10 06:52:04.476475 | controller | 2026-06-10T06:52:04-04:00 INFO Deleting object, name: zookeeper 2026-06-10 06:52:04.481746 | controller | 2026-06-10T06:52:04-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:52:04.487300 | controller | 2026-06-10T06:52:04-04:00 INFO Creating secret, name: zuul-keystore-password 2026-06-10 06:52:04.487345 | controller | 2026-06-10T06:52:04-04:00 INFO Creating object, name: zuul-keystore-password 2026-06-10 06:52:04.492651 | controller | 2026-06-10T06:52:04-04:00 INFO Creating secret, name: zuul-auth-secret 2026-06-10 06:52:04.492763 | controller | 2026-06-10T06:52:04-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-10 06:52:04.498033 | controller | 2026-06-10T06:52:04-04:00 INFO Creating config map name: zuul-scheduler-tooling-config-map 2026-06-10 06:52:04.498132 | controller | 2026-06-10T06:52:04-04:00 INFO Creating object, name: zuul-scheduler-tooling-config-map 2026-06-10 06:52:04.505485 | controller | 2026-06-10T06:52:04-04:00 INFO Creating config map name: git-server-pi-config-map 2026-06-10 06:52:04.505545 | controller | 2026-06-10T06:52:04-04:00 INFO Creating object, name: git-server-pi-config-map 2026-06-10 06:52:04.515762 | controller | 2026-06-10T06:52:04-04:00 INFO Creating object, name: git-server 2026-06-10 06:52:04.522002 | controller | 2026-06-10T06:52:04-04:00 INFO Creating secret, name: mariadb-root-password 2026-06-10 06:52:04.522051 | controller | 2026-06-10T06:52:04-04:00 INFO Creating object, name: mariadb-root-password 2026-06-10 06:52:04.526250 | controller | 2026-06-10T06:52:04-04:00 INFO Creating secret, name: mariadb-config-secrets 2026-06-10 06:52:04.526294 | controller | 2026-06-10T06:52:04-04:00 INFO Creating object, name: mariadb-config-secrets 2026-06-10 06:52:04.869363 | controller | 2026-06-10T06:52:04-04:00 INFO Creating secret, name: mariadb-initdb-secrets 2026-06-10 06:52:04.869480 | controller | 2026-06-10T06:52:04-04:00 INFO Creating object, name: mariadb-initdb-secrets 2026-06-10 06:52:05.074191 | controller | 2026-06-10T06:52:05-04:00 INFO Creating object, name: mariadb 2026-06-10 06:52:05.268205 | controller | 2026-06-10T06:52:05-04:00 INFO Creating ssh key, name: logserver-keys 2026-06-10 06:52:05.859355 | controller | 2026-06-10T06:52:05-04:00 INFO Creating object, name: logserver-keys 2026-06-10 06:52:05.866524 | controller | 2026-06-10T06:52:05-04:00 INFO Creating ssh key, name: logserver-uploader-keys 2026-06-10 06:52:07.421835 | controller | 2026-06-10T06:52:07-04:00 INFO Creating object, name: logserver-uploader-keys 2026-06-10 06:52:07.434235 | controller | 2026-06-10T06:52:07-04:00 INFO Creating ssh key, name: logserver-uploader-spare-keys 2026-06-10 06:52:08.749908 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: logserver-uploader-spare-keys 2026-06-10 06:52:08.760964 | controller | 2026-06-10T06:52:08-04:00 INFO Creating config map name: logserver-config-map 2026-06-10 06:52:08.761023 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: logserver-config-map 2026-06-10 06:52:08.770567 | controller | 2026-06-10T06:52:08-04:00 INFO Creating service, name: logserver 2026-06-10 06:52:08.770606 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: logserver 2026-06-10 06:52:08.783582 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: logserver 2026-06-10 06:52:08.791999 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: gateway 2026-06-10 06:52:08.813348 | controller | 2026-06-10T06:52:08-04:00 INFO Creating config map name: gateway-config-map 2026-06-10 06:52:08.813406 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: gateway-config-map 2026-06-10 06:52:08.819749 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: gateway 2026-06-10 06:52:08.825536 | controller | 2026-06-10T06:52:08-04:00 INFO Creating service, name: hound-search 2026-06-10 06:52:08.825584 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: hound-search 2026-06-10 06:52:08.849653 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: hound-search 2026-06-10 06:52:08.857793 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: logjuicer-pvc 2026-06-10 06:52:08.862847 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: logjuicer 2026-06-10 06:52:08.880685 | controller | 2026-06-10T06:52:08-04:00 INFO Creating object, name: logjuicer 2026-06-10 06:52:08.885719 | controller | 2026-06-10T06:52:08-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-10 06:52:08.887206 | controller | 2026-06-10T06:52:08-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-10 06:52:10.334321 | controller | 2026-06-10T06:52:10-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-10 06:52:12.916223 | controller | 2026-06-10T06:52:12-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-10 06:52:12.922919 | controller | 2026-06-10T06:52:12-04:00 INFO Zookeeper certs were updated, nuking ZK clients... 2026-06-10 06:52:12.932650 | controller | 2026-06-10T06:52:12-04:00 INFO Creating config map name: zookeeper-pi-config-map 2026-06-10 06:52:12.932693 | controller | 2026-06-10T06:52:12-04:00 INFO Creating object, name: zookeeper-pi-config-map 2026-06-10 06:52:12.939374 | controller | 2026-06-10T06:52:12-04:00 INFO Creating service, name: zookeeper-headless 2026-06-10 06:52:12.939402 | controller | 2026-06-10T06:52:12-04:00 INFO Creating object, name: zookeeper-headless 2026-06-10 06:52:12.946044 | controller | 2026-06-10T06:52:12-04:00 INFO Creating service, name: zookeeper 2026-06-10 06:52:12.946088 | controller | 2026-06-10T06:52:12-04:00 INFO Creating object, name: zookeeper 2026-06-10 06:52:12.958082 | controller | 2026-06-10T06:52:12-04:00 INFO Creating PodDisruptionBudget, name: zookeeper 2026-06-10 06:52:12.958137 | controller | 2026-06-10T06:52:12-04:00 INFO Creating object, name: zookeeper 2026-06-10 06:52:12.976231 | controller | 2026-06-10T06:52:12-04:00 INFO Creating object, name: zookeeper 2026-06-10 06:52:12.981155 | controller | 2026-06-10T06:52:12-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-10 06:52:12.981244 | controller | 2026-06-10T06:52:12-04:00 INFO 2026-06-10 06:52:12.981253 | controller | - Config: Waiting ... 2026-06-10 06:52:12.981259 | controller | - Gateway: Waiting ... 2026-06-10 06:52:12.981265 | controller | - GitServer: Waiting ... 2026-06-10 06:52:12.981270 | controller | - HoundSearch: Waiting ... 2026-06-10 06:52:12.981275 | controller | - LogJuicer: Waiting ... 2026-06-10 06:52:12.981281 | controller | - Logserver: Waiting ... 2026-06-10 06:52:12.981286 | controller | - MariaDB: Waiting ... 2026-06-10 06:52:12.981291 | controller | - Zookeeper: Waiting ... 2026-06-10 06:52:12.981297 | controller | 2026-06-10 06:52:12.981314 | controller | 2026-06-10T06:52:12-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:52:17.986587 | controller | 2026-06-10T06:52:17-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:52:17.988327 | controller | 2026-06-10T06:52:17-04:00 INFO Nothing to clean up. 2026-06-10 06:52:18.007691 | controller | 2026-06-10T06:52:18-04:00 INFO Creating service, name: git-server 2026-06-10 06:52:18.007790 | controller | 2026-06-10T06:52:18-04:00 INFO Creating object, name: git-server 2026-06-10 06:52:18.016808 | controller | 2026-06-10T06:52:18-04:00 INFO Creating service, name: git-server-rw 2026-06-10 06:52:18.016906 | controller | 2026-06-10T06:52:18-04:00 INFO Creating object, name: git-server-rw 2026-06-10 06:52:18.028045 | controller | 2026-06-10T06:52:18-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 06:52:28.073972 | controller | 2026-06-10T06:52:28-04:00 INFO Creating service, name: mariadb 2026-06-10 06:52:28.074063 | controller | 2026-06-10T06:52:28-04:00 INFO Creating object, name: mariadb 2026-06-10 06:52:28.168392 | controller | 2026-06-10T06:52:28-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-10 06:52:56.280081 | controller | 2026-06-10T06:52:56-04:00 INFO Creating object, name: zuul-db-connection 2026-06-10 06:52:56.285345 | controller | 2026-06-10T06:52:56-04:00 INFO Creating object, name: zuul-db-provision 2026-06-10 06:52:56.316814 | controller | 2026-06-10T06:52:56-04:00 INFO Waiting for statefulset, name: logserver 2026-06-10 06:53:12.392544 | controller | 2026-06-10T06:53:12-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:53:12.552544 | controller | 2026-06-10T06:53:12-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:53:12.561549 | controller | 2026-06-10T06:53:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:53:12.561635 | controller | 2026-06-10T06:53:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:53:12.572060 | controller | 2026-06-10T06:53:12-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-10 06:53:42.692858 | controller | 2026-06-10T06:53:42-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-10 06:53:42.702518 | controller | 2026-06-10T06:53:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:53:42.702600 | controller | 2026-06-10T06:53:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:53:42.731333 | controller | 2026-06-10T06:53:42-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 06:53:42.731446 | controller | 2026-06-10T06:53:42-04:00 INFO 2026-06-10 06:53:42.731456 | controller | - Config: Waiting ... 2026-06-10 06:53:42.731463 | controller | - Gateway: OK 2026-06-10 06:53:42.731469 | controller | - GitServer: OK 2026-06-10 06:53:42.731475 | controller | - HoundSearch: Waiting ... 2026-06-10 06:53:42.731480 | controller | - LogJuicer: OK 2026-06-10 06:53:42.731486 | controller | - Logserver: OK 2026-06-10 06:53:42.731491 | controller | - MariaDB: Waiting ... 2026-06-10 06:53:42.731496 | controller | - Zookeeper: OK 2026-06-10 06:53:42.731502 | controller | 2026-06-10 06:53:42.731508 | controller | 2026-06-10T06:53:42-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:53:47.742786 | controller | 2026-06-10T06:53:47-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:53:47.745157 | controller | 2026-06-10T06:53:47-04:00 INFO Nothing to clean up. 2026-06-10 06:53:48.359755 | controller | 2026-06-10T06:53:48-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:53:48.359841 | controller | 2026-06-10T06:53:48-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:53:48.369777 | controller | 2026-06-10T06:53:48-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-10 06:54:18.487681 | controller | 2026-06-10T06:54:18-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-10 06:54:18.497437 | controller | 2026-06-10T06:54:18-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:54:18.497519 | controller | 2026-06-10T06:54:18-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:54:18.524773 | controller | 2026-06-10T06:54:18-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:54:18.527481 | controller | 2026-06-10T06:54:18-04:00 INFO Creating ssh key, name: zuul-spare-ssh-key 2026-06-10 06:54:18.632351 | controller | 2026-06-10T06:54:18-04:00 INFO Creating object, name: zuul-spare-ssh-key 2026-06-10 06:54:18.640607 | controller | 2026-06-10T06:54:18-04:00 INFO Creating config map name: np-statsd-config-map 2026-06-10 06:54:18.640691 | controller | 2026-06-10T06:54:18-04:00 INFO Creating object, name: np-statsd-config-map 2026-06-10 06:54:18.647016 | controller | 2026-06-10T06:54:18-04:00 INFO Creating config map name: nodepool-tooling-config-map 2026-06-10 06:54:18.647098 | controller | 2026-06-10T06:54:18-04:00 INFO Creating object, name: nodepool-tooling-config-map 2026-06-10 06:54:18.650607 | controller | 2026-06-10T06:54:18-04:00 INFO Creating config map name: nodepool-launcher-extra-config-config-map 2026-06-10 06:54:18.650662 | controller | 2026-06-10T06:54:18-04:00 INFO Creating object, name: nodepool-launcher-extra-config-config-map 2026-06-10 06:54:18.656105 | controller | 2026-06-10T06:54:18-04:00 INFO Creating object, name: nodepool-launcher 2026-06-10 06:54:18.662034 | controller | 2026-06-10T06:54:18-04:00 INFO Creating ssh key, name: nodepool-builder-ssh-key 2026-06-10 06:54:19.479268 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: nodepool-builder-ssh-key 2026-06-10 06:54:19.489858 | controller | 2026-06-10T06:54:19-04:00 INFO Creating config map name: nodepool-builder-extra-config-config-map 2026-06-10 06:54:19.489913 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: nodepool-builder-extra-config-config-map 2026-06-10 06:54:19.500678 | controller | 2026-06-10T06:54:19-04:00 INFO Creating service, name: nodepool-builder 2026-06-10 06:54:19.500769 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 06:54:19.514394 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 06:54:19.533323 | controller | 2026-06-10T06:54:19-04:00 INFO Creating secret, name: zuul-config 2026-06-10 06:54:19.533456 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: zuul-config 2026-06-10 06:54:19.543013 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:54:19.556105 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: zuul-executor-headless 2026-06-10 06:54:19.569271 | controller | 2026-06-10T06:54:19-04:00 INFO Creating config map name: zuul-extra-config-map 2026-06-10 06:54:19.569376 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: zuul-extra-config-map 2026-06-10 06:54:19.900877 | controller | 2026-06-10T06:54:19-04:00 INFO Creating config map name: zuul-statsd-config-map 2026-06-10 06:54:19.900982 | controller | 2026-06-10T06:54:19-04:00 INFO Creating object, name: zuul-statsd-config-map 2026-06-10 06:54:20.297199 | controller | 2026-06-10T06:54:20-04:00 INFO Creating config map name: zuul-logging-config-map 2026-06-10 06:54:20.297274 | controller | 2026-06-10T06:54:20-04:00 INFO Creating object, name: zuul-logging-config-map 2026-06-10 06:54:20.500573 | controller | 2026-06-10T06:54:20-04:00 INFO Creating object, name: zuul-scheduler 2026-06-10 06:54:21.300374 | controller | 2026-06-10T06:54:21-04:00 INFO Creating object, name: zuul-executor 2026-06-10 06:54:22.099776 | controller | 2026-06-10T06:54:22-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:54:22.901497 | controller | 2026-06-10T06:54:22-04:00 INFO Creating object, name: zuul-merger 2026-06-10 06:54:22.908096 | controller | 2026-06-10T06:54:22-04:00 INFO 2026-06-10 06:54:22.908220 | controller | - Config: Waiting ... 2026-06-10 06:54:22.908243 | controller | - Gateway: OK 2026-06-10 06:54:22.908256 | controller | - GitServer: OK 2026-06-10 06:54:22.908270 | controller | - HoundSearch: Waiting ... 2026-06-10 06:54:22.908282 | controller | - LogJuicer: OK 2026-06-10 06:54:22.908295 | controller | - Logserver: OK 2026-06-10 06:54:22.908307 | controller | - MariaDB: OK 2026-06-10 06:54:22.908320 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:54:22.908333 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:54:22.908346 | controller | - Zookeeper: OK 2026-06-10 06:54:22.908358 | controller | - Zuul: Waiting ... 2026-06-10 06:54:22.908371 | controller | - |_ Config Secrets: OK 2026-06-10 06:54:22.908383 | controller | - |_ Executor: Waiting ... 2026-06-10 06:54:22.908395 | controller | - |_ Merger: Waiting ... 2026-06-10 06:54:22.908408 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:54:22.908420 | controller | - |_ Web: Waiting ... 2026-06-10 06:54:22.908433 | controller | 2026-06-10 06:54:22.908445 | controller | 2026-06-10T06:54:22-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:54:27.918543 | controller | 2026-06-10T06:54:27-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:54:27.921133 | controller | 2026-06-10T06:54:27-04:00 INFO Nothing to clean up. 2026-06-10 06:54:28.536481 | controller | 2026-06-10T06:54:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:54:28.536607 | controller | 2026-06-10T06:54:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:54:28.548160 | controller | 2026-06-10T06:54:28-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-10 06:54:58.652013 | controller | 2026-06-10T06:54:58-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-10 06:54:58.666214 | controller | 2026-06-10T06:54:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:54:58.666301 | controller | 2026-06-10T06:54:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:54:58.695660 | controller | 2026-06-10T06:54:58-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:54:58.711375 | controller | 2026-06-10T06:54:58-04:00 INFO Creating object, name: nodepool-launcher 2026-06-10 06:54:58.717188 | controller | 2026-06-10T06:54:58-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:54:58.717245 | controller | 2026-06-10T06:54:58-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:54:59.283080 | controller | 2026-06-10T06:54:59-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:55:29.398427 | controller | 2026-06-10T06:55:29-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:55:29.413701 | controller | 2026-06-10T06:55:29-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-10 06:55:55.540979 | controller | 2026-06-10T06:55:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:55:55.541090 | controller | 2026-06-10T06:55:55-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:55:55.566619 | controller | 2026-06-10T06:55:55-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-10 06:55:57.573668 | controller | 2026-06-10T06:55:57-04:00 INFO 2026-06-10 06:55:57.573767 | controller | - Config: Waiting ... 2026-06-10 06:55:57.573778 | controller | - Gateway: OK 2026-06-10 06:55:57.573787 | controller | - GitServer: OK 2026-06-10 06:55:57.573796 | controller | - HoundSearch: Waiting ... 2026-06-10 06:55:57.573804 | controller | - LogJuicer: OK 2026-06-10 06:55:57.573811 | controller | - Logserver: OK 2026-06-10 06:55:57.573819 | controller | - MariaDB: OK 2026-06-10 06:55:57.573827 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:55:57.573835 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:55:57.573843 | controller | - Zookeeper: OK 2026-06-10 06:55:57.573851 | controller | - Zuul: Waiting ... 2026-06-10 06:55:57.573859 | controller | - |_ Config Secrets: OK 2026-06-10 06:55:57.573867 | controller | - |_ Executor: OK 2026-06-10 06:55:57.573875 | controller | - |_ Merger: OK 2026-06-10 06:55:57.573882 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:55:57.573890 | controller | - |_ Web: Waiting ... 2026-06-10 06:55:57.573898 | controller | 2026-06-10 06:55:57.573907 | controller | 2026-06-10T06:55:57-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:56:02.584657 | controller | 2026-06-10T06:56:02-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:56:02.586712 | controller | 2026-06-10T06:56:02-04:00 INFO Nothing to clean up. 2026-06-10 06:56:03.195824 | controller | 2026-06-10T06:56:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:56:03.195928 | controller | 2026-06-10T06:56:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:56:03.203080 | controller | 2026-06-10T06:56:03-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-10 06:56:33.332733 | controller | 2026-06-10T06:56:33-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-10 06:56:33.339950 | controller | 2026-06-10T06:56:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:56:33.339972 | controller | 2026-06-10T06:56:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:56:33.358445 | controller | 2026-06-10T06:56:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:56:33.371984 | controller | 2026-06-10T06:56:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:56:33.372115 | controller | 2026-06-10T06:56:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:56:33.955524 | controller | 2026-06-10T06:56:33-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 06:56:33.966679 | controller | 2026-06-10T06:56:33-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 06:56:35.542578 | controller | 2026-06-10T06:56:35-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:56:35.542632 | controller | 2026-06-10T06:56:35-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:56:36.349218 | controller | 2026-06-10T06:56:36-04:00 INFO 2026-06-10 06:56:36.349281 | controller | - Config: Waiting ... 2026-06-10 06:56:36.349288 | controller | - Gateway: OK 2026-06-10 06:56:36.349293 | controller | - GitServer: OK 2026-06-10 06:56:36.349299 | controller | - HoundSearch: Waiting ... 2026-06-10 06:56:36.349303 | controller | - LogJuicer: OK 2026-06-10 06:56:36.349308 | controller | - Logserver: OK 2026-06-10 06:56:36.349313 | controller | - MariaDB: OK 2026-06-10 06:56:36.349318 | controller | - NodePoolBuilder: OK 2026-06-10 06:56:36.349323 | controller | - NodePoolLauncher: OK 2026-06-10 06:56:36.349327 | controller | - Zookeeper: OK 2026-06-10 06:56:36.349332 | controller | - Zuul: Waiting ... 2026-06-10 06:56:36.349337 | controller | - |_ Config Secrets: OK 2026-06-10 06:56:36.349342 | controller | - |_ Executor: OK 2026-06-10 06:56:36.349347 | controller | - |_ Merger: OK 2026-06-10 06:56:36.349352 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:56:36.349357 | controller | - |_ Web: OK 2026-06-10 06:56:36.349362 | controller | 2026-06-10 06:56:36.349367 | controller | 2026-06-10T06:56:36-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 06:56:41.358357 | controller | 2026-06-10T06:56:41-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:56:41.359929 | controller | 2026-06-10T06:56:41-04:00 INFO Nothing to clean up. 2026-06-10 06:56:41.978081 | controller | 2026-06-10T06:56:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:56:41.978162 | controller | 2026-06-10T06:56:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:56:41.994055 | controller | 2026-06-10T06:56:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:56:41.994264 | controller | 2026-06-10T06:56:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:56:42.375449 | controller | 2026-06-10T06:56:42-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:56:42.767900 | controller | 2026-06-10T06:56:42-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:56:42.767949 | controller | 2026-06-10T06:56:42-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:56:43.779732 | controller | 2026-06-10T06:56:43-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:56:43.779781 | controller | 2026-06-10T06:56:43-04:00 INFO deployment rollout not ok, name: zuul-weeder 2026-06-10 06:56:44.762651 | controller | 2026-06-10T06:56:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:56:44.762726 | controller | 2026-06-10T06:56:44-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:56:45.570420 | controller | 2026-06-10T06:56:45-04:00 INFO 2026-06-10 06:56:45.570496 | controller | - Config: Waiting ... 2026-06-10 06:56:45.570512 | controller | - Gateway: OK 2026-06-10 06:56:45.570525 | controller | - GitServer: OK 2026-06-10 06:56:45.570536 | controller | - HoundSearch: OK 2026-06-10 06:56:45.570547 | controller | - LogJuicer: OK 2026-06-10 06:56:45.570557 | controller | - Logserver: OK 2026-06-10 06:56:45.570568 | controller | - MariaDB: OK 2026-06-10 06:56:45.570580 | controller | - NodePoolBuilder: OK 2026-06-10 06:56:45.570592 | controller | - NodePoolLauncher: OK 2026-06-10 06:56:45.570606 | controller | - Zookeeper: OK 2026-06-10 06:56:45.570618 | controller | - Zuul: Waiting ... 2026-06-10 06:56:45.570629 | controller | - |_ Config Secrets: OK 2026-06-10 06:56:45.570655 | controller | - |_ Executor: OK 2026-06-10 06:56:45.570666 | controller | - |_ Merger: OK 2026-06-10 06:56:45.570677 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:56:45.570690 | controller | - |_ Web: OK 2026-06-10 06:56:45.570703 | controller | 2026-06-10 06:56:45.570724 | controller | 2026-06-10T06:56:45-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 06:56:50.579029 | controller | 2026-06-10T06:56:50-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:56:50.580945 | controller | 2026-06-10T06:56:50-04:00 INFO Nothing to clean up. 2026-06-10 06:56:51.192201 | controller | 2026-06-10T06:56:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:56:51.192267 | controller | 2026-06-10T06:56:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:56:51.204832 | controller | 2026-06-10T06:56:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:56:51.204868 | controller | 2026-06-10T06:56:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:56:51.603689 | controller | 2026-06-10T06:56:51-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:56:51.990088 | controller | 2026-06-10T06:56:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:56:51.990232 | controller | 2026-06-10T06:56:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:56:53.003599 | controller | 2026-06-10T06:56:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:56:53.003659 | controller | 2026-06-10T06:56:53-04:00 INFO deployment rollout not ok, name: zuul-weeder 2026-06-10 06:56:53.983488 | controller | 2026-06-10T06:56:53-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:56:53.983549 | controller | 2026-06-10T06:56:53-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:56:54.792361 | controller | 2026-06-10T06:56:54-04:00 INFO 2026-06-10 06:56:54.792422 | controller | - Config: Waiting ... 2026-06-10 06:56:54.792429 | controller | - Gateway: OK 2026-06-10 06:56:54.792434 | controller | - GitServer: OK 2026-06-10 06:56:54.792439 | controller | - HoundSearch: OK 2026-06-10 06:56:54.792444 | controller | - LogJuicer: OK 2026-06-10 06:56:54.792449 | controller | - Logserver: OK 2026-06-10 06:56:54.792455 | controller | - MariaDB: OK 2026-06-10 06:56:54.792460 | controller | - NodePoolBuilder: OK 2026-06-10 06:56:54.792465 | controller | - NodePoolLauncher: OK 2026-06-10 06:56:54.792470 | controller | - Zookeeper: OK 2026-06-10 06:56:54.792475 | controller | - Zuul: Waiting ... 2026-06-10 06:56:54.792480 | controller | - |_ Config Secrets: OK 2026-06-10 06:56:54.792485 | controller | - |_ Executor: OK 2026-06-10 06:56:54.792490 | controller | - |_ Merger: OK 2026-06-10 06:56:54.792495 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:56:54.792500 | controller | - |_ Web: OK 2026-06-10 06:56:54.792528 | controller | 2026-06-10 06:56:54.792538 | controller | 2026-06-10T06:56:54-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-10 06:56:59.806120 | controller | 2026-06-10T06:56:59-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:56:59.807874 | controller | 2026-06-10T06:56:59-04:00 INFO Nothing to clean up. 2026-06-10 06:57:00.418813 | controller | 2026-06-10T06:57:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:57:00.418872 | controller | 2026-06-10T06:57:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:57:00.433871 | controller | 2026-06-10T06:57:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:57:00.433941 | controller | 2026-06-10T06:57:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:57:00.826510 | controller | 2026-06-10T06:57:00-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:57:01.213272 | controller | 2026-06-10T06:57:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:57:01.213334 | controller | 2026-06-10T06:57:01-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:57:02.244990 | controller | 2026-06-10T06:57:02-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:57:02.245121 | controller | 2026-06-10T06:57:02-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:57:03.206585 | controller | 2026-06-10T06:57:03-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:57:03.206646 | controller | 2026-06-10T06:57:03-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:57:04.205258 | controller | 2026-06-10T06:57:04-04:00 INFO Creating config map name: pymod-sf-operator-config-map 2026-06-10 06:57:04.205327 | controller | 2026-06-10T06:57:04-04:00 INFO Creating object, name: pymod-sf-operator-config-map 2026-06-10 06:57:04.607208 | controller | 2026-06-10T06:57:04-04:00 INFO Creating object, name: config-updater 2026-06-10 06:57:04.613940 | controller | 2026-06-10T06:57:04-04:00 INFO Creating object, name: config-updater-role 2026-06-10 06:57:04.618962 | controller | 2026-06-10T06:57:04-04:00 INFO Creating object, name: config-updater 2026-06-10 06:57:04.622842 | controller | 2026-06-10T06:57:04-04:00 INFO Creating the config-update service account secret 2026-06-10 06:57:04.622861 | controller | 2026-06-10T06:57:04-04:00 INFO Creating object, name: config-update-secrets 2026-06-10 06:57:04.627194 | controller | 2026-06-10T06:57:04-04:00 INFO Creating base secret job 2026-06-10 06:57:04.627587 | controller | 2026-06-10T06:57:04-04:00 INFO Creating object, name: config-base-secret-1-917-ns 2026-06-10 06:57:04.631978 | controller | 2026-06-10T06:57:04-04:00 INFO 2026-06-10 06:57:04.631996 | controller | - Config: Waiting ... 2026-06-10 06:57:04.632001 | controller | - Gateway: OK 2026-06-10 06:57:04.632005 | controller | - GitServer: OK 2026-06-10 06:57:04.632010 | controller | - HoundSearch: OK 2026-06-10 06:57:04.632014 | controller | - LogJuicer: OK 2026-06-10 06:57:04.632020 | controller | - Logserver: OK 2026-06-10 06:57:04.632024 | controller | - MariaDB: OK 2026-06-10 06:57:04.632030 | controller | - NodePoolBuilder: OK 2026-06-10 06:57:04.632035 | controller | - NodePoolLauncher: OK 2026-06-10 06:57:04.632039 | controller | - Zookeeper: OK 2026-06-10 06:57:04.632043 | controller | - Zuul: OK 2026-06-10 06:57:04.632048 | controller | 2026-06-10 06:57:04.632072 | controller | 2026-06-10T06:57:04-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-10 06:57:09.641125 | controller | 2026-06-10T06:57:09-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:57:09.643962 | controller | 2026-06-10T06:57:09-04:00 INFO Nothing to clean up. 2026-06-10 06:57:10.259584 | controller | 2026-06-10T06:57:10-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:57:10.259653 | controller | 2026-06-10T06:57:10-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:57:10.283897 | controller | 2026-06-10T06:57:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:57:10.283964 | controller | 2026-06-10T06:57:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:57:10.658057 | controller | 2026-06-10T06:57:10-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:57:11.061289 | controller | 2026-06-10T06:57:11-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:57:11.061380 | controller | 2026-06-10T06:57:11-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:57:12.075753 | controller | 2026-06-10T06:57:12-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:57:12.075861 | controller | 2026-06-10T06:57:12-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:57:13.045365 | controller | 2026-06-10T06:57:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:57:13.045445 | controller | 2026-06-10T06:57:13-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:57:14.250634 | controller | 2026-06-10T06:57:14-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 06:57:14.442336 | controller | 2026-06-10T06:57:14-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 06:57:19.690968 | controller | 2026-06-10T06:57:19-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:57:19.696093 | controller | 2026-06-10T06:57:19-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-10 06:57:19.699748 | controller | 2026-06-10T06:57:19-04:00 INFO 2026-06-10 06:57:19.699789 | controller | - Config: Waiting ... 2026-06-10 06:57:19.699795 | controller | - Gateway: OK 2026-06-10 06:57:19.699800 | controller | - GitServer: OK 2026-06-10 06:57:19.699805 | controller | - HoundSearch: OK 2026-06-10 06:57:19.699809 | controller | - LogJuicer: OK 2026-06-10 06:57:19.699814 | controller | - Logserver: OK 2026-06-10 06:57:19.699819 | controller | - MariaDB: OK 2026-06-10 06:57:19.699824 | controller | - NodePoolBuilder: OK 2026-06-10 06:57:19.699829 | controller | - NodePoolLauncher: OK 2026-06-10 06:57:19.699834 | controller | - Zookeeper: OK 2026-06-10 06:57:19.699838 | controller | - Zuul: OK 2026-06-10 06:57:19.699843 | controller | 2026-06-10 06:57:19.699848 | controller | 2026-06-10T06:57:19-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-10 06:57:24.711525 | controller | 2026-06-10T06:57:24-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:57:24.713221 | controller | 2026-06-10T06:57:24-04:00 INFO Nothing to clean up. 2026-06-10 06:57:25.328504 | controller | 2026-06-10T06:57:25-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:57:25.328578 | controller | 2026-06-10T06:57:25-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:57:25.341811 | controller | 2026-06-10T06:57:25-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:57:25.341879 | controller | 2026-06-10T06:57:25-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:57:25.735977 | controller | 2026-06-10T06:57:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:57:26.120624 | controller | 2026-06-10T06:57:26-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:57:26.120681 | controller | 2026-06-10T06:57:26-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:57:27.145789 | controller | 2026-06-10T06:57:27-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:57:27.145872 | controller | 2026-06-10T06:57:27-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:57:28.113724 | controller | 2026-06-10T06:57:28-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:57:28.923102 | controller | 2026-06-10T06:57:28-04:00 INFO 2026-06-10 06:57:28.923204 | controller | - Config: Waiting ... 2026-06-10 06:57:28.923215 | controller | - Gateway: OK 2026-06-10 06:57:28.923221 | controller | - GitServer: OK 2026-06-10 06:57:28.923226 | controller | - HoundSearch: OK 2026-06-10 06:57:28.923232 | controller | - LogJuicer: OK 2026-06-10 06:57:28.923237 | controller | - Logserver: OK 2026-06-10 06:57:28.923242 | controller | - MariaDB: OK 2026-06-10 06:57:28.923248 | controller | - NodePoolBuilder: OK 2026-06-10 06:57:28.923253 | controller | - NodePoolLauncher: OK 2026-06-10 06:57:28.923258 | controller | - Zookeeper: OK 2026-06-10 06:57:28.923264 | controller | - Zuul: Waiting ... 2026-06-10 06:57:28.923269 | controller | - |_ Config Secrets: OK 2026-06-10 06:57:28.923274 | controller | - |_ Executor: OK 2026-06-10 06:57:28.923280 | controller | - |_ Merger: OK 2026-06-10 06:57:28.923285 | controller | - |_ Scheduler: OK 2026-06-10 06:57:28.923290 | controller | - |_ Web: Waiting ... 2026-06-10 06:57:28.923296 | controller | 2026-06-10 06:57:28.923302 | controller | 2026-06-10T06:57:28-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-10 06:57:33.932274 | controller | 2026-06-10T06:57:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:57:33.934325 | controller | 2026-06-10T06:57:33-04:00 INFO Nothing to clean up. 2026-06-10 06:57:34.550691 | controller | 2026-06-10T06:57:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:57:34.550775 | controller | 2026-06-10T06:57:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:57:34.568233 | controller | 2026-06-10T06:57:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:57:34.568315 | controller | 2026-06-10T06:57:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:57:34.947720 | controller | 2026-06-10T06:57:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:57:35.342275 | controller | 2026-06-10T06:57:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:57:35.342384 | controller | 2026-06-10T06:57:35-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:57:36.358731 | controller | 2026-06-10T06:57:36-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:57:36.358803 | controller | 2026-06-10T06:57:36-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:57:37.334311 | controller | 2026-06-10T06:57:37-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:57:37.334426 | controller | 2026-06-10T06:57:37-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:57:38.146100 | controller | 2026-06-10T06:57:38-04:00 INFO 2026-06-10 06:57:38.146187 | controller | - Config: Waiting ... 2026-06-10 06:57:38.146198 | controller | - Gateway: OK 2026-06-10 06:57:38.146206 | controller | - GitServer: OK 2026-06-10 06:57:38.146213 | controller | - HoundSearch: OK 2026-06-10 06:57:38.146220 | controller | - LogJuicer: OK 2026-06-10 06:57:38.146228 | controller | - Logserver: OK 2026-06-10 06:57:38.146236 | controller | - MariaDB: OK 2026-06-10 06:57:38.146243 | controller | - NodePoolBuilder: OK 2026-06-10 06:57:38.146250 | controller | - NodePoolLauncher: OK 2026-06-10 06:57:38.146256 | controller | - Zookeeper: OK 2026-06-10 06:57:38.146263 | controller | - Zuul: Waiting ... 2026-06-10 06:57:38.146269 | controller | - |_ Config Secrets: OK 2026-06-10 06:57:38.146275 | controller | - |_ Executor: OK 2026-06-10 06:57:38.146281 | controller | - |_ Merger: OK 2026-06-10 06:57:38.146287 | controller | - |_ Scheduler: OK 2026-06-10 06:57:38.146294 | controller | - |_ Web: Waiting ... 2026-06-10 06:57:38.146300 | controller | 2026-06-10 06:57:38.146307 | controller | 2026-06-10T06:57:38-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-10 06:57:43.153738 | controller | 2026-06-10T06:57:43-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:57:43.155193 | controller | 2026-06-10T06:57:43-04:00 INFO Nothing to clean up. 2026-06-10 06:57:43.778571 | controller | 2026-06-10T06:57:43-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:57:43.778655 | controller | 2026-06-10T06:57:43-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:57:43.802501 | controller | 2026-06-10T06:57:43-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:57:43.802615 | controller | 2026-06-10T06:57:43-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:57:44.175637 | controller | 2026-06-10T06:57:44-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:57:44.569669 | controller | 2026-06-10T06:57:44-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:57:44.569744 | controller | 2026-06-10T06:57:44-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:57:45.588488 | controller | 2026-06-10T06:57:45-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:57:45.588564 | controller | 2026-06-10T06:57:45-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:57:46.559437 | controller | 2026-06-10T06:57:46-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:57:46.559519 | controller | 2026-06-10T06:57:46-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:57:47.375341 | controller | 2026-06-10T06:57:47-04:00 INFO 2026-06-10 06:57:47.375409 | controller | - Config: Waiting ... 2026-06-10 06:57:47.375418 | controller | - Gateway: OK 2026-06-10 06:57:47.375425 | controller | - GitServer: OK 2026-06-10 06:57:47.375432 | controller | - HoundSearch: OK 2026-06-10 06:57:47.375439 | controller | - LogJuicer: OK 2026-06-10 06:57:47.375446 | controller | - Logserver: OK 2026-06-10 06:57:47.375453 | controller | - MariaDB: OK 2026-06-10 06:57:47.375460 | controller | - NodePoolBuilder: OK 2026-06-10 06:57:47.375467 | controller | - NodePoolLauncher: OK 2026-06-10 06:57:47.375474 | controller | - Zookeeper: OK 2026-06-10 06:57:47.375481 | controller | - Zuul: Waiting ... 2026-06-10 06:57:47.375488 | controller | - |_ Config Secrets: OK 2026-06-10 06:57:47.375495 | controller | - |_ Executor: OK 2026-06-10 06:57:47.375512 | controller | - |_ Merger: OK 2026-06-10 06:57:47.375517 | controller | - |_ Scheduler: OK 2026-06-10 06:57:47.375522 | controller | - |_ Web: Waiting ... 2026-06-10 06:57:47.375528 | controller | 2026-06-10 06:57:47.375538 | controller | 2026-06-10T06:57:47-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-10 06:57:52.384372 | controller | 2026-06-10T06:57:52-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:57:52.386237 | controller | 2026-06-10T06:57:52-04:00 INFO Nothing to clean up. 2026-06-10 06:57:53.001520 | controller | 2026-06-10T06:57:53-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:57:53.001583 | controller | 2026-06-10T06:57:53-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:57:53.018769 | controller | 2026-06-10T06:57:53-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:57:53.018811 | controller | 2026-06-10T06:57:53-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:57:53.408141 | controller | 2026-06-10T06:57:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:57:53.801384 | controller | 2026-06-10T06:57:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:57:53.801459 | controller | 2026-06-10T06:57:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:57:54.818401 | controller | 2026-06-10T06:57:54-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:57:54.818512 | controller | 2026-06-10T06:57:54-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:57:55.788833 | controller | 2026-06-10T06:57:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:57:55.788956 | controller | 2026-06-10T06:57:55-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:57:56.595899 | controller | 2026-06-10T06:57:56-04:00 INFO 2026-06-10 06:57:56.595968 | controller | - Config: Waiting ... 2026-06-10 06:57:56.595979 | controller | - Gateway: OK 2026-06-10 06:57:56.595988 | controller | - GitServer: OK 2026-06-10 06:57:56.595996 | controller | - HoundSearch: OK 2026-06-10 06:57:56.596004 | controller | - LogJuicer: OK 2026-06-10 06:57:56.596012 | controller | - Logserver: OK 2026-06-10 06:57:56.596020 | controller | - MariaDB: OK 2026-06-10 06:57:56.596028 | controller | - NodePoolBuilder: OK 2026-06-10 06:57:56.596036 | controller | - NodePoolLauncher: OK 2026-06-10 06:57:56.596044 | controller | - Zookeeper: OK 2026-06-10 06:57:56.596052 | controller | - Zuul: Waiting ... 2026-06-10 06:57:56.596060 | controller | - |_ Config Secrets: OK 2026-06-10 06:57:56.596068 | controller | - |_ Executor: OK 2026-06-10 06:57:56.596076 | controller | - |_ Merger: OK 2026-06-10 06:57:56.596084 | controller | - |_ Scheduler: OK 2026-06-10 06:57:56.596092 | controller | - |_ Web: Waiting ... 2026-06-10 06:57:56.596100 | controller | 2026-06-10 06:57:56.596158 | controller | 2026-06-10T06:57:56-04:00 INFO [attempt #13] Waiting 5s for the next reconcile call ... 2026-06-10 06:58:01.602040 | controller | 2026-06-10T06:58:01-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:58:01.604043 | controller | 2026-06-10T06:58:01-04:00 INFO Nothing to clean up. 2026-06-10 06:58:02.227346 | controller | 2026-06-10T06:58:02-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:58:02.227484 | controller | 2026-06-10T06:58:02-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:58:02.252798 | controller | 2026-06-10T06:58:02-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:58:02.252895 | controller | 2026-06-10T06:58:02-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:58:02.630953 | controller | 2026-06-10T06:58:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:58:03.022633 | controller | 2026-06-10T06:58:03-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:58:03.022780 | controller | 2026-06-10T06:58:03-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:58:04.033868 | controller | 2026-06-10T06:58:04-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:58:04.033981 | controller | 2026-06-10T06:58:04-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:58:05.006347 | controller | 2026-06-10T06:58:05-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:58:05.006449 | controller | 2026-06-10T06:58:05-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:58:05.813420 | controller | 2026-06-10T06:58:05-04:00 INFO 2026-06-10 06:58:05.813482 | controller | - Config: Waiting ... 2026-06-10 06:58:05.813499 | controller | - Gateway: OK 2026-06-10 06:58:05.813506 | controller | - GitServer: OK 2026-06-10 06:58:05.813511 | controller | - HoundSearch: OK 2026-06-10 06:58:05.813516 | controller | - LogJuicer: OK 2026-06-10 06:58:05.813520 | controller | - Logserver: OK 2026-06-10 06:58:05.813524 | controller | - MariaDB: OK 2026-06-10 06:58:05.813528 | controller | - NodePoolBuilder: OK 2026-06-10 06:58:05.813533 | controller | - NodePoolLauncher: OK 2026-06-10 06:58:05.813537 | controller | - Zookeeper: OK 2026-06-10 06:58:05.813541 | controller | - Zuul: Waiting ... 2026-06-10 06:58:05.813545 | controller | - |_ Config Secrets: OK 2026-06-10 06:58:05.813549 | controller | - |_ Executor: OK 2026-06-10 06:58:05.813554 | controller | - |_ Merger: OK 2026-06-10 06:58:05.813558 | controller | - |_ Scheduler: OK 2026-06-10 06:58:05.813562 | controller | - |_ Web: Waiting ... 2026-06-10 06:58:05.813566 | controller | 2026-06-10 06:58:05.813595 | controller | 2026-06-10T06:58:05-04:00 INFO [attempt #14] Waiting 5s for the next reconcile call ... 2026-06-10 06:58:10.822522 | controller | 2026-06-10T06:58:10-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:58:10.824921 | controller | 2026-06-10T06:58:10-04:00 INFO Nothing to clean up. 2026-06-10 06:58:11.435592 | controller | 2026-06-10T06:58:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:58:11.435659 | controller | 2026-06-10T06:58:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:58:11.447757 | controller | 2026-06-10T06:58:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:58:11.447836 | controller | 2026-06-10T06:58:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:58:11.847257 | controller | 2026-06-10T06:58:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:58:12.234958 | controller | 2026-06-10T06:58:12-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:58:12.235063 | controller | 2026-06-10T06:58:12-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:58:13.253453 | controller | 2026-06-10T06:58:13-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:58:13.253530 | controller | 2026-06-10T06:58:13-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:58:14.227237 | controller | 2026-06-10T06:58:14-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:58:14.227306 | controller | 2026-06-10T06:58:14-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:58:15.036373 | controller | 2026-06-10T06:58:15-04:00 INFO 2026-06-10 06:58:15.036433 | controller | - Config: Waiting ... 2026-06-10 06:58:15.036438 | controller | - Gateway: OK 2026-06-10 06:58:15.036443 | controller | - GitServer: OK 2026-06-10 06:58:15.036447 | controller | - HoundSearch: OK 2026-06-10 06:58:15.036451 | controller | - LogJuicer: OK 2026-06-10 06:58:15.036454 | controller | - Logserver: OK 2026-06-10 06:58:15.036458 | controller | - MariaDB: OK 2026-06-10 06:58:15.036461 | controller | - NodePoolBuilder: OK 2026-06-10 06:58:15.036465 | controller | - NodePoolLauncher: OK 2026-06-10 06:58:15.036468 | controller | - Zookeeper: OK 2026-06-10 06:58:15.036472 | controller | - Zuul: Waiting ... 2026-06-10 06:58:15.036475 | controller | - |_ Config Secrets: OK 2026-06-10 06:58:15.036479 | controller | - |_ Executor: OK 2026-06-10 06:58:15.036483 | controller | - |_ Merger: OK 2026-06-10 06:58:15.036486 | controller | - |_ Scheduler: OK 2026-06-10 06:58:15.036490 | controller | - |_ Web: Waiting ... 2026-06-10 06:58:15.036493 | controller | 2026-06-10 06:58:15.036497 | controller | 2026-06-10T06:58:15-04:00 INFO [attempt #15] Waiting 5s for the next reconcile call ... 2026-06-10 06:58:20.045283 | controller | 2026-06-10T06:58:20-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:58:20.047201 | controller | 2026-06-10T06:58:20-04:00 INFO Nothing to clean up. 2026-06-10 06:58:20.659817 | controller | 2026-06-10T06:58:20-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:58:20.659920 | controller | 2026-06-10T06:58:20-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:58:20.676897 | controller | 2026-06-10T06:58:20-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:58:20.677018 | controller | 2026-06-10T06:58:20-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:58:21.071899 | controller | 2026-06-10T06:58:21-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:58:21.458621 | controller | 2026-06-10T06:58:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:58:21.458707 | controller | 2026-06-10T06:58:21-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:58:22.473629 | controller | 2026-06-10T06:58:22-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:58:22.473734 | controller | 2026-06-10T06:58:22-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:58:23.449204 | controller | 2026-06-10T06:58:23-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:58:23.449278 | controller | 2026-06-10T06:58:23-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:58:24.262195 | controller | 2026-06-10T06:58:24-04:00 INFO 2026-06-10 06:58:24.262263 | controller | - Config: Waiting ... 2026-06-10 06:58:24.262270 | controller | - Gateway: OK 2026-06-10 06:58:24.262276 | controller | - GitServer: OK 2026-06-10 06:58:24.262281 | controller | - HoundSearch: OK 2026-06-10 06:58:24.262285 | controller | - LogJuicer: OK 2026-06-10 06:58:24.262290 | controller | - Logserver: OK 2026-06-10 06:58:24.262295 | controller | - MariaDB: OK 2026-06-10 06:58:24.262301 | controller | - NodePoolBuilder: OK 2026-06-10 06:58:24.262306 | controller | - NodePoolLauncher: OK 2026-06-10 06:58:24.262311 | controller | - Zookeeper: OK 2026-06-10 06:58:24.262316 | controller | - Zuul: Waiting ... 2026-06-10 06:58:24.262320 | controller | - |_ Config Secrets: OK 2026-06-10 06:58:24.262325 | controller | - |_ Executor: OK 2026-06-10 06:58:24.262330 | controller | - |_ Merger: OK 2026-06-10 06:58:24.262335 | controller | - |_ Scheduler: OK 2026-06-10 06:58:24.262340 | controller | - |_ Web: Waiting ... 2026-06-10 06:58:24.262345 | controller | 2026-06-10 06:58:24.262395 | controller | 2026-06-10T06:58:24-04:00 INFO [attempt #16] Waiting 5s for the next reconcile call ... 2026-06-10 06:58:29.269005 | controller | 2026-06-10T06:58:29-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:58:29.271786 | controller | 2026-06-10T06:58:29-04:00 INFO Nothing to clean up. 2026-06-10 06:58:29.880776 | controller | 2026-06-10T06:58:29-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:58:29.880844 | controller | 2026-06-10T06:58:29-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:58:29.897369 | controller | 2026-06-10T06:58:29-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:58:29.897457 | controller | 2026-06-10T06:58:29-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:58:30.296232 | controller | 2026-06-10T06:58:30-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:58:30.688402 | controller | 2026-06-10T06:58:30-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:58:30.688500 | controller | 2026-06-10T06:58:30-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:58:31.694608 | controller | 2026-06-10T06:58:31-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:58:31.694686 | controller | 2026-06-10T06:58:31-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:58:32.675531 | controller | 2026-06-10T06:58:32-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:58:32.675654 | controller | 2026-06-10T06:58:32-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:58:33.882123 | controller | 2026-06-10T06:58:33-04:00 INFO 2026-06-10 06:58:33.882193 | controller | - Config: OK 2026-06-10 06:58:33.882203 | controller | - Gateway: OK 2026-06-10 06:58:33.882209 | controller | - GitServer: OK 2026-06-10 06:58:33.882214 | controller | - HoundSearch: OK 2026-06-10 06:58:33.882218 | controller | - LogJuicer: OK 2026-06-10 06:58:33.882224 | controller | - Logserver: OK 2026-06-10 06:58:33.882229 | controller | - MariaDB: OK 2026-06-10 06:58:33.882234 | controller | - NodePoolBuilder: OK 2026-06-10 06:58:33.882239 | controller | - NodePoolLauncher: OK 2026-06-10 06:58:33.882245 | controller | - Zookeeper: OK 2026-06-10 06:58:33.882250 | controller | - Zuul: OK 2026-06-10 06:58:33.882255 | controller | 2026-06-10 06:58:33.882261 | controller | 2026-06-10T06:58:33-04:00 INFO Updating controller configmap ... 2026-06-10 06:58:34.271460 | controller | 2026-06-10T06:58:34-04:00 INFO Standalone reconcile done. 2026-06-10 10:58:34.587637 | controller | ok: Runtime: 0:09:07.538004 2026-06-10 10:58:34.600137 | 2026-06-10 10:58:34.600232 | PLAY RECAP 2026-06-10 10:58:34.600313 | controller | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 10:58:34.600406 | 2026-06-10 10:58:34.736192 | RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-deploy.yaml@master] 2026-06-10 10:58:34.738691 | RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-tests.yaml@master] 2026-06-10 10:58:35.483900 | 2026-06-10 10:58:35.484029 | PLAY [controller] 2026-06-10 10:58:35.518374 | 2026-06-10 10:58:35.518468 | TASK [Install test dependencies] 2026-06-10 10:58:36.755590 | controller | ok: Nothing to do 2026-06-10 10:58:36.765257 | 2026-06-10 10:58:36.765442 | TASK [setup-variables : Include global variables] 2026-06-10 10:58:36.813220 | controller | skipping: Conditional result was False 2026-06-10 10:58:36.822677 | 2026-06-10 10:58:36.822823 | TASK [setup-variables : Include zuul variables if zuul.executor is defined] 2026-06-10 10:58:36.887702 | controller | ok 2026-06-10 10:58:36.906416 | 2026-06-10 10:58:36.906568 | TASK [setup-nodepool-ns : Set up nodepool namespace] 2026-06-10 06:58:37.733507 | controller | 2026-06-10T06:58:37-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:58:37.733578 | controller | 2026-06-10 06:58:37.741550 | controller | 2026-06-10T06:58:37-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:58:37.741577 | controller | 2026-06-10 06:58:37.742236 | controller | 2026-06-10T06:58:37-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:58:37.742249 | controller | 2026-06-10 06:58:37.744691 | controller | 2026-06-10T06:58:37-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:58:37.744703 | controller | 2026-06-10 06:58:37.747342 | controller | 2026-06-10T06:58:37-04:00 INFO Creating "nodepool" in nodepool 2026-06-10 06:58:37.748965 | controller | 2026-06-10T06:58:37-04:00 INFO "nodepool" created 2026-06-10 06:58:37.750394 | controller | 2026-06-10T06:58:37-04:00 INFO Creating "nodepool-sa" in nodepool 2026-06-10 06:58:37.751860 | controller | 2026-06-10T06:58:37-04:00 INFO "nodepool-sa" created 2026-06-10 06:58:37.754128 | controller | 2026-06-10T06:58:37-04:00 INFO Creating "nodepool-role" in nodepool 2026-06-10 06:58:37.755637 | controller | 2026-06-10T06:58:37-04:00 INFO "nodepool-role" created 2026-06-10 06:58:37.756939 | controller | 2026-06-10T06:58:37-04:00 INFO Creating "nodepool-rb" in nodepool 2026-06-10 06:58:37.758805 | controller | 2026-06-10T06:58:37-04:00 INFO "nodepool-rb" created 2026-06-10 06:58:37.760437 | controller | 2026-06-10T06:58:37-04:00 INFO Creating "nodepool-token" in nodepool 2026-06-10 06:58:37.761997 | controller | 2026-06-10T06:58:37-04:00 INFO "nodepool-token" created 2026-06-10 06:58:38.767594 | controller | 2026-06-10T06:58:38-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:58:38.767661 | controller | 2026-06-10 06:58:38.772355 | controller | 2026-06-10T06:58:38-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:58:38.772390 | controller | 2026-06-10 06:58:38.778603 | controller | 2026-06-10T06:58:38-04:00 INFO Creating "nodepool-providers-secrets" in sf 2026-06-10 06:58:38.781702 | controller | 2026-06-10T06:58:38-04:00 INFO "nodepool-providers-secrets" created 2026-06-10 10:58:38.950314 | controller | ok: Runtime: 0:00:01.391798 2026-06-10 10:58:38.968445 | 2026-06-10 10:58:38.968635 | TASK [utils/get-gerrit-admin-user-api-key : Get Gerrit admin user API key] 2026-06-10 10:58:39.510531 | controller | ok: Runtime: 0:00:00.071105 2026-06-10 10:58:39.517220 | 2026-06-10 10:58:39.517303 | TASK [utils/get-gerrit-admin-user-api-key : Set fact for _gerrit_admin_api_key] 2026-06-10 10:58:39.561057 | controller | ok 2026-06-10 10:58:39.572594 | 2026-06-10 10:58:39.572680 | LOOP [utils/config-repo-reset : Config repo, reset] 2026-06-10 10:58:40.042819 | controller | Fetching origin 2026-06-10 10:58:40.043229 | controller | Fetching gerrit 2026-06-10 10:58:40.043287 | controller | From https://gerrit.sfop.me/a/config 2026-06-10 10:58:40.043328 | controller | * [new branch] master -> gerrit/master 2026-06-10 10:58:40.043447 | controller | ok: Item: git fetch --all Runtime: 0:00:00.133083 2026-06-10 10:58:40.325877 | controller | Already on 'master' 2026-06-10 10:58:40.326012 | controller | Your branch is up to date with 'origin/master'. 2026-06-10 10:58:40.326051 | controller | ok: Item: git checkout master Runtime: 0:00:00.006048 2026-06-10 10:58:40.488590 | controller | HEAD is now at d1d7925 Automatic update 2026-06-10 10:58:40.488725 | controller | ok: Item: git reset --hard origin/master -- Runtime: 0:00:00.008019 2026-06-10 10:58:40.701309 | controller | ok: Item: git clean -f -d Runtime: 0:00:00.005102 2026-06-10 10:58:40.724775 | 2026-06-10 10:58:40.725002 | TASK [utils/setup-log-forwarding-stack : Deploy Loki via included role] 2026-06-10 10:58:40.761923 | controller | ok 2026-06-10 10:58:40.775616 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/deploy-loki.yaml 2026-06-10 10:58:40.781706 | 2026-06-10 10:58:40.781793 | TASK [utils/setup-log-forwarding-stack : Deploy Loki] 2026-06-10 06:58:41.290656 | controller | configmap/loki-config created 2026-06-10 06:58:41.299412 | controller | deployment.apps/test-loki created 2026-06-10 06:58:41.324246 | controller | service/loki-http created 2026-06-10 10:58:41.819307 | controller | ok: Runtime: 0:00:00.307108 2026-06-10 10:58:41.828052 | 2026-06-10 10:58:41.828217 | TASK [utils/setup-log-forwarding-stack : Wait for Loki deployment to be ready] 2026-06-10 10:58:52.502979 | controller | ok: Runtime: 0:00:00.071366 2026-06-10 10:58:52.512710 | 2026-06-10 10:58:52.512875 | TASK [utils/setup-log-forwarding-stack : Deploy Fluent Bit via included role] 2026-06-10 10:58:52.546016 | controller | ok 2026-06-10 10:58:52.558126 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/deploy-fluentbit.yaml 2026-06-10 10:58:52.569311 | 2026-06-10 10:58:52.569478 | TASK [utils/setup-log-forwarding-stack : Deploy Fluent Bit] 2026-06-10 06:58:53.124495 | controller | configmap/test-fluentbit-config created 2026-06-10 06:58:53.133919 | controller | pod/test-fluentbit created 2026-06-10 06:58:53.152270 | controller | service/fluent-bit-forward-input created 2026-06-10 10:58:53.608510 | controller | ok: Runtime: 0:00:00.330875 2026-06-10 10:58:53.617888 | 2026-06-10 10:58:53.618022 | TASK [utils/setup-log-forwarding-stack : Wait for Fluent Bit pod to be ready] 2026-06-10 10:59:04.276949 | controller | ok: Runtime: 0:00:00.066678 2026-06-10 10:59:04.286278 | 2026-06-10 10:59:04.286440 | TASK [utils/setup-log-forwarding-stack : Install LogCLI] 2026-06-10 10:59:04.310066 | controller | ok 2026-06-10 10:59:04.324154 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/install-logCLI.yaml 2026-06-10 10:59:04.335834 | 2026-06-10 10:59:04.335955 | TASK [utils/setup-log-forwarding-stack : check if logCLI is present] 2026-06-10 10:59:04.745239 | controller | ok 2026-06-10 10:59:04.754240 | 2026-06-10 10:59:04.754393 | TASK [utils/setup-log-forwarding-stack : download and unzip logCLI] 2026-06-10 10:59:06.563215 | controller | changed 2026-06-10 10:59:06.572647 | 2026-06-10 10:59:06.572779 | TASK [utils/setup-log-forwarding-stack : ensure ~/bin exists] 2026-06-10 10:59:07.025395 | controller | changed 2026-06-10 10:59:07.035328 | 2026-06-10 10:59:07.035531 | TASK [utils/setup-log-forwarding-stack : move logCLI to ~/bin/] 2026-06-10 10:59:07.581021 | controller | ok: Runtime: 0:00:00.003960 2026-06-10 10:59:07.599948 | 2026-06-10 10:59:07.600076 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:59:07.878323 | controller | ok 2026-06-10 10:59:07.887096 | 2026-06-10 10:59:07.887223 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:59:08.146556 | controller | ok 2026-06-10 10:59:08.154703 | 2026-06-10 10:59:08.154828 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:59:08.740174 | controller | 2026-06-10T06:59:08-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:59:08.740259 | controller | 2026-06-10 06:59:08.752374 | controller | 2026-06-10T06:59:08-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:59:08.752403 | controller | 2026-06-10 06:59:08.766603 | controller | 2026-06-10T06:59:08-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:59:08.768513 | controller | 2026-06-10T06:59:08-04:00 INFO Nothing to clean up. 2026-06-10 06:59:09.393688 | controller | 2026-06-10T06:59:09-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:59:09.506667 | controller | 2026-06-10T06:59:09-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:59:09.513531 | controller | 2026-06-10T06:59:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:59:09.513609 | controller | 2026-06-10T06:59:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:59:09.533892 | controller | 2026-06-10T06:59:09-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:59:09.533945 | controller | 2026-06-10T06:59:09-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:59:09.793905 | controller | 2026-06-10T06:59:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:59:10.178708 | controller | 2026-06-10T06:59:10-04:00 INFO Creating object, name: zuul-capacity 2026-06-10 06:59:10.188429 | controller | 2026-06-10T06:59:10-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-10 06:59:10.188505 | controller | 2026-06-10T06:59:10-04:00 INFO Updating object, name: nodepool-launcher 2026-06-10 06:59:10.387552 | controller | 2026-06-10T06:59:10-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-10 06:59:10.387627 | controller | 2026-06-10T06:59:10-04:00 INFO Updating object, name: nodepool-builder 2026-06-10 06:59:11.193795 | controller | 2026-06-10T06:59:11-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:59:11.193847 | controller | 2026-06-10T06:59:11-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:59:12.370023 | controller | 2026-06-10T06:59:12-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:59:12.370072 | controller | 2026-06-10T06:59:12-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:59:13.580241 | controller | 2026-06-10T06:59:13-04:00 INFO 2026-06-10 06:59:13.580305 | controller | - Config: OK 2026-06-10 06:59:13.580319 | controller | - Gateway: OK 2026-06-10 06:59:13.580328 | controller | - GitServer: OK 2026-06-10 06:59:13.580337 | controller | - HoundSearch: OK 2026-06-10 06:59:13.580346 | controller | - LogJuicer: OK 2026-06-10 06:59:13.580355 | controller | - Logserver: OK 2026-06-10 06:59:13.580363 | controller | - MariaDB: OK 2026-06-10 06:59:13.580372 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:59:13.580381 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:59:13.580390 | controller | - Zookeeper: OK 2026-06-10 06:59:13.580398 | controller | - Zuul: OK 2026-06-10 06:59:13.580407 | controller | 2026-06-10 06:59:13.580417 | controller | 2026-06-10T06:59:13-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:59:18.589414 | controller | 2026-06-10T06:59:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:59:18.591614 | controller | 2026-06-10T06:59:18-04:00 INFO Nothing to clean up. 2026-06-10 06:59:19.206888 | controller | 2026-06-10T06:59:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:59:19.206956 | controller | 2026-06-10T06:59:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:59:19.245476 | controller | 2026-06-10T06:59:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:59:19.245540 | controller | 2026-06-10T06:59:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:59:19.617339 | controller | 2026-06-10T06:59:19-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:59:20.007347 | controller | 2026-06-10T06:59:20-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:59:20.007417 | controller | 2026-06-10T06:59:20-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:59:21.017784 | controller | 2026-06-10T06:59:21-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:59:21.017834 | controller | 2026-06-10T06:59:21-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:59:21.995737 | controller | 2026-06-10T06:59:21-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:59:21.995819 | controller | 2026-06-10T06:59:21-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:59:23.201670 | controller | 2026-06-10T06:59:23-04:00 INFO 2026-06-10 06:59:23.201730 | controller | - Config: OK 2026-06-10 06:59:23.201737 | controller | - Gateway: OK 2026-06-10 06:59:23.201742 | controller | - GitServer: OK 2026-06-10 06:59:23.201747 | controller | - HoundSearch: OK 2026-06-10 06:59:23.201752 | controller | - LogJuicer: OK 2026-06-10 06:59:23.201757 | controller | - Logserver: OK 2026-06-10 06:59:23.201762 | controller | - MariaDB: OK 2026-06-10 06:59:23.201768 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:59:23.201774 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:59:23.201779 | controller | - Zookeeper: OK 2026-06-10 06:59:23.201784 | controller | - Zuul: OK 2026-06-10 06:59:23.201789 | controller | 2026-06-10 06:59:23.201795 | controller | 2026-06-10T06:59:23-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:59:28.210447 | controller | 2026-06-10T06:59:28-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:59:28.213310 | controller | 2026-06-10T06:59:28-04:00 INFO Nothing to clean up. 2026-06-10 06:59:28.827133 | controller | 2026-06-10T06:59:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:59:28.827226 | controller | 2026-06-10T06:59:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:59:28.851936 | controller | 2026-06-10T06:59:28-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:59:28.851979 | controller | 2026-06-10T06:59:28-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:59:29.235526 | controller | 2026-06-10T06:59:29-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:59:29.634075 | controller | 2026-06-10T06:59:29-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:59:29.634118 | controller | 2026-06-10T06:59:29-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:59:30.641964 | controller | 2026-06-10T06:59:30-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:59:30.642014 | controller | 2026-06-10T06:59:30-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:59:31.617352 | controller | 2026-06-10T06:59:31-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:59:31.617410 | controller | 2026-06-10T06:59:31-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:59:32.820940 | controller | 2026-06-10T06:59:32-04:00 INFO 2026-06-10 06:59:32.820975 | controller | - Config: OK 2026-06-10 06:59:32.820980 | controller | - Gateway: OK 2026-06-10 06:59:32.820984 | controller | - GitServer: OK 2026-06-10 06:59:32.820988 | controller | - HoundSearch: OK 2026-06-10 06:59:32.820991 | controller | - LogJuicer: OK 2026-06-10 06:59:32.820995 | controller | - Logserver: OK 2026-06-10 06:59:32.820999 | controller | - MariaDB: OK 2026-06-10 06:59:32.821003 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:59:32.821006 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:59:32.821010 | controller | - Zookeeper: OK 2026-06-10 06:59:32.821014 | controller | - Zuul: OK 2026-06-10 06:59:32.821017 | controller | 2026-06-10 06:59:32.821048 | controller | 2026-06-10T06:59:32-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:59:37.826565 | controller | 2026-06-10T06:59:37-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:59:37.828487 | controller | 2026-06-10T06:59:37-04:00 INFO Nothing to clean up. 2026-06-10 06:59:38.436102 | controller | 2026-06-10T06:59:38-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:59:38.436187 | controller | 2026-06-10T06:59:38-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:59:38.446361 | controller | 2026-06-10T06:59:38-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:59:38.446423 | controller | 2026-06-10T06:59:38-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:59:38.849001 | controller | 2026-06-10T06:59:38-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:59:39.238878 | controller | 2026-06-10T06:59:39-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:59:39.238929 | controller | 2026-06-10T06:59:39-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:59:40.248426 | controller | 2026-06-10T06:59:40-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:59:40.248514 | controller | 2026-06-10T06:59:40-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:59:41.230863 | controller | 2026-06-10T06:59:41-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:59:41.230920 | controller | 2026-06-10T06:59:41-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:59:42.434081 | controller | 2026-06-10T06:59:42-04:00 INFO 2026-06-10 06:59:42.434126 | controller | - Config: OK 2026-06-10 06:59:42.434135 | controller | - Gateway: OK 2026-06-10 06:59:42.434142 | controller | - GitServer: OK 2026-06-10 06:59:42.434149 | controller | - HoundSearch: OK 2026-06-10 06:59:42.434155 | controller | - LogJuicer: OK 2026-06-10 06:59:42.434162 | controller | - Logserver: OK 2026-06-10 06:59:42.434198 | controller | - MariaDB: OK 2026-06-10 06:59:42.434206 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:59:42.434211 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:59:42.434216 | controller | - Zookeeper: OK 2026-06-10 06:59:42.434221 | controller | - Zuul: OK 2026-06-10 06:59:42.434226 | controller | 2026-06-10 06:59:42.434232 | controller | 2026-06-10T06:59:42-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:59:47.439399 | controller | 2026-06-10T06:59:47-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:59:47.441070 | controller | 2026-06-10T06:59:47-04:00 INFO Nothing to clean up. 2026-06-10 06:59:48.065881 | controller | 2026-06-10T06:59:48-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:59:48.065930 | controller | 2026-06-10T06:59:48-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:59:48.085048 | controller | 2026-06-10T06:59:48-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:59:48.085093 | controller | 2026-06-10T06:59:48-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:59:48.454960 | controller | 2026-06-10T06:59:48-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:59:48.855902 | controller | 2026-06-10T06:59:48-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:59:48.855966 | controller | 2026-06-10T06:59:48-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:59:49.866938 | controller | 2026-06-10T06:59:49-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:59:49.866989 | controller | 2026-06-10T06:59:49-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:59:50.842722 | controller | 2026-06-10T06:59:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:59:50.842758 | controller | 2026-06-10T06:59:50-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:59:52.053442 | controller | 2026-06-10T06:59:52-04:00 INFO 2026-06-10 06:59:52.053487 | controller | - Config: OK 2026-06-10 06:59:52.053493 | controller | - Gateway: OK 2026-06-10 06:59:52.053498 | controller | - GitServer: OK 2026-06-10 06:59:52.053502 | controller | - HoundSearch: OK 2026-06-10 06:59:52.053507 | controller | - LogJuicer: OK 2026-06-10 06:59:52.053511 | controller | - Logserver: OK 2026-06-10 06:59:52.053515 | controller | - MariaDB: OK 2026-06-10 06:59:52.053519 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:59:52.053524 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:59:52.053528 | controller | - Zookeeper: OK 2026-06-10 06:59:52.053532 | controller | - Zuul: OK 2026-06-10 06:59:52.053536 | controller | 2026-06-10 06:59:52.053541 | controller | 2026-06-10T06:59:52-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 06:59:57.063002 | controller | 2026-06-10T06:59:57-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:59:57.065875 | controller | 2026-06-10T06:59:57-04:00 INFO Nothing to clean up. 2026-06-10 06:59:57.682041 | controller | 2026-06-10T06:59:57-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:59:57.682120 | controller | 2026-06-10T06:59:57-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:59:57.702875 | controller | 2026-06-10T06:59:57-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:59:57.702977 | controller | 2026-06-10T06:59:57-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:59:58.090551 | controller | 2026-06-10T06:59:58-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:59:58.476006 | controller | 2026-06-10T06:59:58-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:59:58.476077 | controller | 2026-06-10T06:59:58-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:59:59.490934 | controller | 2026-06-10T06:59:59-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:59:59.490989 | controller | 2026-06-10T06:59:59-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:00:00.466137 | controller | 2026-06-10T07:00:00-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:00:00.466211 | controller | 2026-06-10T07:00:00-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:00:01.674845 | controller | 2026-06-10T07:00:01-04:00 INFO 2026-06-10 07:00:01.674912 | controller | - Config: OK 2026-06-10 07:00:01.674922 | controller | - Gateway: OK 2026-06-10 07:00:01.674943 | controller | - GitServer: OK 2026-06-10 07:00:01.674951 | controller | - HoundSearch: OK 2026-06-10 07:00:01.674959 | controller | - LogJuicer: OK 2026-06-10 07:00:01.674966 | controller | - Logserver: OK 2026-06-10 07:00:01.674973 | controller | - MariaDB: OK 2026-06-10 07:00:01.674980 | controller | - NodePoolBuilder: OK 2026-06-10 07:00:01.674987 | controller | - NodePoolLauncher: OK 2026-06-10 07:00:01.674994 | controller | - Zookeeper: OK 2026-06-10 07:00:01.675001 | controller | - Zuul: OK 2026-06-10 07:00:01.675008 | controller | 2026-06-10 07:00:01.675023 | controller | 2026-06-10T07:00:01-04:00 INFO Updating controller configmap ... 2026-06-10 07:00:02.065597 | controller | 2026-06-10T07:00:02-04:00 INFO Standalone reconcile done. 2026-06-10 11:00:02.279195 | controller | ok: Runtime: 0:00:53.660185 2026-06-10 11:00:02.284267 | 2026-06-10 11:00:02.284332 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:00:02.814414 | controller | ok: Runtime: 0:00:00.005886 2026-06-10 11:00:02.828886 | 2026-06-10 11:00:02.829013 | TASK [golang-integration-tests : Run integration tests] 2026-06-10 07:00:03.141196 | controller | go test -timeout 0 -v ./tests/... -args --ginkgo.v --ginkgo.no-color 2026-06-10 07:00:03.153046 | controller | go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 2026-06-10 07:00:03.153112 | controller | go: downloading github.com/onsi/gomega v1.33.1 2026-06-10 07:00:03.153147 | controller | go: downloading github.com/onsi/ginkgo/v2 v2.17.2 2026-06-10 07:00:16.629570 | controller | === RUN TestSFOP 2026-06-10 07:00:16.631066 | controller | Running Suite: Software Factory CI - /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests 2026-06-10 07:00:16.631094 | controller | ======================================================================================================================== 2026-06-10 07:00:16.631100 | controller | Random Seed: 1781089216 2026-06-10 07:00:16.631105 | controller | 2026-06-10 07:00:16.631108 | controller | Will run 16 of 16 specs 2026-06-10 07:00:16.631112 | controller | ------------------------------ 2026-06-10 07:00:16.631193 | controller | [BeforeSuite] 2026-06-10 07:00:16.631209 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/main_test.go:56 2026-06-10 07:00:16.632398 | controller | 2026-06-10T07:00:16-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:00:16.632417 | controller | 2026-06-10 07:00:16.640772 | controller | 2026-06-10T07:00:16-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:00:16.640820 | controller | 2026-06-10 07:00:16.644746 | controller | [BeforeSuite] PASSED [0.013 seconds] 2026-06-10 07:00:16.644771 | controller | ------------------------------ 2026-06-10 07:00:16.644779 | controller | Test Env Has namespace 2026-06-10 07:00:16.644789 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/main_test.go:168 2026-06-10 07:00:16.644842 | controller | • [0.000 seconds] 2026-06-10 07:00:16.644848 | controller | ------------------------------ 2026-06-10 07:00:16.644863 | controller | Deploy Tests Reconcile default 2026-06-10 07:00:16.644867 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:11 2026-06-10 07:00:16.646820 | controller | 2026-06-10T07:00:16-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:00:16.647727 | controller | 2026-06-10T07:00:16-04:00 INFO Nothing to clean up. 2026-06-10 07:00:17.257771 | controller | 2026-06-10T07:00:17-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:00:17.257842 | controller | 2026-06-10T07:00:17-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:00:17.264359 | controller | 2026-06-10T07:00:17-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:00:17.350982 | controller | 2026-06-10T07:00:17-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:00:17.358285 | controller | 2026-06-10T07:00:17-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:00:17.358344 | controller | 2026-06-10T07:00:17-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:00:17.372298 | controller | 2026-06-10T07:00:17-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:00:17.372355 | controller | 2026-06-10T07:00:17-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:00:17.666775 | controller | 2026-06-10T07:00:17-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:00:18.061467 | controller | 2026-06-10T07:00:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:00:18.061552 | controller | 2026-06-10T07:00:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:00:19.079327 | controller | 2026-06-10T07:00:19-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:00:19.079413 | controller | 2026-06-10T07:00:19-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:00:20.249494 | controller | 2026-06-10T07:00:20-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:00:20.249554 | controller | 2026-06-10T07:00:20-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:00:21.458479 | controller | 2026-06-10T07:00:21-04:00 INFO 2026-06-10 07:00:21.458543 | controller | - Config: OK 2026-06-10 07:00:21.458549 | controller | - Gateway: OK 2026-06-10 07:00:21.458553 | controller | - GitServer: OK 2026-06-10 07:00:21.458557 | controller | - HoundSearch: OK 2026-06-10 07:00:21.458560 | controller | - LogJuicer: OK 2026-06-10 07:00:21.458581 | controller | - Logserver: OK 2026-06-10 07:00:21.458585 | controller | - MariaDB: OK 2026-06-10 07:00:21.458589 | controller | - NodePoolBuilder: OK 2026-06-10 07:00:21.458592 | controller | - NodePoolLauncher: OK 2026-06-10 07:00:21.458597 | controller | - Zookeeper: OK 2026-06-10 07:00:21.458600 | controller | - Zuul: OK 2026-06-10 07:00:21.458604 | controller | 2026-06-10 07:00:21.458785 | controller | • [4.814 seconds] 2026-06-10 07:00:21.458789 | controller | ------------------------------ 2026-06-10 07:00:21.458793 | controller | Deploy Tests Match zuul secrets 2026-06-10 07:00:21.458797 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:14 2026-06-10 07:00:21.460512 | controller | 2026-06-10T07:00:21-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /etc/zuul/zuul.conf] 2026-06-10 07:00:21.575142 | controller | • [0.116 seconds] 2026-06-10 07:00:21.575246 | controller | ------------------------------ 2026-06-10 07:00:21.575264 | controller | Deploy Tests Connect to services 2026-06-10 07:00:21.575277 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:17 2026-06-10 07:00:21.716921 | controller | • [0.142 seconds] 2026-06-10 07:00:21.716963 | controller | ------------------------------ 2026-06-10 07:00:21.716969 | controller | Project Private Keys Rotation ensures no config errors 2026-06-10 07:00:21.716973 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:26 2026-06-10 07:00:21.716985 | controller | STEP: Ensuring tenants have no config errors @ 06/10/26 07:00:21.716 2026-06-10 07:00:21.716989 | controller | 2026-06-10T07:00:21-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/internal/config-errors] 2026-06-10 07:00:21.819692 | controller | 2026-06-10T07:00:21-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/demo-tenant/config-errors] 2026-06-10 07:00:21.909710 | controller | • [0.193 seconds] 2026-06-10 07:00:21.909816 | controller | ------------------------------ 2026-06-10 07:00:21.909824 | controller | Project Private Keys Rotation inject new secret in demo-project 2026-06-10 07:00:21.909833 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:31 2026-06-10 07:00:21.909849 | controller | STEP: Ensure a test ssh key is available @ 06/10/26 07:00:21.909 2026-06-10 07:00:21.918362 | controller | STEP: Loading test secret from fixture (fake SSH key) @ 06/10/26 07:00:21.918 2026-06-10 07:00:21.918479 | controller | STEP: Writing test secret into zuul-scheduler pod and encrypting with zuul-client @ 06/10/26 07:00:21.918 2026-06-10 07:00:21.918499 | controller | 2026-06-10T07:00:21-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat > /tmp/secret-to-encrypt.txt] 2026-06-10 07:00:22.024043 | controller | 2026-06-10T07:00:22-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat /tmp/secret-to-encrypt.txt | zuul-client encrypt --tenant demo-tenant --project demo-project] 2026-06-10 07:00:22.555941 | controller | writing RSA key 2026-06-10 07:00:22.658272 | controller | STEP: Replacing placeholders in secrets.yaml @ 06/10/26 07:00:22.657 2026-06-10 07:00:22.718123 | controller | STEP: Writing .zuul.d/secrets.yaml in repo @ 06/10/26 07:00:22.717 2026-06-10 07:00:22.718191 | controller | STEP: Committing and pushing locally as admin (HTTPS) @ 06/10/26 07:00:22.718 2026-06-10 07:00:23.315269 | controller | • [1.405 seconds] 2026-06-10 07:00:23.315328 | controller | ------------------------------ 2026-06-10 07:00:23.315576 | controller | Project Private Keys Rotation runs rotate-projects-private-keys 2026-06-10 07:00:23.315599 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:78 2026-06-10 07:00:23.315608 | controller | STEP: Fetching project private keys state from ZooKeeper (to validate change after) @ 06/10/26 07:00:23.315 2026-06-10 07:00:23.315620 | controller | 2026-06-10T07:00:23-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-10 07:00:23.315628 | controller | import configparser 2026-06-10 07:00:23.315634 | controller | import hashlib 2026-06-10 07:00:23.315642 | controller | from zuul.zk import ZooKeeperClient 2026-06-10 07:00:23.315661 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-10 07:00:23.315669 | controller | from zuul.lib import encryption 2026-06-10 07:00:23.315677 | controller | 2026-06-10 07:00:23.315684 | controller | config = configparser.ConfigParser() 2026-06-10 07:00:23.315692 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-10 07:00:23.315698 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-10 07:00:23.315707 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-10 07:00:23.315730 | controller | zk_client.connect() 2026-06-10 07:00:23.315738 | controller | lines = [] 2026-06-10 07:00:23.315746 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-10 07:00:23.315753 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-10 07:00:23.315761 | controller | created = obj["keys"][0]["created"] 2026-06-10 07:00:23.315767 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-10 07:00:23.315775 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-10 07:00:23.315781 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-10 07:00:23.315788 | controller | if "demo-project" in path: 2026-06-10 07:00:23.315794 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-10 07:00:23.315801 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-10 07:00:23.315808 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-10 07:00:23.315814 | controller | f.write(pem) 2026-06-10 07:00:23.315821 | controller | print("\n".join(sorted(lines))) 2026-06-10 07:00:23.315827 | controller | PYEOF 2026-06-10 07:00:23.315834 | controller | ] 2026-06-10 07:00:24.645369 | controller | STEP: Project keys before rotation: /keystorage/gerrit/config/config/secrets 1781088951 b70bd51526ed8c9d 2026-06-10 07:00:24.645452 | controller | /keystorage/gerrit/demo-project/demo-project/secrets 1781088952 1da115544435f84d 2026-06-10 07:00:24.645465 | controller | /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 1781088948 fcacfbfaca1a07a5 2026-06-10 07:00:24.645474 | controller | /keystorage/git-server/system-config/system-config/secrets 1781088950 f9acb7757a31cde0 2026-06-10 07:00:24.645495 | controller | /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 1781088951 c5b98f8872c196f8 2026-06-10 07:00:24.645504 | controller | @ 06/10/26 07:00:24.645 2026-06-10 07:00:24.645513 | controller | STEP: Preparing SSH key for rotation from zuul-ssh-key secret @ 06/10/26 07:00:24.645 2026-06-10 07:00:24.648530 | controller | STEP: Running rotate-projects-private-keys @ 06/10/26 07:00:24.648 2026-06-10 07:00:24.650959 | controller | 2026-06-10T07:00:24-04:00 INFO Creating object, name: zuul-kazoo 2026-06-10 07:00:26.669128 | controller | 2026-06-10T07:00:26-04:00 INFO Deleting object, name: config-base-secret-1-917-ns 2026-06-10 07:00:26.671802 | controller | 2026-06-10T07:00:26-04:00 INFO KubeAPIWarningLogger child pods are preserved by default when jobs are deleted; set propagationPolicy=Background to remove them or set propagationPolicy=Orphan to suppress this warning 2026-06-10 07:00:26.671831 | controller | 2026-06-10T07:00:26-04:00 INFO Deleting object, name: zs-internal-tenant-reconfigure 2026-06-10 07:00:26.675986 | controller | 2026-06-10T07:00:26-04:00 INFO Creating config map name: zk-clients-need-refresh-config-map 2026-06-10 07:00:26.676029 | controller | 2026-06-10T07:00:26-04:00 INFO Creating object, name: zk-clients-need-refresh-config-map 2026-06-10 07:00:26.679322 | controller | 2026-06-10T07:00:26-04:00 INFO Creating config map name: zuul-needs-full-reconfigure-config-map 2026-06-10 07:00:26.679385 | controller | 2026-06-10T07:00:26-04:00 INFO Creating object, name: zuul-needs-full-reconfigure-config-map 2026-06-10 07:00:26.681609 | controller | 2026-06-10T07:00:26-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /tmp/rotate-projects-private-keys.py && chmod 755 /tmp/*.py] 2026-06-10 07:00:26.775635 | controller | 2026-06-10T07:00:26-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /var/lib/zuul/.ssh_push_key && chmod 0600 /var/lib/zuul/.ssh_push_key] 2026-06-10 07:00:26.877695 | controller | 2026-06-10T07:00:26-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /var/lib/zuul/main.yaml] 2026-06-10 07:00:26.954663 | controller | 2026-06-10T07:00:26-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /var/lib/zuul/main.yaml] 2026-06-10 07:00:27.035396 | controller | 2026-06-10T07:00:27-04:00 INFO Running pod execution pod: zuul-kazoo, command: [env PYTHONUNBUFFERED=1 /tmp/rotate-projects-private-keys.py --age 9223372036854775807 --author zuul --email zuul@sfop.me --logserver-key LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlKS0FJQkFBS0NBZ0VBMXdBZXdMajl4US9BMUVjb1JNeUZia01GWmUzb1UydUs2ZXFJdTRwdnNzcE5zM2ZSCnlYb2hEdlUrQnJJY1ZDbmh0ZmRMbGt6dGNnc3FUeGhZVWJNSWZnbXpUNEdGWFVEOEVPb1RWTjdlQm1WQUZXMzQKMVMwcU1FdnBUOXNXNTZUckdtQ2krNzkwclBXcVNQTjBwMWhnbXJmZjJOdjRWSk1nU0FSQ25vWDlsbnAvbytDZwprMmhsQlRreUJ0UWRwVitnd2VEM3NPVkk1SUhqMTZqREY5SXVoVThzZ3pFNmNCTVovdjBXcEhDZ2p1N1FhWDRwCllHMWFKdjM4K0hSdk1tb21XV0Z5VFNLKzhLNUJPNTlDSXpYVk9YUzZFMHV0U2ZuR2tER0o1UVpIN28vdWtKa1oKQnB5b0dxajNuUkZxc1JNK2FHU2RmRjcrcE9nYldWcjVKdXBKTEN1Nml0dVhFZHVuTEFUTENLbHlUWEpZcE9TYgpvNWwzQkxxbmdPanhCUFo3Y05iOGZGd3M4RHRsWEo0dldOREtFVzVTZDJ4VzVPT3E2WVNMT1FOd3BEYkJZekJECkxQZXBBQkVUYXR0OTR4bE15eGFHWGpIa24yL09iQjdBSzNiQVV6NHlJU1BpdFdSVjZWSUkzQThpcDk2U1VPWnoKMjRaVGhaR1QxbXR3WFFnOHdGSUlRek5qeDJwVTdLMmEyMVFnVnNuZm5qSElJb1pUV0s2WnpjVWEwUlVjOHRVZwpBZFBKTVVoZUx2djkxWC9hdUZWNnhObEx5a0NBQW0vTldYSDNpMkdrdW1HMkloMDVQMVRIK0NUbXowb3lPdGxqCmhDQldlZSt0QVZ3RU1zSk9IalVlMTJGamIyUkIxSC90ZjNrOU90cks5MVZDWjUybmRBU2RrU1g0VDJjQ0F3RUEKQVFLQ0FnQXBWeWNTcEo5N0dFdmFKWEtMUkE0MkJPSVJWdDBReVVGMU4va2hvbjFXMGdwOXcwRE43bE5UOW5qKwpQWm92M3J1OVN2YzRSOE55WmM0MVZqZUpRaFVVY3hZTTZCazJOS3dqTG5jWUh5QnU5MklLbk5JM1gwOVk2UWZNCktKTHZhS2tpTWNtMUFYdlVWQXB4cTRwWE1TbW1Ka3YrL1ZEekpqVVRVK3BoWTk4K1k0Q09pTDhuaU8rL3dOOHIKSy9jbFdNMW02alNHU1pncUJBWEU3bmpRNkVPMHMyUWFOOG1HU2szNTF6MVcwaWMxUkp0RFpaUmNLN3FsZTduOApKaHRKaE5xcDNoQmwzNVA2WUc4L29TQk05K2x2ZWsvYkxQWjlubTc5UStGQmJlTS83bUlQWUxqTXFpdXlzS0dvCkRIVEk1cXZkMDlicVZyTzlEcnBQaEt3YXZQcmpqcHgxTjhQT1F3c3NFd2VmRHJucEdmRkJNQTluQWxzM0JMQ2sKRU9VVHFHdnZ4VW1MbEM3S25UZmVtUldzbzZGbnJncUV6d01rQnRDRWhpLzBkRjUxZHlsOVlqK3lkNnUxUUtUQgpvNk1WT1hvTlVHWERCNFpFQldhSDNhckNlbCtJa2YzaVNrRVlRYXNNRWd1Y0NUUDUzUy9pbVkxQ2tJbVVFZUNuCnUrM1ZJY3c4akxsUmY5OTZ4aUdtdEQ0ZUNmcHZRY1JjNjAzUzZIWFA5MmRZYXlYUW42ZmpsQVJZNnJtd1V2VEEKU3lvcUcwL1J6bmVBY3AwMGFQbm1NcHc2c0QrcEJVa0hBTkd2bGNTK3FCTU5MYWJSd3NhemZNUVF3azA4cEdXUQptSjlvdU9Kb0NrbXNsUTRmNWxzWnF4RXM4bStiWFpYaDlpZHNUWHppMXY0cVJTT2tQUUtDQVFFQTgvbEpOM1JWClZLdFpnMnc2R2J5Q1d2Um1wcFQyNXB1cG9JUzlueGhYWWJlRlMwZnhxS2NLVmRndm5qRTdoSk9zc0VmbFN0NE8KS1MrWndUMDB3b0RUVldtdVYramg2YjJqMFF1azVOcUxMVkpIRzVlaHVpZFRTU2lhYlNNSmRCbVpaMG5LVEllUApQQ0l4K0p1a0pGbEJuZkhNeEdIZUJRam94YUxlUTdTdHNwUUx6a1Q1eFhybis5WFRVZE9nZXJaUGRFTFh6MDEzCktjS2hZMkQybk0zbWkwdTY4NGExOTBSYXlXTzhpek9sb2VuUzgzUHROUlp2b1BKV0g5ZTJqek1VUEovZU9JYWYKdTRXS0E3L0daTWFIVjRjUElsbEZRZHFES0JZVTkxMUpsYW1lUDBnT1lnb3RHdTc3Y21UY0g4NTNxVmV6ZFc2Uwp0UUhER1VsNU44YVZKUUtDQVFFQTRaazRERVVtU2xyU3ZDQ3I2c3NUcTNzaXN1Z21YRFZxUFIxMGxsYjhXa3YwCnRBaktmdnNXNkJJa052TU8rc1RCU2VMUlBTSEp2T1gxYXZIbEtrRzlYWDBkbXQ0YVdndmk1QjFoL0RDUTl4aHYKQnJsUnZLSEZDRUZnU28yVUtHcXRBcERHa2pkVldvdnpJblhyK1NEL0hCTS82cHp4V2NGbkN6K2dmbnkzbWFWZAp6UDIzOEhSYzc0b0ZrRXZYTzRhMWJpUHY4aVJvUzcxdVRzSzdmNlovL3lnY2NwdnRzSVpIZkQ2Umhqa1hhSHNkCi9zQkNiTk9XWGJuMDA2SUlwNHJCUzYxeTNHbEVWSTBzRW1CT1FpN2Izclc2cnNxQnNyY3U5K1QraDl1Y2VBOC8KSy9RME56REZuNkhQMjE1eGhrRzN5RWpPUlJIZjY5MERMLzVFYWdsYW13S0NBUUExVmI3cXo1VEQ4TDJkM0hBTApKa205Qm9iV1RlUjMzM25DcktJamdKZkJ0ZlArMUJpQlRSc1dUeStCdnJPTE5SbGdaQ2M5S2JRZ3p0WW5CM0ZjClAvdHpSYW9tYjdQTGZIaXVmNFFwdjVrRGJHUTlKUFhuRXlmQzY0eHU1V1R3cTVQOG40WUpHdTEyTU9lSnVNUzIKYVlqTnRZbVZPT3ZuWDk0N0xuSzJqRzRZaHdMWnNmeUVibXorZ2Uweml6Y2UvSEg4TVl5cU5mNGg4L21DRnpWeQpiQlRGU09YeXpQVkV1LzI0M3dQZHpvbWNEdGpkL2U3U1Z6aEtFQkJCMzg5NkdLbUk0MDRhUDF2MkM2aHpUMTVZCnM0bnMyWXEyY0p1U1hPbWxQc2FRbVNVNEFaK2xwK1AxRE5KMWU5OGVUK3hMUzhDZkZad1d4UGh2ZDVZbUZnalkKNTZHOUFvSUJBSGd1YUtaWXZESkdPaGVRakNWVHRCSFpVajNlNEhpazJBamd4VGhGOUwraDBTV0x2Vnh5Sk56Uwp0VW8wcDlrOXNmN0xmMDVDenJiOEJvYkhnSjdIVUxzYUtRaFVvOU11bTl1TUw2UHF0Mzk3RzhLSVBSdnhUZ2E4CjFUQkNSaGhWNC84RzQxSUhla2YzWURvT2VmeEk4TDZqSHB5OHFQSjR6bzAyZ3loL0Jrd3F4VktzSXk1TXZ5bTQKN0lmWGJ6eTk0YTlnSTdacVlNbGdqQUFYUzAyUWhJV0lkOGN5ZkNoYkVJQUcwUDJoTW5BdGptZ2xRZjI1cGxRZAorV2c0M01Wc2JOY0UrcmlpdDQwWGU4LzZVQlovNGcrOTNrcTVoT2lSSDh0ZHlaN1Vob0dMRmNzem1CR0N3UFR1CmZjM09mVW5aZnErSzRaLzRDc01iQlZTdTd3RlNKRnNDZ2dFQkFNTlZKb3JYYVYxb0oyV3pSV2Z2STM0bld5dEgKZ09TQTRIOUlFVzM0SVpJMWVpTEV3SkFOZVlNS3FsdDluemhvUWwyQjJiY2hsa082U2pxMUZETVJOYndJL0dtSAppK01lc1ovYXVsVVlISkV3Z0wybXFOSkZjTlFHSXY2b0FjM2VsSldSZlYzczFUU1RpZzZVZVRQZnVtcFRGK1BLClF3WHFwK2l2dTkxSEhPTkZsYUxQQzFrYi8xb2V6S2dUVklFU1Z3d2RFMnV4SDhDeUg0T1NXdlJaRzNYRmpSb0sKYkRRcEc1UVdoVlpnKzZKQWRGeHRrbitRRWdpdVpSUFdPU2JtM1NvemFHbGQ3bUpxd0tpZTRHVkgwTTZzb1dPOAo3L3JJMjREYXRVbzczMGVqL0tVWUk4alBSSFF2YURVQUpoK0ZPR0lHODY5bk4xdG5WTll5SjlYdXI5dz0KLS0tLS1FTkQgUlNBIFBSSVZBVEUgS0VZLS0tLS0K] 2026-06-10 07:00:28.176442 | controller | [+] Collecting keys from ZooKeeper 2026-06-10 07:00:28.202684 | controller | [+] Deleting /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh because Non secrets private key 2026-06-10 07:00:28.498298 | controller | [+] Deleting /keystorage/git-server/system-config/system-config/ssh because Non secrets private key 2026-06-10 07:00:29.298947 | controller | [+] Deleting /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh because Non secrets private key 2026-06-10 07:00:29.630722 | controller | [+] Deleting /keystorage/gerrit/demo-project/demo-project/ssh because Non secrets private key 2026-06-10 07:00:29.785386 | controller | [+] Deleting /keystorage/gerrit/config/config/ssh because Non secrets private key 2026-06-10 07:00:29.901917 | controller | [+] Deleting /keystorage2/tenant/internal/ssh because Non secrets private key 2026-06-10 07:00:29.906406 | controller | [+] Deleting /keystorage2/tenant/demo-tenant/ssh because Non secrets private key 2026-06-10 07:00:29.931474 | controller | [+] Deleting /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets because zuul-jobs do not have secrets 2026-06-10 07:00:29.959734 | controller | [+] Deleting /keystorage/git-server/system-config/system-config/secrets because internal repo will be handled in reconcile 2026-06-10 07:00:29.964712 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/demo-tenant-config to /tmp/current-repo 2026-06-10 07:00:29.970365 | controller | Cloning into '/tmp/current-repo'... 2026-06-10 07:00:29.988264 | controller | Warning: Permanently added '[gerrit-sshd.sf]:29418' (ED25519) to the list of known hosts. 2026-06-10 07:00:30.081329 | controller | Total 9 (delta 0), reused 9 (delta 0) 2026-06-10 07:00:30.129678 | controller | [+] Deleting /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets because project had no secret 2026-06-10 07:00:30.177851 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/demo-project to /tmp/current-repo 2026-06-10 07:00:30.183029 | controller | Cloning into '/tmp/current-repo'... 2026-06-10 07:00:30.357092 | controller | Total 10 (delta 0), reused 10 (delta 0) 2026-06-10 07:00:30.831117 | controller | [+] Re-Encrypting secret(s) in /tmp/current-repo/.zuul.d/secrets.yaml 2026-06-10 07:00:30.846077 | controller | [master 6f67f48] Automatic secret re-encryption 2026-06-10 07:00:30.846123 | controller | 1 file changed, 10 insertions(+), 10 deletions(-) 2026-06-10 07:00:31.137702 | controller | remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 07:00:31.137768 | controller | To ssh://gerrit-sshd.sf:29418/demo-project 2026-06-10 07:00:31.137777 | controller | 6e83e8e..6f67f48 master -> master 2026-06-10 07:00:31.139865 | controller | [+] Updating key for /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-10 07:00:31.193335 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/config to /tmp/current-repo 2026-06-10 07:00:31.198222 | controller | Cloning into '/tmp/current-repo'... 2026-06-10 07:00:31.435554 | controller | Total 5 (delta 0), reused 5 (delta 0) 2026-06-10 07:00:31.449236 | controller | [+] Deleting /keystorage/gerrit/config/config/secrets because project had no secret 2026-06-10 07:00:31.691807 | controller | STEP: Reconciling @ 06/10/26 07:00:31.691 2026-06-10 07:00:31.695732 | controller | 2026-06-10T07:00:31-04:00 INFO Restarting zookeeper clients to refresh connection cache... 2026-06-10 07:00:31.717297 | controller | 2026-06-10T07:00:31-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-10 07:00:31.717377 | controller | 2026-06-10T07:00:31-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-10 07:00:31.804538 | controller | 2026-06-10T07:00:31-04:00 INFO Deleting object, name: nodepool-builder 2026-06-10 07:00:31.807867 | controller | 2026-06-10T07:00:31-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-6678ff4998-lklrp launcher 2026-06-10 07:00:31.807907 | controller | 2026-06-10T07:00:31-04:00 INFO Running pod execution pod: nodepool-launcher-6678ff4998-lklrp, command: [kill -9 1] 2026-06-10 07:00:31.919969 | controller | 2026-06-10T07:00:31-04:00 INFO Deleting object, name: launcher 2026-06-10 07:00:31.921926 | controller | 2026-06-10T07:00:31-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-zlqvm mariadb-client 2026-06-10 07:00:31.921952 | controller | 2026-06-10T07:00:31-04:00 INFO Running pod execution pod: zuul-db-provision-zlqvm, command: [kill -9 1] 2026-06-10 07:00:31.942890 | controller | 2026-06-10T07:00:31-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 07:00:31.945672 | controller | 2026-06-10T07:00:31-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-10 07:00:31.945732 | controller | 2026-06-10T07:00:31-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-10 07:00:32.000835 | controller | 2026-06-10T07:00:32-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 07:00:32.003988 | controller | 2026-06-10T07:00:32-04:00 INFO Warning: Killing ZooKeeper client: zuul-kazoo zuul-kazoo 2026-06-10 07:00:32.004010 | controller | 2026-06-10T07:00:32-04:00 INFO Running pod execution pod: zuul-kazoo, command: [kill -9 1] 2026-06-10 07:00:32.078118 | controller | 2026-06-10T07:00:32-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 07:00:32.080680 | controller | 2026-06-10T07:00:32-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-10 07:00:32.080698 | controller | 2026-06-10T07:00:32-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-10 07:00:32.172602 | controller | 2026-06-10T07:00:32-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 07:00:32.176441 | controller | 2026-06-10T07:00:32-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-10 07:00:32.176481 | controller | 2026-06-10T07:00:32-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-10 07:00:32.253137 | controller | 2026-06-10T07:00:32-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-10 07:00:32.256640 | controller | 2026-06-10T07:00:32-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-675cb66969-zb9s5 zuul-web 2026-06-10 07:00:32.256687 | controller | 2026-06-10T07:00:32-04:00 INFO Running pod execution pod: zuul-web-675cb66969-zb9s5, command: [kill -9 1] 2026-06-10 07:00:32.340254 | controller | 2026-06-10T07:00:32-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:00:32.344626 | controller | 2026-06-10T07:00:32-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-qwhpw zuul-weeder 2026-06-10 07:00:32.344659 | controller | 2026-06-10T07:00:32-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-qwhpw, command: [kill -9 1] 2026-06-10 07:00:32.427568 | controller | 2026-06-10T07:00:32-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 07:00:32.430662 | controller | 2026-06-10T07:00:32-04:00 INFO Deleting object, name: zk-clients-need-refresh-config-map 2026-06-10 07:00:32.433254 | controller | 2026-06-10T07:00:32-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:00:32.842877 | controller | 2026-06-10T07:00:32-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:00:32.842947 | controller | 2026-06-10T07:00:32-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:00:32.851839 | controller | 2026-06-10T07:00:32-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:00:32.965524 | controller | 2026-06-10T07:00:32-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:00:32.971542 | controller | 2026-06-10T07:00:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:00:32.971570 | controller | 2026-06-10T07:00:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:00:32.987787 | controller | 2026-06-10T07:00:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:00:32.987848 | controller | 2026-06-10T07:00:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:00:33.614546 | controller | 2026-06-10T07:00:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:00:33.656220 | controller | 2026-06-10T07:00:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:00:33.656311 | controller | 2026-06-10T07:00:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:00:33.854023 | controller | 2026-06-10T07:00:33-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 07:00:34.656763 | controller | 2026-06-10T07:00:34-04:00 INFO Creating object, name: zuul-scheduler 2026-06-10 07:00:35.037365 | controller | 2026-06-10T07:00:35-04:00 INFO Creating object, name: zuul-executor 2026-06-10 07:00:35.834608 | controller | 2026-06-10T07:00:35-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:00:36.637083 | controller | 2026-06-10T07:00:36-04:00 INFO Creating object, name: zuul-merger 2026-06-10 07:00:36.641585 | controller | 2026-06-10T07:00:36-04:00 INFO 2026-06-10 07:00:36.641617 | controller | - Config: Waiting ... 2026-06-10 07:00:36.641624 | controller | - Gateway: OK 2026-06-10 07:00:36.641629 | controller | - GitServer: OK 2026-06-10 07:00:36.641634 | controller | - HoundSearch: OK 2026-06-10 07:00:36.641639 | controller | - LogJuicer: OK 2026-06-10 07:00:36.641643 | controller | - Logserver: OK 2026-06-10 07:00:36.641648 | controller | - MariaDB: OK 2026-06-10 07:00:36.641653 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:00:36.641658 | controller | - NodePoolLauncher: OK 2026-06-10 07:00:36.641662 | controller | - Zookeeper: OK 2026-06-10 07:00:36.641667 | controller | - Zuul: Waiting ... 2026-06-10 07:00:36.641672 | controller | - |_ Config Secrets: OK 2026-06-10 07:00:36.641680 | controller | - |_ Executor: Waiting ... 2026-06-10 07:00:36.641685 | controller | - |_ Merger: Waiting ... 2026-06-10 07:00:36.641690 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:00:36.641695 | controller | - |_ Web: Waiting ... 2026-06-10 07:00:36.641701 | controller | 2026-06-10 07:00:38.136497 | controller | 2026-06-10T07:00:38-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:00:38.138950 | controller | 2026-06-10T07:00:38-04:00 INFO Nothing to clean up. 2026-06-10 07:00:38.548066 | controller | 2026-06-10T07:00:38-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:00:38.548131 | controller | 2026-06-10T07:00:38-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:00:38.561268 | controller | 2026-06-10T07:00:38-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:00:38.561331 | controller | 2026-06-10T07:00:38-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:00:38.579465 | controller | 2026-06-10T07:00:38-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:00:38.579526 | controller | 2026-06-10T07:00:38-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:00:38.956538 | controller | 2026-06-10T07:00:38-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:00:39.357657 | controller | 2026-06-10T07:00:39-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:00:39.357861 | controller | 2026-06-10T07:00:39-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:00:40.470137 | controller | 2026-06-10T07:00:40-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:01:10.573282 | controller | 2026-06-10T07:01:10-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:01:10.610313 | controller | 2026-06-10T07:01:10-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:01:10.610364 | controller | 2026-06-10T07:01:10-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:01:10.986558 | controller | 2026-06-10T07:01:10-04:00 INFO 2026-06-10 07:01:10.986638 | controller | - Config: Waiting ... 2026-06-10 07:01:10.986649 | controller | - Gateway: OK 2026-06-10 07:01:10.986658 | controller | - GitServer: OK 2026-06-10 07:01:10.986666 | controller | - HoundSearch: OK 2026-06-10 07:01:10.986674 | controller | - LogJuicer: OK 2026-06-10 07:01:10.986682 | controller | - Logserver: OK 2026-06-10 07:01:10.986690 | controller | - MariaDB: OK 2026-06-10 07:01:10.986698 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:01:10.986706 | controller | - NodePoolLauncher: OK 2026-06-10 07:01:10.986714 | controller | - Zookeeper: OK 2026-06-10 07:01:10.986722 | controller | - Zuul: Waiting ... 2026-06-10 07:01:10.986730 | controller | - |_ Config Secrets: OK 2026-06-10 07:01:10.986738 | controller | - |_ Executor: OK 2026-06-10 07:01:10.986768 | controller | - |_ Merger: OK 2026-06-10 07:01:10.986776 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:01:10.986784 | controller | - |_ Web: Waiting ... 2026-06-10 07:01:10.986793 | controller | 2026-06-10 07:01:11.993063 | controller | 2026-06-10T07:01:11-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:01:11.995701 | controller | 2026-06-10T07:01:11-04:00 INFO Nothing to clean up. 2026-06-10 07:01:12.596372 | controller | 2026-06-10T07:01:12-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:01:12.596439 | controller | 2026-06-10T07:01:12-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:01:12.608589 | controller | 2026-06-10T07:01:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:01:12.608669 | controller | 2026-06-10T07:01:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:01:12.626852 | controller | 2026-06-10T07:01:12-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:01:12.626906 | controller | 2026-06-10T07:01:12-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:01:13.010307 | controller | 2026-06-10T07:01:13-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:01:13.410211 | controller | 2026-06-10T07:01:13-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:01:13.410274 | controller | 2026-06-10T07:01:13-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:01:14.782469 | controller | 2026-06-10T07:01:14-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:01:38.906314 | controller | 2026-06-10T07:01:38-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 07:01:38.960791 | controller | 2026-06-10T07:01:38-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:01:38.960841 | controller | 2026-06-10T07:01:38-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:01:39.326581 | controller | 2026-06-10T07:01:39-04:00 INFO 2026-06-10 07:01:39.326648 | controller | - Config: Waiting ... 2026-06-10 07:01:39.326655 | controller | - Gateway: OK 2026-06-10 07:01:39.326661 | controller | - GitServer: OK 2026-06-10 07:01:39.326666 | controller | - HoundSearch: OK 2026-06-10 07:01:39.326671 | controller | - LogJuicer: OK 2026-06-10 07:01:39.326676 | controller | - Logserver: OK 2026-06-10 07:01:39.326681 | controller | - MariaDB: OK 2026-06-10 07:01:39.326686 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:01:39.326692 | controller | - NodePoolLauncher: OK 2026-06-10 07:01:39.326697 | controller | - Zookeeper: OK 2026-06-10 07:01:39.326702 | controller | - Zuul: Waiting ... 2026-06-10 07:01:39.326707 | controller | - |_ Config Secrets: OK 2026-06-10 07:01:39.326712 | controller | - |_ Executor: OK 2026-06-10 07:01:39.326717 | controller | - |_ Merger: OK 2026-06-10 07:01:39.326722 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:01:39.326727 | controller | - |_ Web: OK 2026-06-10 07:01:39.326732 | controller | 2026-06-10 07:01:40.331653 | controller | 2026-06-10T07:01:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:01:40.333331 | controller | 2026-06-10T07:01:40-04:00 INFO Nothing to clean up. 2026-06-10 07:01:40.934709 | controller | 2026-06-10T07:01:40-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:01:40.934770 | controller | 2026-06-10T07:01:40-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:01:40.949721 | controller | 2026-06-10T07:01:40-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:01:40.949790 | controller | 2026-06-10T07:01:40-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:01:40.965983 | controller | 2026-06-10T07:01:40-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:01:40.966040 | controller | 2026-06-10T07:01:40-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:01:41.348536 | controller | 2026-06-10T07:01:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:01:41.752852 | controller | 2026-06-10T07:01:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:01:41.752926 | controller | 2026-06-10T07:01:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:01:43.126402 | controller | 2026-06-10T07:01:43-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:01:43.126463 | controller | 2026-06-10T07:01:43-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:01:44.716247 | controller | 2026-06-10T07:01:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:01:44.716361 | controller | 2026-06-10T07:01:44-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:01:45.918994 | controller | 2026-06-10T07:01:45-04:00 INFO Creating base secret job 2026-06-10 07:01:45.919286 | controller | 2026-06-10T07:01:45-04:00 INFO Creating object, name: config-base-secret-1-917-ns 2026-06-10 07:01:45.922157 | controller | 2026-06-10T07:01:45-04:00 INFO 2026-06-10 07:01:45.922189 | controller | - Config: Waiting ... 2026-06-10 07:01:45.922195 | controller | - Gateway: OK 2026-06-10 07:01:45.922200 | controller | - GitServer: OK 2026-06-10 07:01:45.922205 | controller | - HoundSearch: OK 2026-06-10 07:01:45.922209 | controller | - LogJuicer: OK 2026-06-10 07:01:45.922214 | controller | - Logserver: OK 2026-06-10 07:01:45.922218 | controller | - MariaDB: OK 2026-06-10 07:01:45.922223 | controller | - NodePoolBuilder: OK 2026-06-10 07:01:45.922227 | controller | - NodePoolLauncher: OK 2026-06-10 07:01:45.922232 | controller | - Zookeeper: OK 2026-06-10 07:01:45.922237 | controller | - Zuul: OK 2026-06-10 07:01:45.922242 | controller | 2026-06-10 07:01:46.926286 | controller | 2026-06-10T07:01:46-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:01:46.928372 | controller | 2026-06-10T07:01:46-04:00 INFO Nothing to clean up. 2026-06-10 07:01:47.530185 | controller | 2026-06-10T07:01:47-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:01:47.530247 | controller | 2026-06-10T07:01:47-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:01:47.541218 | controller | 2026-06-10T07:01:47-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:01:47.541298 | controller | 2026-06-10T07:01:47-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:01:47.552633 | controller | 2026-06-10T07:01:47-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:01:47.552677 | controller | 2026-06-10T07:01:47-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:01:47.944285 | controller | 2026-06-10T07:01:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:01:48.338943 | controller | 2026-06-10T07:01:48-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:01:48.339012 | controller | 2026-06-10T07:01:48-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:01:49.729213 | controller | 2026-06-10T07:01:49-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:01:49.729279 | controller | 2026-06-10T07:01:49-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:01:51.316062 | controller | 2026-06-10T07:01:51-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:01:51.316136 | controller | 2026-06-10T07:01:51-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:01:52.525593 | controller | 2026-06-10T07:01:52-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 07:01:52.714222 | controller | 2026-06-10T07:01:52-04:00 INFO Deleting object, name: zuul-needs-full-reconfigure-config-map 2026-06-10 07:01:52.916783 | controller | 2026-06-10T07:01:52-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler full-reconfigure] 2026-06-10 07:01:57.987818 | controller | 2026-06-10T07:01:57-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:01:57.993966 | controller | 2026-06-10T07:01:57-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-10 07:01:57.998474 | controller | 2026-06-10T07:01:57-04:00 INFO 2026-06-10 07:01:57.998564 | controller | - Config: Waiting ... 2026-06-10 07:01:57.998600 | controller | - Gateway: OK 2026-06-10 07:01:57.998610 | controller | - GitServer: OK 2026-06-10 07:01:57.998619 | controller | - HoundSearch: OK 2026-06-10 07:01:57.998628 | controller | - LogJuicer: OK 2026-06-10 07:01:57.998637 | controller | - Logserver: OK 2026-06-10 07:01:57.998645 | controller | - MariaDB: OK 2026-06-10 07:01:57.998653 | controller | - NodePoolBuilder: OK 2026-06-10 07:01:57.998662 | controller | - NodePoolLauncher: OK 2026-06-10 07:01:57.998671 | controller | - Zookeeper: OK 2026-06-10 07:01:57.998680 | controller | - Zuul: OK 2026-06-10 07:01:57.998689 | controller | 2026-06-10 07:01:59.004944 | controller | 2026-06-10T07:01:59-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:01:59.006917 | controller | 2026-06-10T07:01:59-04:00 INFO Nothing to clean up. 2026-06-10 07:01:59.606125 | controller | 2026-06-10T07:01:59-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:01:59.606208 | controller | 2026-06-10T07:01:59-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:01:59.616374 | controller | 2026-06-10T07:01:59-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:01:59.616433 | controller | 2026-06-10T07:01:59-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:01:59.636358 | controller | 2026-06-10T07:01:59-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:01:59.636419 | controller | 2026-06-10T07:01:59-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:00.013786 | controller | 2026-06-10T07:02:00-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:00.414892 | controller | 2026-06-10T07:02:00-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:00.414959 | controller | 2026-06-10T07:02:00-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:01.427688 | controller | 2026-06-10T07:02:01-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:01.427746 | controller | 2026-06-10T07:02:01-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:02.407013 | controller | 2026-06-10T07:02:02-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:02:03.217600 | controller | 2026-06-10T07:02:03-04:00 INFO 2026-06-10 07:02:03.217690 | controller | - Config: Waiting ... 2026-06-10 07:02:03.217827 | controller | - Gateway: OK 2026-06-10 07:02:03.217918 | controller | - GitServer: OK 2026-06-10 07:02:03.218019 | controller | - HoundSearch: OK 2026-06-10 07:02:03.218109 | controller | - LogJuicer: OK 2026-06-10 07:02:03.218429 | controller | - Logserver: OK 2026-06-10 07:02:03.218549 | controller | - MariaDB: OK 2026-06-10 07:02:03.218677 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:03.218781 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:03.218880 | controller | - Zookeeper: OK 2026-06-10 07:02:03.218995 | controller | - Zuul: Waiting ... 2026-06-10 07:02:03.219109 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:03.219245 | controller | - |_ Executor: OK 2026-06-10 07:02:03.219342 | controller | - |_ Merger: OK 2026-06-10 07:02:03.219442 | controller | - |_ Scheduler: OK 2026-06-10 07:02:03.219532 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:03.219573 | controller | 2026-06-10 07:02:04.227744 | controller | 2026-06-10T07:02:04-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:04.230460 | controller | 2026-06-10T07:02:04-04:00 INFO Nothing to clean up. 2026-06-10 07:02:04.831194 | controller | 2026-06-10T07:02:04-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:04.831277 | controller | 2026-06-10T07:02:04-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:04.845892 | controller | 2026-06-10T07:02:04-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:04.845949 | controller | 2026-06-10T07:02:04-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:04.869196 | controller | 2026-06-10T07:02:04-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:04.869281 | controller | 2026-06-10T07:02:04-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:05.255512 | controller | 2026-06-10T07:02:05-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:05.637048 | controller | 2026-06-10T07:02:05-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:05.637117 | controller | 2026-06-10T07:02:05-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:07.017274 | controller | 2026-06-10T07:02:07-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:07.017350 | controller | 2026-06-10T07:02:07-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:08.607573 | controller | 2026-06-10T07:02:08-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:08.607623 | controller | 2026-06-10T07:02:08-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:09.434555 | controller | 2026-06-10T07:02:09-04:00 INFO 2026-06-10 07:02:09.434628 | controller | - Config: Waiting ... 2026-06-10 07:02:09.434638 | controller | - Gateway: OK 2026-06-10 07:02:09.434646 | controller | - GitServer: OK 2026-06-10 07:02:09.434653 | controller | - HoundSearch: OK 2026-06-10 07:02:09.434659 | controller | - LogJuicer: OK 2026-06-10 07:02:09.434682 | controller | - Logserver: OK 2026-06-10 07:02:09.434688 | controller | - MariaDB: OK 2026-06-10 07:02:09.434693 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:09.434699 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:09.434705 | controller | - Zookeeper: OK 2026-06-10 07:02:09.434711 | controller | - Zuul: Waiting ... 2026-06-10 07:02:09.434717 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:09.434723 | controller | - |_ Executor: OK 2026-06-10 07:02:09.434728 | controller | - |_ Merger: OK 2026-06-10 07:02:09.434734 | controller | - |_ Scheduler: OK 2026-06-10 07:02:09.434740 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:09.434746 | controller | 2026-06-10 07:02:10.440714 | controller | 2026-06-10T07:02:10-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:10.443425 | controller | 2026-06-10T07:02:10-04:00 INFO Nothing to clean up. 2026-06-10 07:02:11.044518 | controller | 2026-06-10T07:02:11-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:11.044571 | controller | 2026-06-10T07:02:11-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:11.055978 | controller | 2026-06-10T07:02:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:11.056030 | controller | 2026-06-10T07:02:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:11.074349 | controller | 2026-06-10T07:02:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:11.074405 | controller | 2026-06-10T07:02:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:11.454183 | controller | 2026-06-10T07:02:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:11.859892 | controller | 2026-06-10T07:02:11-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:11.859972 | controller | 2026-06-10T07:02:11-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:13.217438 | controller | 2026-06-10T07:02:13-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:13.217496 | controller | 2026-06-10T07:02:13-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:14.808251 | controller | 2026-06-10T07:02:14-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:14.808317 | controller | 2026-06-10T07:02:14-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:15.618710 | controller | 2026-06-10T07:02:15-04:00 INFO 2026-06-10 07:02:15.618793 | controller | - Config: Waiting ... 2026-06-10 07:02:15.618805 | controller | - Gateway: OK 2026-06-10 07:02:15.618815 | controller | - GitServer: OK 2026-06-10 07:02:15.618824 | controller | - HoundSearch: OK 2026-06-10 07:02:15.618833 | controller | - LogJuicer: OK 2026-06-10 07:02:15.618842 | controller | - Logserver: OK 2026-06-10 07:02:15.618912 | controller | - MariaDB: OK 2026-06-10 07:02:15.618933 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:15.618943 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:15.618952 | controller | - Zookeeper: OK 2026-06-10 07:02:15.618961 | controller | - Zuul: Waiting ... 2026-06-10 07:02:15.618970 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:15.618978 | controller | - |_ Executor: OK 2026-06-10 07:02:15.618987 | controller | - |_ Merger: OK 2026-06-10 07:02:15.618995 | controller | - |_ Scheduler: OK 2026-06-10 07:02:15.619004 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:15.619013 | controller | 2026-06-10 07:02:16.626841 | controller | 2026-06-10T07:02:16-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:16.638279 | controller | 2026-06-10T07:02:16-04:00 INFO Nothing to clean up. 2026-06-10 07:02:17.231282 | controller | 2026-06-10T07:02:17-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:17.231350 | controller | 2026-06-10T07:02:17-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:17.247054 | controller | 2026-06-10T07:02:17-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:17.247117 | controller | 2026-06-10T07:02:17-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:17.270666 | controller | 2026-06-10T07:02:17-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:17.270731 | controller | 2026-06-10T07:02:17-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:17.646715 | controller | 2026-06-10T07:02:17-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:18.046090 | controller | 2026-06-10T07:02:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:18.046195 | controller | 2026-06-10T07:02:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:19.418971 | controller | 2026-06-10T07:02:19-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:19.419036 | controller | 2026-06-10T07:02:19-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:21.007656 | controller | 2026-06-10T07:02:21-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:21.007727 | controller | 2026-06-10T07:02:21-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:21.815667 | controller | 2026-06-10T07:02:21-04:00 INFO 2026-06-10 07:02:21.815748 | controller | - Config: Waiting ... 2026-06-10 07:02:21.815759 | controller | - Gateway: OK 2026-06-10 07:02:21.815768 | controller | - GitServer: OK 2026-06-10 07:02:21.815776 | controller | - HoundSearch: OK 2026-06-10 07:02:21.815784 | controller | - LogJuicer: OK 2026-06-10 07:02:21.815792 | controller | - Logserver: OK 2026-06-10 07:02:21.815800 | controller | - MariaDB: OK 2026-06-10 07:02:21.815807 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:21.815815 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:21.815823 | controller | - Zookeeper: OK 2026-06-10 07:02:21.815830 | controller | - Zuul: Waiting ... 2026-06-10 07:02:21.815838 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:21.815846 | controller | - |_ Executor: OK 2026-06-10 07:02:21.815854 | controller | - |_ Merger: OK 2026-06-10 07:02:21.815862 | controller | - |_ Scheduler: OK 2026-06-10 07:02:21.815869 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:21.815877 | controller | 2026-06-10 07:02:22.822004 | controller | 2026-06-10T07:02:22-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:22.824376 | controller | 2026-06-10T07:02:22-04:00 INFO Nothing to clean up. 2026-06-10 07:02:23.441421 | controller | 2026-06-10T07:02:23-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:23.441500 | controller | 2026-06-10T07:02:23-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:23.453349 | controller | 2026-06-10T07:02:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:23.453417 | controller | 2026-06-10T07:02:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:23.471105 | controller | 2026-06-10T07:02:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:23.471210 | controller | 2026-06-10T07:02:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:23.842541 | controller | 2026-06-10T07:02:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:24.235866 | controller | 2026-06-10T07:02:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:24.235931 | controller | 2026-06-10T07:02:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:25.620064 | controller | 2026-06-10T07:02:25-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:25.620132 | controller | 2026-06-10T07:02:25-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:27.207386 | controller | 2026-06-10T07:02:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:27.207464 | controller | 2026-06-10T07:02:27-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:28.018579 | controller | 2026-06-10T07:02:28-04:00 INFO 2026-06-10 07:02:28.018667 | controller | - Config: Waiting ... 2026-06-10 07:02:28.018679 | controller | - Gateway: OK 2026-06-10 07:02:28.018689 | controller | - GitServer: OK 2026-06-10 07:02:28.018699 | controller | - HoundSearch: OK 2026-06-10 07:02:28.018710 | controller | - LogJuicer: OK 2026-06-10 07:02:28.018722 | controller | - Logserver: OK 2026-06-10 07:02:28.018733 | controller | - MariaDB: OK 2026-06-10 07:02:28.018745 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:28.018756 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:28.018767 | controller | - Zookeeper: OK 2026-06-10 07:02:28.018778 | controller | - Zuul: Waiting ... 2026-06-10 07:02:28.018789 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:28.018801 | controller | - |_ Executor: OK 2026-06-10 07:02:28.018812 | controller | - |_ Merger: OK 2026-06-10 07:02:28.018823 | controller | - |_ Scheduler: OK 2026-06-10 07:02:28.018834 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:28.018845 | controller | 2026-06-10 07:02:29.026267 | controller | 2026-06-10T07:02:29-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:29.029500 | controller | 2026-06-10T07:02:29-04:00 INFO Nothing to clean up. 2026-06-10 07:02:29.629560 | controller | 2026-06-10T07:02:29-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:29.629702 | controller | 2026-06-10T07:02:29-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:29.645052 | controller | 2026-06-10T07:02:29-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:29.645214 | controller | 2026-06-10T07:02:29-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:29.667416 | controller | 2026-06-10T07:02:29-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:29.667499 | controller | 2026-06-10T07:02:29-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:30.043264 | controller | 2026-06-10T07:02:30-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:30.437148 | controller | 2026-06-10T07:02:30-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:30.437275 | controller | 2026-06-10T07:02:30-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:31.819920 | controller | 2026-06-10T07:02:31-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:31.820012 | controller | 2026-06-10T07:02:31-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:33.406966 | controller | 2026-06-10T07:02:33-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:33.407034 | controller | 2026-06-10T07:02:33-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:34.215121 | controller | 2026-06-10T07:02:34-04:00 INFO 2026-06-10 07:02:34.215241 | controller | - Config: Waiting ... 2026-06-10 07:02:34.215254 | controller | - Gateway: OK 2026-06-10 07:02:34.215301 | controller | - GitServer: OK 2026-06-10 07:02:34.215338 | controller | - HoundSearch: OK 2026-06-10 07:02:34.215399 | controller | - LogJuicer: OK 2026-06-10 07:02:34.215441 | controller | - Logserver: OK 2026-06-10 07:02:34.215507 | controller | - MariaDB: OK 2026-06-10 07:02:34.215563 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:34.215612 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:34.215687 | controller | - Zookeeper: OK 2026-06-10 07:02:34.215729 | controller | - Zuul: Waiting ... 2026-06-10 07:02:34.215769 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:34.215825 | controller | - |_ Executor: OK 2026-06-10 07:02:34.215969 | controller | - |_ Merger: OK 2026-06-10 07:02:34.215985 | controller | - |_ Scheduler: OK 2026-06-10 07:02:34.215993 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:34.216001 | controller | 2026-06-10 07:02:35.223449 | controller | 2026-06-10T07:02:35-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:35.225895 | controller | 2026-06-10T07:02:35-04:00 INFO Nothing to clean up. 2026-06-10 07:02:35.826262 | controller | 2026-06-10T07:02:35-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:35.826352 | controller | 2026-06-10T07:02:35-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:35.839268 | controller | 2026-06-10T07:02:35-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:35.839317 | controller | 2026-06-10T07:02:35-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:35.861497 | controller | 2026-06-10T07:02:35-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:35.861552 | controller | 2026-06-10T07:02:35-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:36.243581 | controller | 2026-06-10T07:02:36-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:36.643116 | controller | 2026-06-10T07:02:36-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:36.643202 | controller | 2026-06-10T07:02:36-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:38.020492 | controller | 2026-06-10T07:02:38-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:38.020586 | controller | 2026-06-10T07:02:38-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:39.609430 | controller | 2026-06-10T07:02:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:39.609517 | controller | 2026-06-10T07:02:39-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:40.416620 | controller | 2026-06-10T07:02:40-04:00 INFO 2026-06-10 07:02:40.416697 | controller | - Config: Waiting ... 2026-06-10 07:02:40.416725 | controller | - Gateway: OK 2026-06-10 07:02:40.416735 | controller | - GitServer: OK 2026-06-10 07:02:40.416744 | controller | - HoundSearch: OK 2026-06-10 07:02:40.416753 | controller | - LogJuicer: OK 2026-06-10 07:02:40.416761 | controller | - Logserver: OK 2026-06-10 07:02:40.416770 | controller | - MariaDB: OK 2026-06-10 07:02:40.416779 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:40.416792 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:40.416804 | controller | - Zookeeper: OK 2026-06-10 07:02:40.416817 | controller | - Zuul: Waiting ... 2026-06-10 07:02:40.416828 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:40.416841 | controller | - |_ Executor: OK 2026-06-10 07:02:40.416851 | controller | - |_ Merger: OK 2026-06-10 07:02:40.416861 | controller | - |_ Scheduler: OK 2026-06-10 07:02:40.416872 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:40.416882 | controller | 2026-06-10 07:02:41.422913 | controller | 2026-06-10T07:02:41-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:41.424817 | controller | 2026-06-10T07:02:41-04:00 INFO Nothing to clean up. 2026-06-10 07:02:42.028685 | controller | 2026-06-10T07:02:42-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:42.028765 | controller | 2026-06-10T07:02:42-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:42.038668 | controller | 2026-06-10T07:02:42-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:42.038733 | controller | 2026-06-10T07:02:42-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:42.052475 | controller | 2026-06-10T07:02:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:42.052528 | controller | 2026-06-10T07:02:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:42.447249 | controller | 2026-06-10T07:02:42-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:42.841790 | controller | 2026-06-10T07:02:42-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:42.841868 | controller | 2026-06-10T07:02:42-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:44.221402 | controller | 2026-06-10T07:02:44-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:44.221460 | controller | 2026-06-10T07:02:44-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:45.807195 | controller | 2026-06-10T07:02:45-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:45.807266 | controller | 2026-06-10T07:02:45-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:46.629515 | controller | 2026-06-10T07:02:46-04:00 INFO 2026-06-10 07:02:46.629589 | controller | - Config: Waiting ... 2026-06-10 07:02:46.629597 | controller | - Gateway: OK 2026-06-10 07:02:46.629603 | controller | - GitServer: OK 2026-06-10 07:02:46.629608 | controller | - HoundSearch: OK 2026-06-10 07:02:46.629614 | controller | - LogJuicer: OK 2026-06-10 07:02:46.629619 | controller | - Logserver: OK 2026-06-10 07:02:46.629624 | controller | - MariaDB: OK 2026-06-10 07:02:46.629630 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:46.629635 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:46.629640 | controller | - Zookeeper: OK 2026-06-10 07:02:46.629646 | controller | - Zuul: Waiting ... 2026-06-10 07:02:46.629651 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:46.629656 | controller | - |_ Executor: OK 2026-06-10 07:02:46.629661 | controller | - |_ Merger: OK 2026-06-10 07:02:46.629666 | controller | - |_ Scheduler: OK 2026-06-10 07:02:46.629672 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:46.629677 | controller | 2026-06-10 07:02:47.635160 | controller | 2026-06-10T07:02:47-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:47.638067 | controller | 2026-06-10T07:02:47-04:00 INFO Nothing to clean up. 2026-06-10 07:02:48.236893 | controller | 2026-06-10T07:02:48-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:48.236951 | controller | 2026-06-10T07:02:48-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:48.244803 | controller | 2026-06-10T07:02:48-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:48.244848 | controller | 2026-06-10T07:02:48-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:48.258472 | controller | 2026-06-10T07:02:48-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:48.258528 | controller | 2026-06-10T07:02:48-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:48.661892 | controller | 2026-06-10T07:02:48-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:49.055629 | controller | 2026-06-10T07:02:49-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:49.055736 | controller | 2026-06-10T07:02:49-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:50.427236 | controller | 2026-06-10T07:02:50-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:50.427315 | controller | 2026-06-10T07:02:50-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:52.007025 | controller | 2026-06-10T07:02:52-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:52.007084 | controller | 2026-06-10T07:02:52-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:52.815540 | controller | 2026-06-10T07:02:52-04:00 INFO 2026-06-10 07:02:52.815613 | controller | - Config: Waiting ... 2026-06-10 07:02:52.815625 | controller | - Gateway: OK 2026-06-10 07:02:52.815634 | controller | - GitServer: OK 2026-06-10 07:02:52.815642 | controller | - HoundSearch: OK 2026-06-10 07:02:52.815650 | controller | - LogJuicer: OK 2026-06-10 07:02:52.815657 | controller | - Logserver: OK 2026-06-10 07:02:52.815665 | controller | - MariaDB: OK 2026-06-10 07:02:52.815673 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:52.815681 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:52.815688 | controller | - Zookeeper: OK 2026-06-10 07:02:52.815704 | controller | - Zuul: Waiting ... 2026-06-10 07:02:52.815712 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:52.815720 | controller | - |_ Executor: OK 2026-06-10 07:02:52.815728 | controller | - |_ Merger: OK 2026-06-10 07:02:52.815736 | controller | - |_ Scheduler: OK 2026-06-10 07:02:52.815744 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:52.815752 | controller | 2026-06-10 07:02:53.820499 | controller | 2026-06-10T07:02:53-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:02:53.822943 | controller | 2026-06-10T07:02:53-04:00 INFO Nothing to clean up. 2026-06-10 07:02:54.428930 | controller | 2026-06-10T07:02:54-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:02:54.429006 | controller | 2026-06-10T07:02:54-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:02:54.443850 | controller | 2026-06-10T07:02:54-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:02:54.443913 | controller | 2026-06-10T07:02:54-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:02:54.460247 | controller | 2026-06-10T07:02:54-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:02:54.460306 | controller | 2026-06-10T07:02:54-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:02:54.848713 | controller | 2026-06-10T07:02:54-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:02:55.240054 | controller | 2026-06-10T07:02:55-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:02:55.240129 | controller | 2026-06-10T07:02:55-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:02:56.613907 | controller | 2026-06-10T07:02:56-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:02:56.613975 | controller | 2026-06-10T07:02:56-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:02:58.207131 | controller | 2026-06-10T07:02:58-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:02:58.207201 | controller | 2026-06-10T07:02:58-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:02:59.015546 | controller | 2026-06-10T07:02:59-04:00 INFO 2026-06-10 07:02:59.015626 | controller | - Config: Waiting ... 2026-06-10 07:02:59.015639 | controller | - Gateway: OK 2026-06-10 07:02:59.015649 | controller | - GitServer: OK 2026-06-10 07:02:59.015657 | controller | - HoundSearch: OK 2026-06-10 07:02:59.015666 | controller | - LogJuicer: OK 2026-06-10 07:02:59.015675 | controller | - Logserver: OK 2026-06-10 07:02:59.015684 | controller | - MariaDB: OK 2026-06-10 07:02:59.015693 | controller | - NodePoolBuilder: OK 2026-06-10 07:02:59.015701 | controller | - NodePoolLauncher: OK 2026-06-10 07:02:59.015710 | controller | - Zookeeper: OK 2026-06-10 07:02:59.015718 | controller | - Zuul: Waiting ... 2026-06-10 07:02:59.015727 | controller | - |_ Config Secrets: OK 2026-06-10 07:02:59.015735 | controller | - |_ Executor: OK 2026-06-10 07:02:59.015744 | controller | - |_ Merger: OK 2026-06-10 07:02:59.015753 | controller | - |_ Scheduler: OK 2026-06-10 07:02:59.015777 | controller | - |_ Web: Waiting ... 2026-06-10 07:02:59.015787 | controller | 2026-06-10 07:03:00.020361 | controller | 2026-06-10T07:03:00-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:03:00.021628 | controller | 2026-06-10T07:03:00-04:00 INFO Nothing to clean up. 2026-06-10 07:03:00.626139 | controller | 2026-06-10T07:03:00-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:03:00.626250 | controller | 2026-06-10T07:03:00-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:03:00.639762 | controller | 2026-06-10T07:03:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:03:00.639818 | controller | 2026-06-10T07:03:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:03:00.659475 | controller | 2026-06-10T07:03:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:03:00.659533 | controller | 2026-06-10T07:03:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:03:01.048287 | controller | 2026-06-10T07:03:01-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:03:01.441756 | controller | 2026-06-10T07:03:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:03:01.441814 | controller | 2026-06-10T07:03:01-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:03:02.813486 | controller | 2026-06-10T07:03:02-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:03:02.813538 | controller | 2026-06-10T07:03:02-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:03:04.406364 | controller | 2026-06-10T07:03:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:03:04.406436 | controller | 2026-06-10T07:03:04-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:03:05.618422 | controller | 2026-06-10T07:03:05-04:00 INFO 2026-06-10 07:03:05.618517 | controller | - Config: OK 2026-06-10 07:03:05.618532 | controller | - Gateway: OK 2026-06-10 07:03:05.618654 | controller | - GitServer: OK 2026-06-10 07:03:05.618730 | controller | - HoundSearch: OK 2026-06-10 07:03:05.618810 | controller | - LogJuicer: OK 2026-06-10 07:03:05.618878 | controller | - Logserver: OK 2026-06-10 07:03:05.618997 | controller | - MariaDB: OK 2026-06-10 07:03:05.619078 | controller | - NodePoolBuilder: OK 2026-06-10 07:03:05.619153 | controller | - NodePoolLauncher: OK 2026-06-10 07:03:05.619294 | controller | - Zookeeper: OK 2026-06-10 07:03:05.619350 | controller | - Zuul: OK 2026-06-10 07:03:05.619362 | controller | 2026-06-10 07:03:05.620270 | controller | • [162.304 seconds] 2026-06-10 07:03:05.620311 | controller | ------------------------------ 2026-06-10 07:03:05.620430 | controller | Project Private Keys Rotation zuul-web config-errors remain empty after rotation 2026-06-10 07:03:05.620443 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:97 2026-06-10 07:03:05.620597 | controller | 2026-06-10T07:03:05-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/internal/config-errors] 2026-06-10 07:03:05.741795 | controller | 2026-06-10T07:03:05-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/demo-tenant/config-errors] 2026-06-10 07:03:05.856369 | controller | • [0.236 seconds] 2026-06-10 07:03:05.856752 | controller | ------------------------------ 2026-06-10 07:03:05.856764 | controller | Project Private Keys Rotation project private keys changed when rotation was applied 2026-06-10 07:03:05.856783 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:101 2026-06-10 07:03:05.856799 | controller | 2026-06-10T07:03:05-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-10 07:03:05.856809 | controller | import configparser 2026-06-10 07:03:05.856817 | controller | import hashlib 2026-06-10 07:03:05.856825 | controller | from zuul.zk import ZooKeeperClient 2026-06-10 07:03:05.856834 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-10 07:03:05.856842 | controller | from zuul.lib import encryption 2026-06-10 07:03:05.856851 | controller | 2026-06-10 07:03:05.856859 | controller | config = configparser.ConfigParser() 2026-06-10 07:03:05.856867 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-10 07:03:05.856875 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-10 07:03:05.856898 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-10 07:03:05.856907 | controller | zk_client.connect() 2026-06-10 07:03:05.856915 | controller | lines = [] 2026-06-10 07:03:05.856923 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-10 07:03:05.856932 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-10 07:03:05.856940 | controller | created = obj["keys"][0]["created"] 2026-06-10 07:03:05.856964 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-10 07:03:05.856972 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-10 07:03:05.856981 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-10 07:03:05.856989 | controller | if "demo-project" in path: 2026-06-10 07:03:05.857000 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-10 07:03:05.857009 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-10 07:03:05.857017 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-10 07:03:05.857025 | controller | f.write(pem) 2026-06-10 07:03:05.857033 | controller | print("\n".join(sorted(lines))) 2026-06-10 07:03:05.857041 | controller | PYEOF 2026-06-10 07:03:05.857049 | controller | ] 2026-06-10 07:03:07.057369 | controller | STEP: Project keys after rotation: /keystorage/gerrit/config/config/secrets 1781089251 3bb863261bb39cb9 2026-06-10 07:03:07.057486 | controller | /keystorage/gerrit/demo-project/demo-project/secrets 1781089230 e8af00a689fecd83 2026-06-10 07:03:07.057497 | controller | /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 1781089247 c6d96b3dcf56980b 2026-06-10 07:03:07.057505 | controller | /keystorage/git-server/system-config/system-config/secrets 1781089252 b9c2c98d97529170 2026-06-10 07:03:07.057513 | controller | /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 1781089247 e2e870654040323f 2026-06-10 07:03:07.057521 | controller | @ 06/10/26 07:03:07.057 2026-06-10 07:03:07.057529 | controller | STEP: Pulling latest demo-project and reading .zuul.d/secrets.yaml @ 06/10/26 07:03:07.057 2026-06-10 07:03:07.128319 | controller | STEP: Writing secrets.yaml into zuul-scheduler pod for decryption @ 06/10/26 07:03:07.128 2026-06-10 07:03:07.128384 | controller | 2026-06-10T07:03:07-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat > /tmp/secrets.yaml] 2026-06-10 07:03:07.218369 | controller | STEP: Loading project keys state (writes demo-project key to /tmp/demo-project.key) @ 06/10/26 07:03:07.218 2026-06-10 07:03:07.218419 | controller | 2026-06-10T07:03:07-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-10 07:03:07.218426 | controller | import configparser 2026-06-10 07:03:07.218432 | controller | import hashlib 2026-06-10 07:03:07.218438 | controller | from zuul.zk import ZooKeeperClient 2026-06-10 07:03:07.218444 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-10 07:03:07.218449 | controller | from zuul.lib import encryption 2026-06-10 07:03:07.218455 | controller | 2026-06-10 07:03:07.218460 | controller | config = configparser.ConfigParser() 2026-06-10 07:03:07.218465 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-10 07:03:07.218470 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-10 07:03:07.218475 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-10 07:03:07.218480 | controller | zk_client.connect() 2026-06-10 07:03:07.218485 | controller | lines = [] 2026-06-10 07:03:07.218496 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-10 07:03:07.218501 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-10 07:03:07.218506 | controller | created = obj["keys"][0]["created"] 2026-06-10 07:03:07.218512 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-10 07:03:07.218517 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-10 07:03:07.218522 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-10 07:03:07.218527 | controller | if "demo-project" in path: 2026-06-10 07:03:07.218532 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-10 07:03:07.218542 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-10 07:03:07.218548 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-10 07:03:07.218553 | controller | f.write(pem) 2026-06-10 07:03:07.218558 | controller | print("\n".join(sorted(lines))) 2026-06-10 07:03:07.218563 | controller | PYEOF 2026-06-10 07:03:07.218569 | controller | ] 2026-06-10 07:03:08.358519 | controller | STEP: Decrypting with Zuul library @ 06/10/26 07:03:08.358 2026-06-10 07:03:08.358584 | controller | 2026-06-10T07:03:08-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-10 07:03:08.358593 | controller | from zuul.lib import encryption 2026-06-10 07:03:08.358629 | controller | import zuul.configloader 2026-06-10 07:03:08.358659 | controller | import zuul.model 2026-06-10 07:03:08.358695 | controller | 2026-06-10 07:03:08.358706 | controller | with open("/tmp/demo-project.key", "rb") as f: 2026-06-10 07:03:08.358746 | controller | private_secrets_key, _ = encryption.deserialize_rsa_keypair(f.read()) 2026-06-10 07:03:08.358772 | controller | with open("/tmp/secrets.yaml") as f: 2026-06-10 07:03:08.358807 | controller | yaml_content = f.read() 2026-06-10 07:03:08.358817 | controller | 2026-06-10 07:03:08.358855 | controller | parser = zuul.configloader.SecretParser(None) 2026-06-10 07:03:08.358889 | controller | sc = zuul.model.SourceContext(None, "project", None, "master", "path") 2026-06-10 07:03:08.358925 | controller | data = zuul.configloader.safe_load_yaml(yaml_content, sc) 2026-06-10 07:03:08.358961 | controller | for element in data: 2026-06-10 07:03:08.358997 | controller | if "secret" not in element: 2026-06-10 07:03:08.359030 | controller | continue 2026-06-10 07:03:08.359056 | controller | secret = parser.fromYaml(element["secret"]) 2026-06-10 07:03:08.359101 | controller | decrypted = secret.decrypt(private_secrets_key).secret_data 2026-06-10 07:03:08.359125 | controller | if isinstance(decrypted, dict) and "password" in decrypted: 2026-06-10 07:03:08.359155 | controller | print(decrypted["password"], end="") 2026-06-10 07:03:08.359207 | controller | elif isinstance(decrypted, str): 2026-06-10 07:03:08.359245 | controller | print(decrypted, end="") 2026-06-10 07:03:08.359285 | controller | else: 2026-06-10 07:03:08.359324 | controller | print(decrypted, end="") 2026-06-10 07:03:08.359360 | controller | break 2026-06-10 07:03:08.359389 | controller | PYEOF 2026-06-10 07:03:08.359418 | controller | ] 2026-06-10 07:03:09.172808 | controller | STEP: Decrypted secret: -----BEGIN OPENSSH PRIVATE KEY----- 2026-06-10 07:03:09.172883 | controller | b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW 2026-06-10 07:03:09.172891 | controller | QyNTUxOQAAACALZzdPTqfOMCsf4ABAMib8hfLqB3tNx4zNnjAcO/G/2wAAAKgMH4NBDB+D 2026-06-10 07:03:09.172897 | controller | QQAAAAtzc2gtZWQyNTUxOQAAACALZzdPTqfOMCsf4ABAMib8hfLqB3tNx4zNnjAcO/G/2w 2026-06-10 07:03:09.172902 | controller | AAAECPU88dFOHYB+bSDDLgyp8IJFakTxoaHqq4nHqZkMoN3QtnN09Op84wKx/gAEAyJvyF 2026-06-10 07:03:09.172907 | controller | 8uoHe03HjM2eMBw78b/bAAAAInp1dWwtd29ya2VyQG5wMDAwMDA2OTYxNi5ub3ZhbG9jYW 2026-06-10 07:03:09.172913 | controller | wBAgM= 2026-06-10 07:03:09.172919 | controller | -----END OPENSSH PRIVATE KEY----- @ 06/10/26 07:03:09.172 2026-06-10 07:03:09.172932 | controller | • [3.316 seconds] 2026-06-10 07:03:09.172939 | controller | ------------------------------ 2026-06-10 07:03:09.172944 | controller | Secret Rotations zuul-auth works 2026-06-10 07:03:09.172951 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:60 2026-06-10 07:03:09.177795 | controller | STEP: Deleting secrets not rotated by the CLI @ 06/10/26 07:03:09.177 2026-06-10 07:03:09.184187 | controller | STEP: Running secret rotation CLI @ 06/10/26 07:03:09.184 2026-06-10 07:03:09.191312 | controller | 2026-06-10T07:03:09-04:00 INFO Warning: At least one executor running, this may cause issues when rotating Zookeeper secrets. 2026-06-10 07:03:09.191347 | controller | 2026-06-10T07:03:09-04:00 INFO Rotating Keystore password... 2026-06-10 07:03:09.196866 | controller | 2026-06-10T07:03:09-04:00 INFO Creating secret, name: zuul-keystore-password-new 2026-06-10 07:03:09.196894 | controller | 2026-06-10T07:03:09-04:00 INFO Creating object, name: zuul-keystore-password-new 2026-06-10 07:03:10.371306 | controller | 2026-06-10T07:03:10-04:00 INFO Running pod execution pod: zuul-kazoo, command: [cat /var/lib/zuul/keys-backup.json] 2026-06-10 07:03:10.463708 | controller | 2026-06-10T07:03:10-04:00 INFO Updating object, name: zuul-keystore-password 2026-06-10 07:03:10.467534 | controller | 2026-06-10T07:03:10-04:00 INFO Deleting object, name: zuul-keystore-password-new 2026-06-10 07:03:10.476465 | controller | 2026-06-10T07:03:10-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 07:03:42.596105 | controller | 2026-06-10T07:03:42-04:00 INFO Killing every kazoo client... 2026-06-10 07:03:42.615652 | controller | 2026-06-10T07:03:42-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-10 07:03:42.615699 | controller | 2026-06-10T07:03:42-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-10 07:03:42.728864 | controller | 2026-06-10T07:03:42-04:00 INFO Deleting object, name: nodepool-builder 2026-06-10 07:03:42.733498 | controller | 2026-06-10T07:03:42-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-6678ff4998-lklrp launcher 2026-06-10 07:03:42.733535 | controller | 2026-06-10T07:03:42-04:00 INFO Running pod execution pod: nodepool-launcher-6678ff4998-lklrp, command: [kill -9 1] 2026-06-10 07:03:42.845029 | controller | 2026-06-10T07:03:42-04:00 INFO Deleting object, name: launcher 2026-06-10 07:03:42.847035 | controller | 2026-06-10T07:03:42-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-zlqvm mariadb-client 2026-06-10 07:03:42.847061 | controller | 2026-06-10T07:03:42-04:00 INFO Running pod execution pod: zuul-db-provision-zlqvm, command: [kill -9 1] 2026-06-10 07:03:42.863123 | controller | 2026-06-10T07:03:42-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 07:03:42.864545 | controller | 2026-06-10T07:03:42-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-10 07:03:42.864564 | controller | 2026-06-10T07:03:42-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-10 07:04:42.476368 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 07:04:42.479957 | controller | 2026-06-10T07:04:42-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-10 07:04:42.480084 | controller | 2026-06-10T07:04:42-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-10 07:04:42.575600 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 07:04:42.579120 | controller | 2026-06-10T07:04:42-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-10 07:04:42.579153 | controller | 2026-06-10T07:04:42-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-10 07:04:42.668309 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-10 07:04:42.672622 | controller | 2026-06-10T07:04:42-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-675cb66969-bffcm zuul-web 2026-06-10 07:04:42.672656 | controller | 2026-06-10T07:04:42-04:00 INFO Running pod execution pod: zuul-web-675cb66969-bffcm, command: [kill -9 1] 2026-06-10 07:04:42.785132 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:04:42.788893 | controller | 2026-06-10T07:04:42-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-txqg6 zuul-weeder 2026-06-10 07:04:42.788918 | controller | 2026-06-10T07:04:42-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-txqg6, command: [kill -9 1] 2026-06-10 07:04:42.884308 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 07:04:42.887288 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zookeeper 2026-06-10 07:04:42.890287 | controller | 2026-06-10T07:04:42-04:00 INFO Rotating Zuul Client Authenticator secret... 2026-06-10 07:04:42.892886 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-auth-secret 2026-06-10 07:04:42.902908 | controller | 2026-06-10T07:04:42-04:00 INFO Creating secret, name: zuul-auth-secret 2026-06-10 07:04:42.902977 | controller | 2026-06-10T07:04:42-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-10 07:04:42.907973 | controller | 2026-06-10T07:04:42-04:00 INFO Rotating Zuul DB Connection secret... 2026-06-10 07:04:42.912840 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-db-connection 2026-06-10 07:04:42.915103 | controller | 2026-06-10T07:04:42-04:00 INFO Rotating Zookeeper certificates... 2026-06-10 07:04:42.918799 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-10 07:04:42.922838 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-10 07:04:42.926294 | controller | 2026-06-10T07:04:42-04:00 INFO Force Restart impacted services... 2026-06-10 07:04:42.926371 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:04:42.928949 | controller | 2026-06-10T07:04:42-04:00 INFO Deleting object, name: zuul-web-675cb66969-bffcm 2026-06-10 07:04:42.931392 | controller | STEP: Reconciling @ 06/10/26 07:04:42.931 2026-06-10 07:04:43.093261 | controller | 2026-06-10T07:04:43-04:00 INFO Restarting zookeeper prior to client cert generation... 2026-06-10 07:04:43.113603 | controller | 2026-06-10T07:04:43-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-6678ff4998-lklrp launcher 2026-06-10 07:04:43.113673 | controller | 2026-06-10T07:04:43-04:00 INFO Running pod execution pod: nodepool-launcher-6678ff4998-lklrp, command: [kill -9 1] 2026-06-10 07:04:43.205449 | controller | 2026-06-10T07:04:43-04:00 INFO Deleting object, name: launcher 2026-06-10 07:04:43.208081 | controller | 2026-06-10T07:04:43-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-zlqvm mariadb-client 2026-06-10 07:04:43.208112 | controller | 2026-06-10T07:04:43-04:00 INFO Running pod execution pod: zuul-db-provision-zlqvm, command: [kill -9 1] 2026-06-10 07:04:43.224722 | controller | 2026-06-10T07:04:43-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 07:04:43.227632 | controller | 2026-06-10T07:04:43-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-10 07:04:43.227656 | controller | 2026-06-10T07:04:43-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-10 07:04:44.077669 | controller | 2026-06-10T07:04:44-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 07:04:44.080450 | controller | 2026-06-10T07:04:44-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-10 07:04:44.080522 | controller | 2026-06-10T07:04:44-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-10 07:05:12.892317 | controller | 2026-06-10T07:05:12-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-10 07:05:12.898270 | controller | 2026-06-10T07:05:12-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-675cb66969-bffcm zuul-web 2026-06-10 07:05:12.898434 | controller | 2026-06-10T07:05:12-04:00 INFO Running pod execution pod: zuul-web-675cb66969-bffcm, command: [kill -9 1] 2026-06-10 07:05:12.987524 | controller | 2026-06-10T07:05:12-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:05:12.990461 | controller | 2026-06-10T07:05:12-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-txqg6 zuul-weeder 2026-06-10 07:05:12.990497 | controller | 2026-06-10T07:05:12-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-txqg6, command: [kill -9 1] 2026-06-10 07:05:12.997422 | controller | 2026-06-10T07:05:12-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 07:05:12.999104 | controller | 2026-06-10T07:05:12-04:00 INFO Deleting object, name: zookeeper 2026-06-10 07:05:13.002667 | controller | 2026-06-10T07:05:13-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:05:13.038376 | controller | 2026-06-10T07:05:13-04:00 INFO Creating object, name: zuul-db-connection 2026-06-10 07:05:13.044276 | controller | 2026-06-10T07:05:13-04:00 INFO Re-provisioning db password 2026-06-10 07:05:13.044338 | controller | 2026-06-10T07:05:13-04:00 INFO Deleting object, name: zuul-db-provision 2026-06-10 07:05:13.048015 | controller | 2026-06-10T07:05:13-04:00 INFO Creating object, name: zuul-db-provision 2026-06-10 07:05:13.613626 | controller | 2026-06-10T07:05:13-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:05:13.613706 | controller | 2026-06-10T07:05:13-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:05:13.623291 | controller | 2026-06-10T07:05:13-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:05:13.758219 | controller | 2026-06-10T07:05:13-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:05:13.766128 | controller | 2026-06-10T07:05:13-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:05:13.766236 | controller | 2026-06-10T07:05:13-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:05:13.790038 | controller | 2026-06-10T07:05:13-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:05:13.790101 | controller | 2026-06-10T07:05:13-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:05:13.805318 | controller | 2026-06-10T07:05:13-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-10 07:05:14.005751 | controller | 2026-06-10T07:05:14-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-10 07:05:15.898063 | controller | 2026-06-10T07:05:15-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-10 07:05:17.264265 | controller | 2026-06-10T07:05:17-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-10 07:05:17.271026 | controller | 2026-06-10T07:05:17-04:00 INFO Zookeeper certs were updated, nuking ZK clients... 2026-06-10 07:05:17.282887 | controller | 2026-06-10T07:05:17-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-6678ff4998-lklrp launcher 2026-06-10 07:05:17.282950 | controller | 2026-06-10T07:05:17-04:00 INFO Running pod execution pod: nodepool-launcher-6678ff4998-lklrp, command: [kill -9 1] 2026-06-10 07:05:17.373391 | controller | 2026-06-10T07:05:17-04:00 INFO Deleting object, name: launcher 2026-06-10 07:05:17.375804 | controller | 2026-06-10T07:05:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-zlqvm mariadb-client 2026-06-10 07:05:17.375875 | controller | 2026-06-10T07:05:17-04:00 INFO Running pod execution pod: zuul-db-provision-zlqvm, command: [kill -9 1] 2026-06-10 07:05:17.393028 | controller | 2026-06-10T07:05:17-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 07:05:17.405576 | controller | 2026-06-10T07:05:17-04:00 INFO Creating object, name: zookeeper 2026-06-10 07:05:17.410280 | controller | 2026-06-10T07:05:17-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-10 07:05:17.410352 | controller | 2026-06-10T07:05:17-04:00 INFO 2026-06-10 07:05:17.410385 | controller | - Config: Waiting ... 2026-06-10 07:05:17.410392 | controller | - Gateway: OK 2026-06-10 07:05:17.410398 | controller | - GitServer: OK 2026-06-10 07:05:17.410404 | controller | - HoundSearch: OK 2026-06-10 07:05:17.410410 | controller | - LogJuicer: OK 2026-06-10 07:05:17.410415 | controller | - Logserver: OK 2026-06-10 07:05:17.410421 | controller | - MariaDB: Waiting ... 2026-06-10 07:05:17.410426 | controller | - Zookeeper: Waiting ... 2026-06-10 07:05:17.410432 | controller | 2026-06-10 07:05:18.414793 | controller | 2026-06-10T07:05:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:05:18.416844 | controller | 2026-06-10T07:05:18-04:00 INFO Nothing to clean up. 2026-06-10 07:05:18.451779 | controller | 2026-06-10T07:05:18-04:00 INFO Creating object, name: zuul-db-provision 2026-06-10 07:05:19.028748 | controller | 2026-06-10T07:05:19-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:05:19.028832 | controller | 2026-06-10T07:05:19-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:05:19.048241 | controller | 2026-06-10T07:05:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:05:19.048288 | controller | 2026-06-10T07:05:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:05:19.069638 | controller | 2026-06-10T07:05:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:05:19.069712 | controller | 2026-06-10T07:05:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:05:19.436067 | controller | 2026-06-10T07:05:19-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-10 07:06:01.599215 | controller | 2026-06-10T07:06:01-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 07:06:01.599293 | controller | 2026-06-10T07:06:01-04:00 INFO 2026-06-10 07:06:01.599299 | controller | - Config: Waiting ... 2026-06-10 07:06:01.599303 | controller | - Gateway: OK 2026-06-10 07:06:01.599307 | controller | - GitServer: OK 2026-06-10 07:06:01.599311 | controller | - HoundSearch: OK 2026-06-10 07:06:01.599315 | controller | - LogJuicer: OK 2026-06-10 07:06:01.599318 | controller | - Logserver: OK 2026-06-10 07:06:01.599322 | controller | - MariaDB: Waiting ... 2026-06-10 07:06:01.599326 | controller | - Zookeeper: OK 2026-06-10 07:06:01.599340 | controller | 2026-06-10 07:06:02.602953 | controller | 2026-06-10T07:06:02-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:06:02.604657 | controller | 2026-06-10T07:06:02-04:00 INFO Nothing to clean up. 2026-06-10 07:06:03.209848 | controller | 2026-06-10T07:06:03-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:06:03.209931 | controller | 2026-06-10T07:06:03-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:06:03.224874 | controller | 2026-06-10T07:06:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:06:03.225148 | controller | 2026-06-10T07:06:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:06:03.246220 | controller | 2026-06-10T07:06:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:06:03.246331 | controller | 2026-06-10T07:06:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:06:03.616626 | controller | 2026-06-10T07:06:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:06:04.019244 | controller | 2026-06-10T07:06:04-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:06:04.019320 | controller | 2026-06-10T07:06:04-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:06:04.214147 | controller | 2026-06-10T07:06:04-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 07:06:05.004049 | controller | 2026-06-10T07:06:05-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:06:05.004138 | controller | 2026-06-10T07:06:05-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:06:05.219570 | controller | 2026-06-10T07:06:05-04:00 INFO Creating object, name: zuul-scheduler 2026-06-10 07:06:05.236798 | controller | 2026-06-10T07:06:05-04:00 INFO Creating object, name: zuul-executor 2026-06-10 07:06:06.005521 | controller | 2026-06-10T07:06:06-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:06:06.806013 | controller | 2026-06-10T07:06:06-04:00 INFO Creating object, name: zuul-merger 2026-06-10 07:06:06.809073 | controller | 2026-06-10T07:06:06-04:00 INFO 2026-06-10 07:06:06.809103 | controller | - Config: Waiting ... 2026-06-10 07:06:06.809113 | controller | - Gateway: OK 2026-06-10 07:06:06.809123 | controller | - GitServer: OK 2026-06-10 07:06:06.809131 | controller | - HoundSearch: OK 2026-06-10 07:06:06.809139 | controller | - LogJuicer: OK 2026-06-10 07:06:06.809147 | controller | - Logserver: OK 2026-06-10 07:06:06.809155 | controller | - MariaDB: OK 2026-06-10 07:06:06.809163 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:06:06.809200 | controller | - NodePoolLauncher: OK 2026-06-10 07:06:06.809208 | controller | - Zookeeper: OK 2026-06-10 07:06:06.809216 | controller | - Zuul: Waiting ... 2026-06-10 07:06:06.809224 | controller | - |_ Config Secrets: OK 2026-06-10 07:06:06.809232 | controller | - |_ Executor: Waiting ... 2026-06-10 07:06:06.809240 | controller | - |_ Merger: Waiting ... 2026-06-10 07:06:06.809253 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:06:06.809261 | controller | - |_ Web: Waiting ... 2026-06-10 07:06:06.809269 | controller | 2026-06-10 07:06:07.813415 | controller | 2026-06-10T07:06:07-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:06:07.814999 | controller | 2026-06-10T07:06:07-04:00 INFO Nothing to clean up. 2026-06-10 07:06:08.418400 | controller | 2026-06-10T07:06:08-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:06:08.418475 | controller | 2026-06-10T07:06:08-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:06:08.428283 | controller | 2026-06-10T07:06:08-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:06:08.428361 | controller | 2026-06-10T07:06:08-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:06:08.454083 | controller | 2026-06-10T07:06:08-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:06:08.454185 | controller | 2026-06-10T07:06:08-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:06:08.830747 | controller | 2026-06-10T07:06:08-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:06:09.232947 | controller | 2026-06-10T07:06:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:06:09.233017 | controller | 2026-06-10T07:06:09-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:06:10.611649 | controller | 2026-06-10T07:06:10-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:06:40.715599 | controller | 2026-06-10T07:06:40-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:06:40.756399 | controller | 2026-06-10T07:06:40-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:06:40.756538 | controller | 2026-06-10T07:06:40-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:06:41.128469 | controller | 2026-06-10T07:06:41-04:00 INFO 2026-06-10 07:06:41.128538 | controller | - Config: Waiting ... 2026-06-10 07:06:41.128544 | controller | - Gateway: OK 2026-06-10 07:06:41.128548 | controller | - GitServer: OK 2026-06-10 07:06:41.128553 | controller | - HoundSearch: OK 2026-06-10 07:06:41.128587 | controller | - LogJuicer: OK 2026-06-10 07:06:41.128616 | controller | - Logserver: OK 2026-06-10 07:06:41.128637 | controller | - MariaDB: OK 2026-06-10 07:06:41.128669 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:06:41.128700 | controller | - NodePoolLauncher: OK 2026-06-10 07:06:41.128721 | controller | - Zookeeper: OK 2026-06-10 07:06:41.128755 | controller | - Zuul: Waiting ... 2026-06-10 07:06:41.128780 | controller | - |_ Config Secrets: OK 2026-06-10 07:06:41.128815 | controller | - |_ Executor: OK 2026-06-10 07:06:41.128848 | controller | - |_ Merger: OK 2026-06-10 07:06:41.128875 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:06:41.128894 | controller | - |_ Web: Waiting ... 2026-06-10 07:06:41.128901 | controller | 2026-06-10 07:06:42.134314 | controller | 2026-06-10T07:06:42-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:06:42.136205 | controller | 2026-06-10T07:06:42-04:00 INFO Nothing to clean up. 2026-06-10 07:06:42.737680 | controller | 2026-06-10T07:06:42-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:06:42.740036 | controller | 2026-06-10T07:06:42-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:06:42.755832 | controller | 2026-06-10T07:06:42-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:06:42.755916 | controller | 2026-06-10T07:06:42-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:06:42.775334 | controller | 2026-06-10T07:06:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:06:42.775417 | controller | 2026-06-10T07:06:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:06:43.154807 | controller | 2026-06-10T07:06:43-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:06:43.549064 | controller | 2026-06-10T07:06:43-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:06:43.549137 | controller | 2026-06-10T07:06:43-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:06:44.925846 | controller | 2026-06-10T07:06:44-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:07:07.023125 | controller | 2026-06-10T07:07:07-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 07:07:07.096333 | controller | 2026-06-10T07:07:07-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:07:07.096444 | controller | 2026-06-10T07:07:07-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:07:07.444372 | controller | 2026-06-10T07:07:07-04:00 INFO 2026-06-10 07:07:07.444447 | controller | - Config: Waiting ... 2026-06-10 07:07:07.444456 | controller | - Gateway: OK 2026-06-10 07:07:07.444463 | controller | - GitServer: OK 2026-06-10 07:07:07.444470 | controller | - HoundSearch: OK 2026-06-10 07:07:07.444477 | controller | - LogJuicer: OK 2026-06-10 07:07:07.444484 | controller | - Logserver: OK 2026-06-10 07:07:07.444490 | controller | - MariaDB: OK 2026-06-10 07:07:07.444497 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:07:07.444505 | controller | - NodePoolLauncher: OK 2026-06-10 07:07:07.444512 | controller | - Zookeeper: OK 2026-06-10 07:07:07.444518 | controller | - Zuul: Waiting ... 2026-06-10 07:07:07.444525 | controller | - |_ Config Secrets: OK 2026-06-10 07:07:07.444532 | controller | - |_ Executor: OK 2026-06-10 07:07:07.444539 | controller | - |_ Merger: OK 2026-06-10 07:07:07.444546 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:07:07.444554 | controller | - |_ Web: OK 2026-06-10 07:07:07.444561 | controller | 2026-06-10 07:07:08.450582 | controller | 2026-06-10T07:07:08-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:07:08.452499 | controller | 2026-06-10T07:07:08-04:00 INFO Nothing to clean up. 2026-06-10 07:07:09.059998 | controller | 2026-06-10T07:07:09-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:07:09.060055 | controller | 2026-06-10T07:07:09-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:07:09.067498 | controller | 2026-06-10T07:07:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:07:09.067540 | controller | 2026-06-10T07:07:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:07:09.081131 | controller | 2026-06-10T07:07:09-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:07:09.081189 | controller | 2026-06-10T07:07:09-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:07:09.477987 | controller | 2026-06-10T07:07:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:07:09.872264 | controller | 2026-06-10T07:07:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:07:09.872345 | controller | 2026-06-10T07:07:09-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:07:11.240862 | controller | 2026-06-10T07:07:11-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:07:11.240941 | controller | 2026-06-10T07:07:11-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:07:12.834392 | controller | 2026-06-10T07:07:12-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:07:12.834492 | controller | 2026-06-10T07:07:12-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:07:14.042503 | controller | 2026-06-10T07:07:14-04:00 INFO Creating base secret job 2026-06-10 07:07:14.042990 | controller | 2026-06-10T07:07:14-04:00 INFO Creating object, name: config-base-secret-1-2837-ns 2026-06-10 07:07:14.046848 | controller | 2026-06-10T07:07:14-04:00 INFO 2026-06-10 07:07:14.046896 | controller | - Config: Waiting ... 2026-06-10 07:07:14.046908 | controller | - Gateway: OK 2026-06-10 07:07:14.046916 | controller | - GitServer: OK 2026-06-10 07:07:14.046925 | controller | - HoundSearch: OK 2026-06-10 07:07:14.046932 | controller | - LogJuicer: OK 2026-06-10 07:07:14.046940 | controller | - Logserver: OK 2026-06-10 07:07:14.046955 | controller | - MariaDB: OK 2026-06-10 07:07:14.046963 | controller | - NodePoolBuilder: OK 2026-06-10 07:07:14.046971 | controller | - NodePoolLauncher: OK 2026-06-10 07:07:14.046979 | controller | - Zookeeper: OK 2026-06-10 07:07:14.047131 | controller | - Zuul: OK 2026-06-10 07:07:14.047139 | controller | 2026-06-10 07:07:15.052825 | controller | 2026-06-10T07:07:15-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:07:15.054224 | controller | 2026-06-10T07:07:15-04:00 INFO Nothing to clean up. 2026-06-10 07:07:15.655439 | controller | 2026-06-10T07:07:15-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:07:15.655530 | controller | 2026-06-10T07:07:15-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:07:15.665417 | controller | 2026-06-10T07:07:15-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:07:15.665501 | controller | 2026-06-10T07:07:15-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:07:15.688400 | controller | 2026-06-10T07:07:15-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:07:15.688487 | controller | 2026-06-10T07:07:15-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:07:16.072905 | controller | 2026-06-10T07:07:16-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:07:16.473813 | controller | 2026-06-10T07:07:16-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:07:16.473887 | controller | 2026-06-10T07:07:16-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:07:17.841565 | controller | 2026-06-10T07:07:17-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:07:17.841634 | controller | 2026-06-10T07:07:17-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:07:19.434827 | controller | 2026-06-10T07:07:19-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:07:19.434919 | controller | 2026-06-10T07:07:19-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:07:20.644795 | controller | 2026-06-10T07:07:20-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 07:07:20.831211 | controller | 2026-06-10T07:07:20-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 07:07:27.244447 | controller | 2026-06-10T07:07:27-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:07:27.248865 | controller | 2026-06-10T07:07:27-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-10 07:07:27.252463 | controller | 2026-06-10T07:07:27-04:00 INFO 2026-06-10 07:07:27.252520 | controller | - Config: Waiting ... 2026-06-10 07:07:27.252531 | controller | - Gateway: OK 2026-06-10 07:07:27.252540 | controller | - GitServer: OK 2026-06-10 07:07:27.252548 | controller | - HoundSearch: OK 2026-06-10 07:07:27.252556 | controller | - LogJuicer: OK 2026-06-10 07:07:27.252563 | controller | - Logserver: OK 2026-06-10 07:07:27.252571 | controller | - MariaDB: OK 2026-06-10 07:07:27.252579 | controller | - NodePoolBuilder: OK 2026-06-10 07:07:27.252588 | controller | - NodePoolLauncher: OK 2026-06-10 07:07:27.252599 | controller | - Zookeeper: OK 2026-06-10 07:07:27.252611 | controller | - Zuul: OK 2026-06-10 07:07:27.252622 | controller | 2026-06-10 07:07:28.259819 | controller | 2026-06-10T07:07:28-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:07:28.262323 | controller | 2026-06-10T07:07:28-04:00 INFO Nothing to clean up. 2026-06-10 07:07:28.863045 | controller | 2026-06-10T07:07:28-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:07:28.863107 | controller | 2026-06-10T07:07:28-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:07:28.875754 | controller | 2026-06-10T07:07:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:07:28.875819 | controller | 2026-06-10T07:07:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:07:28.901972 | controller | 2026-06-10T07:07:28-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:07:28.902041 | controller | 2026-06-10T07:07:28-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:07:29.286191 | controller | 2026-06-10T07:07:29-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:07:29.669718 | controller | 2026-06-10T07:07:29-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:07:29.669774 | controller | 2026-06-10T07:07:29-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:07:30.687292 | controller | 2026-06-10T07:07:30-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:07:30.687392 | controller | 2026-06-10T07:07:30-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:07:31.662431 | controller | 2026-06-10T07:07:31-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:07:32.473778 | controller | 2026-06-10T07:07:32-04:00 INFO 2026-06-10 07:07:32.473875 | controller | - Config: Waiting ... 2026-06-10 07:07:32.473887 | controller | - Gateway: OK 2026-06-10 07:07:32.473895 | controller | - GitServer: OK 2026-06-10 07:07:32.473906 | controller | - HoundSearch: OK 2026-06-10 07:07:32.473914 | controller | - LogJuicer: OK 2026-06-10 07:07:32.473921 | controller | - Logserver: OK 2026-06-10 07:07:32.473929 | controller | - MariaDB: OK 2026-06-10 07:07:32.473937 | controller | - NodePoolBuilder: OK 2026-06-10 07:07:32.473945 | controller | - NodePoolLauncher: OK 2026-06-10 07:07:32.473952 | controller | - Zookeeper: OK 2026-06-10 07:07:32.473960 | controller | - Zuul: Waiting ... 2026-06-10 07:07:32.473968 | controller | - |_ Config Secrets: OK 2026-06-10 07:07:32.473976 | controller | - |_ Executor: OK 2026-06-10 07:07:32.473984 | controller | - |_ Merger: OK 2026-06-10 07:07:32.473991 | controller | - |_ Scheduler: OK 2026-06-10 07:07:32.473999 | controller | - |_ Web: Waiting ... 2026-06-10 07:07:32.474007 | controller | 2026-06-10 07:07:33.479670 | controller | 2026-06-10T07:07:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:07:33.481621 | controller | 2026-06-10T07:07:33-04:00 INFO Nothing to clean up. 2026-06-10 07:07:34.083332 | controller | 2026-06-10T07:07:34-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:07:34.083386 | controller | 2026-06-10T07:07:34-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:07:34.096294 | controller | 2026-06-10T07:07:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:07:34.096351 | controller | 2026-06-10T07:07:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:07:34.121644 | controller | 2026-06-10T07:07:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:07:34.121678 | controller | 2026-06-10T07:07:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:07:34.500966 | controller | 2026-06-10T07:07:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:07:34.899107 | controller | 2026-06-10T07:07:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:07:34.899228 | controller | 2026-06-10T07:07:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:07:36.277072 | controller | 2026-06-10T07:07:36-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:07:36.277163 | controller | 2026-06-10T07:07:36-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:07:37.865769 | controller | 2026-06-10T07:07:37-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:07:37.865862 | controller | 2026-06-10T07:07:37-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:07:38.670669 | controller | 2026-06-10T07:07:38-04:00 INFO 2026-06-10 07:07:38.670750 | controller | - Config: Waiting ... 2026-06-10 07:07:38.670760 | controller | - Gateway: OK 2026-06-10 07:07:38.670769 | controller | - GitServer: OK 2026-06-10 07:07:38.670777 | controller | - HoundSearch: OK 2026-06-10 07:07:38.670785 | controller | - LogJuicer: OK 2026-06-10 07:07:38.670793 | controller | - Logserver: OK 2026-06-10 07:07:38.670800 | controller | - MariaDB: OK 2026-06-10 07:07:38.670808 | controller | - NodePoolBuilder: OK 2026-06-10 07:07:38.670816 | controller | - NodePoolLauncher: OK 2026-06-10 07:07:38.670823 | controller | - Zookeeper: OK 2026-06-10 07:07:38.670831 | controller | - Zuul: Waiting ... 2026-06-10 07:07:38.670839 | controller | - |_ Config Secrets: OK 2026-06-10 07:07:38.670846 | controller | - |_ Executor: OK 2026-06-10 07:07:38.670854 | controller | - |_ Merger: OK 2026-06-10 07:07:38.670862 | controller | - |_ Scheduler: OK 2026-06-10 07:07:38.670869 | controller | - |_ Web: Waiting ... 2026-06-10 07:07:38.670877 | controller | 2026-06-10 07:07:39.676489 | controller | 2026-06-10T07:07:39-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:07:39.678821 | controller | 2026-06-10T07:07:39-04:00 INFO Nothing to clean up. 2026-06-10 07:07:40.282129 | controller | 2026-06-10T07:07:40-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:07:40.282239 | controller | 2026-06-10T07:07:40-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:07:40.298329 | controller | 2026-06-10T07:07:40-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:07:40.298419 | controller | 2026-06-10T07:07:40-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:07:40.329736 | controller | 2026-06-10T07:07:40-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:07:40.329830 | controller | 2026-06-10T07:07:40-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:07:40.701356 | controller | 2026-06-10T07:07:40-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:07:41.100296 | controller | 2026-06-10T07:07:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:07:41.100419 | controller | 2026-06-10T07:07:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:07:42.475252 | controller | 2026-06-10T07:07:42-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:07:42.475344 | controller | 2026-06-10T07:07:42-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:07:44.066865 | controller | 2026-06-10T07:07:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:07:44.066950 | controller | 2026-06-10T07:07:44-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:07:44.866570 | controller | 2026-06-10T07:07:44-04:00 INFO 2026-06-10 07:07:44.866637 | controller | - Config: Waiting ... 2026-06-10 07:07:44.866643 | controller | - Gateway: OK 2026-06-10 07:07:44.866647 | controller | - GitServer: OK 2026-06-10 07:07:44.866650 | controller | - HoundSearch: OK 2026-06-10 07:07:44.866654 | controller | - LogJuicer: OK 2026-06-10 07:07:44.866657 | controller | - Logserver: OK 2026-06-10 07:07:44.866661 | controller | - MariaDB: OK 2026-06-10 07:07:44.866664 | controller | - NodePoolBuilder: OK 2026-06-10 07:07:44.866667 | controller | - NodePoolLauncher: OK 2026-06-10 07:07:44.866671 | controller | - Zookeeper: OK 2026-06-10 07:07:44.866674 | controller | - Zuul: Waiting ... 2026-06-10 07:07:44.866678 | controller | - |_ Config Secrets: OK 2026-06-10 07:07:44.866681 | controller | - |_ Executor: OK 2026-06-10 07:07:44.866685 | controller | - |_ Merger: OK 2026-06-10 07:07:44.866688 | controller | - |_ Scheduler: OK 2026-06-10 07:07:44.866691 | controller | - |_ Web: Waiting ... 2026-06-10 07:07:44.866695 | controller | 2026-06-10 07:07:45.871126 | controller | 2026-06-10T07:07:45-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:07:45.872896 | controller | 2026-06-10T07:07:45-04:00 INFO Nothing to clean up. 2026-06-10 07:07:46.479255 | controller | 2026-06-10T07:07:46-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:07:46.479355 | controller | 2026-06-10T07:07:46-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:07:46.493694 | controller | 2026-06-10T07:07:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:07:46.493771 | controller | 2026-06-10T07:07:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:07:46.515066 | controller | 2026-06-10T07:07:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:07:46.515214 | controller | 2026-06-10T07:07:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:07:46.890746 | controller | 2026-06-10T07:07:46-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:07:47.291193 | controller | 2026-06-10T07:07:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:07:47.291259 | controller | 2026-06-10T07:07:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:07:48.674671 | controller | 2026-06-10T07:07:48-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:07:48.674737 | controller | 2026-06-10T07:07:48-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:07:50.263933 | controller | 2026-06-10T07:07:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:07:50.264011 | controller | 2026-06-10T07:07:50-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:07:51.076814 | controller | 2026-06-10T07:07:51-04:00 INFO 2026-06-10 07:07:51.076892 | controller | - Config: Waiting ... 2026-06-10 07:07:51.076903 | controller | - Gateway: OK 2026-06-10 07:07:51.076911 | controller | - GitServer: OK 2026-06-10 07:07:51.076920 | controller | - HoundSearch: OK 2026-06-10 07:07:51.076928 | controller | - LogJuicer: OK 2026-06-10 07:07:51.076936 | controller | - Logserver: OK 2026-06-10 07:07:51.076943 | controller | - MariaDB: OK 2026-06-10 07:07:51.076951 | controller | - NodePoolBuilder: OK 2026-06-10 07:07:51.076959 | controller | - NodePoolLauncher: OK 2026-06-10 07:07:51.076967 | controller | - Zookeeper: OK 2026-06-10 07:07:51.076974 | controller | - Zuul: Waiting ... 2026-06-10 07:07:51.076982 | controller | - |_ Config Secrets: OK 2026-06-10 07:07:51.076990 | controller | - |_ Executor: OK 2026-06-10 07:07:51.076998 | controller | - |_ Merger: OK 2026-06-10 07:07:51.077005 | controller | - |_ Scheduler: OK 2026-06-10 07:07:51.077013 | controller | - |_ Web: Waiting ... 2026-06-10 07:07:51.077021 | controller | 2026-06-10 07:07:52.083596 | controller | 2026-06-10T07:07:52-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:07:52.085999 | controller | 2026-06-10T07:07:52-04:00 INFO Nothing to clean up. 2026-06-10 07:07:52.686556 | controller | 2026-06-10T07:07:52-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:07:52.686648 | controller | 2026-06-10T07:07:52-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:07:52.697762 | controller | 2026-06-10T07:07:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:07:52.697839 | controller | 2026-06-10T07:07:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:07:52.712754 | controller | 2026-06-10T07:07:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:07:52.712822 | controller | 2026-06-10T07:07:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:07:53.108004 | controller | 2026-06-10T07:07:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:07:53.501712 | controller | 2026-06-10T07:07:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:07:53.501796 | controller | 2026-06-10T07:07:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:07:54.878885 | controller | 2026-06-10T07:07:54-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:07:54.879004 | controller | 2026-06-10T07:07:54-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:07:56.465126 | controller | 2026-06-10T07:07:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:07:56.465251 | controller | 2026-06-10T07:07:56-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:07:57.276730 | controller | 2026-06-10T07:07:57-04:00 INFO 2026-06-10 07:07:57.276826 | controller | - Config: Waiting ... 2026-06-10 07:07:57.276852 | controller | - Gateway: OK 2026-06-10 07:07:57.276862 | controller | - GitServer: OK 2026-06-10 07:07:57.276870 | controller | - HoundSearch: OK 2026-06-10 07:07:57.276878 | controller | - LogJuicer: OK 2026-06-10 07:07:57.276885 | controller | - Logserver: OK 2026-06-10 07:07:57.276893 | controller | - MariaDB: OK 2026-06-10 07:07:57.276900 | controller | - NodePoolBuilder: OK 2026-06-10 07:07:57.276908 | controller | - NodePoolLauncher: OK 2026-06-10 07:07:57.276916 | controller | - Zookeeper: OK 2026-06-10 07:07:57.276923 | controller | - Zuul: Waiting ... 2026-06-10 07:07:57.276931 | controller | - |_ Config Secrets: OK 2026-06-10 07:07:57.276939 | controller | - |_ Executor: OK 2026-06-10 07:07:57.276947 | controller | - |_ Merger: OK 2026-06-10 07:07:57.276954 | controller | - |_ Scheduler: OK 2026-06-10 07:07:57.276962 | controller | - |_ Web: Waiting ... 2026-06-10 07:07:57.276970 | controller | 2026-06-10 07:07:58.281407 | controller | 2026-06-10T07:07:58-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:07:58.283554 | controller | 2026-06-10T07:07:58-04:00 INFO Nothing to clean up. 2026-06-10 07:07:58.886507 | controller | 2026-06-10T07:07:58-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:07:58.886576 | controller | 2026-06-10T07:07:58-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:07:58.897398 | controller | 2026-06-10T07:07:58-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:07:58.897452 | controller | 2026-06-10T07:07:58-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:07:58.915808 | controller | 2026-06-10T07:07:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:07:58.915879 | controller | 2026-06-10T07:07:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:07:59.301346 | controller | 2026-06-10T07:07:59-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:07:59.699761 | controller | 2026-06-10T07:07:59-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:07:59.699813 | controller | 2026-06-10T07:07:59-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:01.077099 | controller | 2026-06-10T07:08:01-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:01.077212 | controller | 2026-06-10T07:08:01-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:02.669006 | controller | 2026-06-10T07:08:02-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:02.669077 | controller | 2026-06-10T07:08:02-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:08:03.482445 | controller | 2026-06-10T07:08:03-04:00 INFO 2026-06-10 07:08:03.482541 | controller | - Config: Waiting ... 2026-06-10 07:08:03.482557 | controller | - Gateway: OK 2026-06-10 07:08:03.482570 | controller | - GitServer: OK 2026-06-10 07:08:03.482581 | controller | - HoundSearch: OK 2026-06-10 07:08:03.482593 | controller | - LogJuicer: OK 2026-06-10 07:08:03.482604 | controller | - Logserver: OK 2026-06-10 07:08:03.482614 | controller | - MariaDB: OK 2026-06-10 07:08:03.482623 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:03.482634 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:03.482646 | controller | - Zookeeper: OK 2026-06-10 07:08:03.482656 | controller | - Zuul: Waiting ... 2026-06-10 07:08:03.482668 | controller | - |_ Config Secrets: OK 2026-06-10 07:08:03.482679 | controller | - |_ Executor: OK 2026-06-10 07:08:03.482690 | controller | - |_ Merger: OK 2026-06-10 07:08:03.482700 | controller | - |_ Scheduler: OK 2026-06-10 07:08:03.482711 | controller | - |_ Web: Waiting ... 2026-06-10 07:08:03.482721 | controller | 2026-06-10 07:08:04.489473 | controller | 2026-06-10T07:08:04-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:04.491452 | controller | 2026-06-10T07:08:04-04:00 INFO Nothing to clean up. 2026-06-10 07:08:05.095741 | controller | 2026-06-10T07:08:05-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:05.095826 | controller | 2026-06-10T07:08:05-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:05.110654 | controller | 2026-06-10T07:08:05-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:08:05.110754 | controller | 2026-06-10T07:08:05-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:08:05.132602 | controller | 2026-06-10T07:08:05-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:05.132684 | controller | 2026-06-10T07:08:05-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:05.501474 | controller | 2026-06-10T07:08:05-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:05.902509 | controller | 2026-06-10T07:08:05-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:05.902592 | controller | 2026-06-10T07:08:05-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:07.280900 | controller | 2026-06-10T07:08:07-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:07.280998 | controller | 2026-06-10T07:08:07-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:08.866579 | controller | 2026-06-10T07:08:08-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:08.866688 | controller | 2026-06-10T07:08:08-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:08:09.674310 | controller | 2026-06-10T07:08:09-04:00 INFO 2026-06-10 07:08:09.674396 | controller | - Config: Waiting ... 2026-06-10 07:08:09.674407 | controller | - Gateway: OK 2026-06-10 07:08:09.674417 | controller | - GitServer: OK 2026-06-10 07:08:09.674426 | controller | - HoundSearch: OK 2026-06-10 07:08:09.674434 | controller | - LogJuicer: OK 2026-06-10 07:08:09.674443 | controller | - Logserver: OK 2026-06-10 07:08:09.674451 | controller | - MariaDB: OK 2026-06-10 07:08:09.674459 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:09.674467 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:09.674476 | controller | - Zookeeper: OK 2026-06-10 07:08:09.674484 | controller | - Zuul: Waiting ... 2026-06-10 07:08:09.674492 | controller | - |_ Config Secrets: OK 2026-06-10 07:08:09.674501 | controller | - |_ Executor: OK 2026-06-10 07:08:09.674509 | controller | - |_ Merger: OK 2026-06-10 07:08:09.674517 | controller | - |_ Scheduler: OK 2026-06-10 07:08:09.674526 | controller | - |_ Web: Waiting ... 2026-06-10 07:08:09.674534 | controller | 2026-06-10 07:08:10.681729 | controller | 2026-06-10T07:08:10-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:10.684610 | controller | 2026-06-10T07:08:10-04:00 INFO Nothing to clean up. 2026-06-10 07:08:11.285973 | controller | 2026-06-10T07:08:11-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:11.286050 | controller | 2026-06-10T07:08:11-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:11.295216 | controller | 2026-06-10T07:08:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:08:11.295266 | controller | 2026-06-10T07:08:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:08:11.310480 | controller | 2026-06-10T07:08:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:11.310547 | controller | 2026-06-10T07:08:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:11.700435 | controller | 2026-06-10T07:08:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:12.095057 | controller | 2026-06-10T07:08:12-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:12.095146 | controller | 2026-06-10T07:08:12-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:13.475638 | controller | 2026-06-10T07:08:13-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:13.475913 | controller | 2026-06-10T07:08:13-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:15.066894 | controller | 2026-06-10T07:08:15-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:15.067005 | controller | 2026-06-10T07:08:15-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:08:15.875257 | controller | 2026-06-10T07:08:15-04:00 INFO 2026-06-10 07:08:15.875335 | controller | - Config: Waiting ... 2026-06-10 07:08:15.875343 | controller | - Gateway: OK 2026-06-10 07:08:15.875349 | controller | - GitServer: OK 2026-06-10 07:08:15.875354 | controller | - HoundSearch: OK 2026-06-10 07:08:15.875359 | controller | - LogJuicer: OK 2026-06-10 07:08:15.875364 | controller | - Logserver: OK 2026-06-10 07:08:15.875369 | controller | - MariaDB: OK 2026-06-10 07:08:15.875373 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:15.875378 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:15.875383 | controller | - Zookeeper: OK 2026-06-10 07:08:15.875388 | controller | - Zuul: Waiting ... 2026-06-10 07:08:15.875393 | controller | - |_ Config Secrets: OK 2026-06-10 07:08:15.875398 | controller | - |_ Executor: OK 2026-06-10 07:08:15.875402 | controller | - |_ Merger: OK 2026-06-10 07:08:15.875407 | controller | - |_ Scheduler: OK 2026-06-10 07:08:15.875437 | controller | - |_ Web: Waiting ... 2026-06-10 07:08:15.875443 | controller | 2026-06-10 07:08:16.883811 | controller | 2026-06-10T07:08:16-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:16.886350 | controller | 2026-06-10T07:08:16-04:00 INFO Nothing to clean up. 2026-06-10 07:08:17.484832 | controller | 2026-06-10T07:08:17-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:17.484923 | controller | 2026-06-10T07:08:17-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:17.497473 | controller | 2026-06-10T07:08:17-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:08:17.497523 | controller | 2026-06-10T07:08:17-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:08:17.515034 | controller | 2026-06-10T07:08:17-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:17.515094 | controller | 2026-06-10T07:08:17-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:17.904980 | controller | 2026-06-10T07:08:17-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:18.294940 | controller | 2026-06-10T07:08:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:18.295010 | controller | 2026-06-10T07:08:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:19.678060 | controller | 2026-06-10T07:08:19-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:19.678150 | controller | 2026-06-10T07:08:19-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:21.262646 | controller | 2026-06-10T07:08:21-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:21.262733 | controller | 2026-06-10T07:08:21-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:08:22.070417 | controller | 2026-06-10T07:08:22-04:00 INFO 2026-06-10 07:08:22.070501 | controller | - Config: Waiting ... 2026-06-10 07:08:22.070512 | controller | - Gateway: OK 2026-06-10 07:08:22.070566 | controller | - GitServer: OK 2026-06-10 07:08:22.070602 | controller | - HoundSearch: OK 2026-06-10 07:08:22.070646 | controller | - LogJuicer: OK 2026-06-10 07:08:22.070711 | controller | - Logserver: OK 2026-06-10 07:08:22.070764 | controller | - MariaDB: OK 2026-06-10 07:08:22.070816 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:22.070876 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:22.070941 | controller | - Zookeeper: OK 2026-06-10 07:08:22.070989 | controller | - Zuul: Waiting ... 2026-06-10 07:08:22.071045 | controller | - |_ Config Secrets: OK 2026-06-10 07:08:22.071102 | controller | - |_ Executor: OK 2026-06-10 07:08:22.071184 | controller | - |_ Merger: OK 2026-06-10 07:08:22.071251 | controller | - |_ Scheduler: OK 2026-06-10 07:08:22.071299 | controller | - |_ Web: Waiting ... 2026-06-10 07:08:22.071313 | controller | 2026-06-10 07:08:23.076638 | controller | 2026-06-10T07:08:23-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:23.078217 | controller | 2026-06-10T07:08:23-04:00 INFO Nothing to clean up. 2026-06-10 07:08:23.682932 | controller | 2026-06-10T07:08:23-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:23.683011 | controller | 2026-06-10T07:08:23-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:23.694758 | controller | 2026-06-10T07:08:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:08:23.694783 | controller | 2026-06-10T07:08:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:08:23.714368 | controller | 2026-06-10T07:08:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:23.714424 | controller | 2026-06-10T07:08:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:24.097032 | controller | 2026-06-10T07:08:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:24.488861 | controller | 2026-06-10T07:08:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:24.488954 | controller | 2026-06-10T07:08:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:25.877600 | controller | 2026-06-10T07:08:25-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:25.877677 | controller | 2026-06-10T07:08:25-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:27.464914 | controller | 2026-06-10T07:08:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:27.465014 | controller | 2026-06-10T07:08:27-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:08:28.277592 | controller | 2026-06-10T07:08:28-04:00 INFO 2026-06-10 07:08:28.277673 | controller | - Config: Waiting ... 2026-06-10 07:08:28.277680 | controller | - Gateway: OK 2026-06-10 07:08:28.277686 | controller | - GitServer: OK 2026-06-10 07:08:28.277692 | controller | - HoundSearch: OK 2026-06-10 07:08:28.277697 | controller | - LogJuicer: OK 2026-06-10 07:08:28.277702 | controller | - Logserver: OK 2026-06-10 07:08:28.277708 | controller | - MariaDB: OK 2026-06-10 07:08:28.277728 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:28.277733 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:28.277738 | controller | - Zookeeper: OK 2026-06-10 07:08:28.277743 | controller | - Zuul: Waiting ... 2026-06-10 07:08:28.277749 | controller | - |_ Config Secrets: OK 2026-06-10 07:08:28.277754 | controller | - |_ Executor: OK 2026-06-10 07:08:28.277759 | controller | - |_ Merger: OK 2026-06-10 07:08:28.277764 | controller | - |_ Scheduler: OK 2026-06-10 07:08:28.277769 | controller | - |_ Web: Waiting ... 2026-06-10 07:08:28.277775 | controller | 2026-06-10 07:08:29.289573 | controller | 2026-06-10T07:08:29-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:29.292279 | controller | 2026-06-10T07:08:29-04:00 INFO Nothing to clean up. 2026-06-10 07:08:29.888573 | controller | 2026-06-10T07:08:29-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:29.888639 | controller | 2026-06-10T07:08:29-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:29.899827 | controller | 2026-06-10T07:08:29-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:08:29.899871 | controller | 2026-06-10T07:08:29-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:08:29.916230 | controller | 2026-06-10T07:08:29-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:29.916305 | controller | 2026-06-10T07:08:29-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:30.303134 | controller | 2026-06-10T07:08:30-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:30.701769 | controller | 2026-06-10T07:08:30-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:30.701859 | controller | 2026-06-10T07:08:30-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:32.079320 | controller | 2026-06-10T07:08:32-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:32.079412 | controller | 2026-06-10T07:08:32-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:33.663774 | controller | 2026-06-10T07:08:33-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:33.663845 | controller | 2026-06-10T07:08:33-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:08:34.871857 | controller | 2026-06-10T07:08:34-04:00 INFO 2026-06-10 07:08:34.871933 | controller | - Config: OK 2026-06-10 07:08:34.871940 | controller | - Gateway: OK 2026-06-10 07:08:34.871945 | controller | - GitServer: OK 2026-06-10 07:08:34.871950 | controller | - HoundSearch: OK 2026-06-10 07:08:34.871955 | controller | - LogJuicer: OK 2026-06-10 07:08:34.871960 | controller | - Logserver: OK 2026-06-10 07:08:34.871965 | controller | - MariaDB: OK 2026-06-10 07:08:34.871970 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:34.871975 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:34.871979 | controller | - Zookeeper: OK 2026-06-10 07:08:34.871984 | controller | - Zuul: OK 2026-06-10 07:08:34.871990 | controller | 2026-06-10 07:08:34.872107 | controller | STEP: Validating secret changed @ 06/10/26 07:08:34.872 2026-06-10 07:08:34.883515 | controller | 2026-06-10T07:08:34-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /etc/zuul/zuul.conf] 2026-06-10 07:08:34.994565 | controller | STEP: Checking the secret is in the zuul.conf @ 06/10/26 07:08:34.994 2026-06-10 07:08:34.996829 | controller | STEP: Checking zuul-client works @ 06/10/26 07:08:34.996 2026-06-10 07:08:34.997088 | controller | 2026-06-10T07:08:34-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c zuul-client -v autohold --tenant internal --project system-config --job testytest --reason testy 2>&1] 2026-06-10 07:08:35.676666 | controller | • [326.504 seconds] 2026-06-10 07:08:35.676736 | controller | ------------------------------ 2026-06-10 07:08:35.676748 | controller | Secret Rotations zuul-db works 2026-06-10 07:08:35.676755 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:71 2026-06-10 07:08:35.676770 | controller | STEP: Checking the secret is in the zuul.conf @ 06/10/26 07:08:35.676 2026-06-10 07:08:35.678942 | controller | STEP: Checking zuul-web works @ 06/10/26 07:08:35.678 2026-06-10 07:08:35.678961 | controller | 2026-06-10T07:08:35-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl zuul-web:9000/api/tenant/demo-tenant/builds] 2026-06-10 07:08:35.849918 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 07:08:35.851527 | controller | Dload Upload Total Spent Left Speed 2026-06-10 07:08:35.895328 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 1854 100 1854 0 0 40304 0 --:--:-- --:--:-- --:--:-- 41200 2026-06-10 07:08:35.897564 | controller | • [0.221 seconds] 2026-06-10 07:08:35.897595 | controller | ------------------------------ 2026-06-10 07:08:35.897777 | controller | Secret Rotations zuul-zk works 2026-06-10 07:08:35.897785 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:81 2026-06-10 07:08:35.897955 | controller | STEP: Checking the key in zuul pod @ 06/10/26 07:08:35.897 2026-06-10 07:08:35.898086 | controller | 2026-06-10T07:08:35-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /tls/client/tls.key] 2026-06-10 07:08:36.009630 | controller | STEP: Connecting with kazoo @ 06/10/26 07:08:36.009 2026-06-10 07:08:36.009701 | controller | 2026-06-10T07:08:36-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [python3 -c import os, kazoo.client; os.chdir('/tls/client'); hosts=list(filter(lambda l: l.startswith('hosts'), open('/etc/zuul/zuul.conf').readlines()))[0].split('=')[1].strip(); client = kazoo.client.KazooClient(hosts, use_ssl=True, keyfile='tls.key', certfile='tls.crt', ca='ca.crt'); client.start(); print(client.get_children('/keystorage2'))] 2026-06-10 07:08:36.395497 | controller | • [0.498 seconds] 2026-06-10 07:08:36.395597 | controller | ------------------------------ 2026-06-10 07:08:36.395723 | controller | gateway tests When setting up extra gateway configuration for a maintenance Should reconcile 2026-06-10 07:08:36.395740 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:28 2026-06-10 07:08:36.395889 | controller | 2026-06-10T07:08:36-04:00 INFO Creating object, name: maintenance-config 2026-06-10 07:08:36.402280 | controller | 2026-06-10T07:08:36-04:00 INFO Creating object, name: maintenance-static-files 2026-06-10 07:08:36.410324 | controller | 2026-06-10T07:08:36-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:36.412130 | controller | 2026-06-10T07:08:36-04:00 INFO Nothing to clean up. 2026-06-10 07:08:37.279111 | controller | 2026-06-10T07:08:37-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:37.279242 | controller | 2026-06-10T07:08:37-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:37.293078 | controller | 2026-06-10T07:08:37-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:08:37.457317 | controller | 2026-06-10T07:08:37-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:08:37.471683 | controller | 2026-06-10T07:08:37-04:00 INFO Statefulset "gateway" configuration changed, applying... 2026-06-10 07:08:37.471740 | controller | 2026-06-10T07:08:37-04:00 DEBUG Reason: annotation 'config-hash' changed from '90a6261c7272ad73ee593bd22fc08aa0b50c9e09eb380107181e1e58b851202b' to 'c04b8e5064e2a2985ef09fcc1ab4c2dda05fbcb2cd18006ff3d01f2b963e17d8' 2026-06-10 07:08:37.471779 | controller | 2026-06-10T07:08:37-04:00 INFO Updating object, name: gateway 2026-06-10 07:08:37.508337 | controller | 2026-06-10T07:08:37-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:37.508422 | controller | 2026-06-10T07:08:37-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:37.698243 | controller | 2026-06-10T07:08:37-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:38.093163 | controller | 2026-06-10T07:08:38-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:38.093265 | controller | 2026-06-10T07:08:38-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:39.480067 | controller | 2026-06-10T07:08:39-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:39.480158 | controller | 2026-06-10T07:08:39-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:41.064127 | controller | 2026-06-10T07:08:41-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:41.064229 | controller | 2026-06-10T07:08:41-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:08:42.268885 | controller | 2026-06-10T07:08:42-04:00 INFO 2026-06-10 07:08:42.268956 | controller | - Config: OK 2026-06-10 07:08:42.268961 | controller | - Gateway: Waiting ... 2026-06-10 07:08:42.268965 | controller | - GitServer: OK 2026-06-10 07:08:42.268969 | controller | - HoundSearch: OK 2026-06-10 07:08:42.268974 | controller | - LogJuicer: OK 2026-06-10 07:08:42.268995 | controller | - Logserver: OK 2026-06-10 07:08:42.269019 | controller | - MariaDB: OK 2026-06-10 07:08:42.269044 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:42.269102 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:42.269113 | controller | - Zookeeper: OK 2026-06-10 07:08:42.269121 | controller | - Zuul: OK 2026-06-10 07:08:42.269128 | controller | 2026-06-10 07:08:43.277717 | controller | 2026-06-10T07:08:43-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:43.279264 | controller | 2026-06-10T07:08:43-04:00 INFO Nothing to clean up. 2026-06-10 07:08:43.880364 | controller | 2026-06-10T07:08:43-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:43.880472 | controller | 2026-06-10T07:08:43-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:43.898710 | controller | 2026-06-10T07:08:43-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:08:43.898787 | controller | 2026-06-10T07:08:43-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:08:44.065548 | controller | 2026-06-10T07:08:44-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:44.065632 | controller | 2026-06-10T07:08:44-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:44.300983 | controller | 2026-06-10T07:08:44-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:45.071030 | controller | 2026-06-10T07:08:45-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:45.071269 | controller | 2026-06-10T07:08:45-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:46.475827 | controller | 2026-06-10T07:08:46-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:46.475931 | controller | 2026-06-10T07:08:46-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:48.063559 | controller | 2026-06-10T07:08:48-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:48.063652 | controller | 2026-06-10T07:08:48-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:08:49.273053 | controller | 2026-06-10T07:08:49-04:00 INFO 2026-06-10 07:08:49.273127 | controller | - Config: OK 2026-06-10 07:08:49.273135 | controller | - Gateway: OK 2026-06-10 07:08:49.273141 | controller | - GitServer: OK 2026-06-10 07:08:49.273147 | controller | - HoundSearch: OK 2026-06-10 07:08:49.273152 | controller | - LogJuicer: OK 2026-06-10 07:08:49.273158 | controller | - Logserver: OK 2026-06-10 07:08:49.273186 | controller | - MariaDB: OK 2026-06-10 07:08:49.273193 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:49.273198 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:49.273204 | controller | - Zookeeper: OK 2026-06-10 07:08:49.273210 | controller | - Zuul: OK 2026-06-10 07:08:49.273215 | controller | 2026-06-10 07:08:49.273254 | controller | • [12.877 seconds] 2026-06-10 07:08:49.273264 | controller | ------------------------------ 2026-06-10 07:08:49.273279 | controller | gateway tests When setting up extra gateway configuration for a maintenance Should serve HTTP 503 and the expected content 2026-06-10 07:08:49.273286 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:55 2026-06-10 07:08:49.273291 | controller | 2026-06-10T07:08:49-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -o - -I gateway:8080/zuul/] 2026-06-10 07:08:49.387063 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 07:08:49.387137 | controller | Dload Upload Total Spent Left Speed 2026-06-10 07:08:49.388679 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 17 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2026-06-10 07:08:49.391612 | controller | 2026-06-10T07:08:49-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl gateway:8080/zuul/api/components] 2026-06-10 07:08:49.477006 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 07:08:49.477079 | controller | Dload Upload Total Spent Left Speed 2026-06-10 07:08:49.478763 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 17 100 17 0 0 17000 0 --:--:-- --:--:-- --:--:-- 17000 2026-06-10 07:08:49.480474 | controller | • [0.207 seconds] 2026-06-10 07:08:49.480502 | controller | ------------------------------ 2026-06-10 07:08:49.480545 | controller | gateway tests When removing the gateway's extra config Should reconcile 2026-06-10 07:08:49.480551 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:65 2026-06-10 07:08:49.486076 | controller | 2026-06-10T07:08:49-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:49.487936 | controller | 2026-06-10T07:08:49-04:00 INFO Nothing to clean up. 2026-06-10 07:08:50.089610 | controller | 2026-06-10T07:08:50-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:50.089693 | controller | 2026-06-10T07:08:50-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:50.098715 | controller | 2026-06-10T07:08:50-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:08:50.184595 | controller | 2026-06-10T07:08:50-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:08:50.263091 | controller | 2026-06-10T07:08:50-04:00 INFO Statefulset "gateway" configuration changed, applying... 2026-06-10 07:08:50.263209 | controller | 2026-06-10T07:08:50-04:00 DEBUG Reason: annotation 'config-hash' changed from 'c04b8e5064e2a2985ef09fcc1ab4c2dda05fbcb2cd18006ff3d01f2b963e17d8' to '90a6261c7272ad73ee593bd22fc08aa0b50c9e09eb380107181e1e58b851202b' 2026-06-10 07:08:50.263228 | controller | 2026-06-10T07:08:50-04:00 INFO Updating object, name: gateway 2026-06-10 07:08:50.664608 | controller | 2026-06-10T07:08:50-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:50.664693 | controller | 2026-06-10T07:08:50-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:50.880477 | controller | 2026-06-10T07:08:50-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:51.668411 | controller | 2026-06-10T07:08:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:51.668494 | controller | 2026-06-10T07:08:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:53.074132 | controller | 2026-06-10T07:08:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:53.074243 | controller | 2026-06-10T07:08:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:08:54.664787 | controller | 2026-06-10T07:08:54-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:08:54.664903 | controller | 2026-06-10T07:08:54-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:08:55.875639 | controller | 2026-06-10T07:08:55-04:00 INFO 2026-06-10 07:08:55.875744 | controller | - Config: OK 2026-06-10 07:08:55.875756 | controller | - Gateway: Waiting ... 2026-06-10 07:08:55.875764 | controller | - GitServer: OK 2026-06-10 07:08:55.875772 | controller | - HoundSearch: OK 2026-06-10 07:08:55.875780 | controller | - LogJuicer: OK 2026-06-10 07:08:55.875788 | controller | - Logserver: OK 2026-06-10 07:08:55.875795 | controller | - MariaDB: OK 2026-06-10 07:08:55.875803 | controller | - NodePoolBuilder: OK 2026-06-10 07:08:55.875811 | controller | - NodePoolLauncher: OK 2026-06-10 07:08:55.875818 | controller | - Zookeeper: OK 2026-06-10 07:08:55.875826 | controller | - Zuul: OK 2026-06-10 07:08:55.875835 | controller | 2026-06-10 07:08:56.882438 | controller | 2026-06-10T07:08:56-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:08:56.883924 | controller | 2026-06-10T07:08:56-04:00 INFO Nothing to clean up. 2026-06-10 07:08:57.496111 | controller | 2026-06-10T07:08:57-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 07:08:57.496225 | controller | 2026-06-10T07:08:57-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 07:08:57.518339 | controller | 2026-06-10T07:08:57-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:08:57.518412 | controller | 2026-06-10T07:08:57-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:08:57.537659 | controller | 2026-06-10T07:08:57-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:08:57.537717 | controller | 2026-06-10T07:08:57-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:08:57.903130 | controller | 2026-06-10T07:08:57-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:08:58.302450 | controller | 2026-06-10T07:08:58-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:08:58.302533 | controller | 2026-06-10T07:08:58-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:08:59.676544 | controller | 2026-06-10T07:08:59-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:08:59.676631 | controller | 2026-06-10T07:08:59-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:09:01.264307 | controller | 2026-06-10T07:09:01-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:09:01.264407 | controller | 2026-06-10T07:09:01-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:09:02.470969 | controller | 2026-06-10T07:09:02-04:00 INFO 2026-06-10 07:09:02.471058 | controller | - Config: OK 2026-06-10 07:09:02.471068 | controller | - Gateway: OK 2026-06-10 07:09:02.471075 | controller | - GitServer: OK 2026-06-10 07:09:02.471082 | controller | - HoundSearch: OK 2026-06-10 07:09:02.471089 | controller | - LogJuicer: OK 2026-06-10 07:09:02.471096 | controller | - Logserver: OK 2026-06-10 07:09:02.471103 | controller | - MariaDB: OK 2026-06-10 07:09:02.471110 | controller | - NodePoolBuilder: OK 2026-06-10 07:09:02.471117 | controller | - NodePoolLauncher: OK 2026-06-10 07:09:02.471124 | controller | - Zookeeper: OK 2026-06-10 07:09:02.471131 | controller | - Zuul: OK 2026-06-10 07:09:02.471139 | controller | 2026-06-10 07:09:02.471153 | controller | • [12.990 seconds] 2026-06-10 07:09:02.471161 | controller | ------------------------------ 2026-06-10 07:09:02.471200 | controller | gateway tests When removing the gateway's extra config Should serve the regular content 2026-06-10 07:09:02.471209 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:71 2026-06-10 07:09:02.471231 | controller | 2026-06-10T07:09:02-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -o - -I gateway:8080/zuul/api/components] 2026-06-10 07:09:02.576973 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 07:09:02.577079 | controller | Dload Upload Total Spent Left Speed 2026-06-10 07:09:02.583468 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 537 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2026-06-10 07:09:02.586111 | controller | 2026-06-10T07:09:02-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl gateway:8080/zuul/api/components] 2026-06-10 07:09:02.748211 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 07:09:02.748878 | controller | Dload Upload Total Spent Left Speed 2026-06-10 07:09:02.754673 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 537 100 537 0 0 89500 0 --:--:-- --:--:-- --:--:-- 89500 2026-06-10 07:09:02.756733 | controller | • [0.286 seconds] 2026-06-10 07:09:02.756756 | controller | ------------------------------ 2026-06-10 07:09:02.756772 | controller | 2026-06-10 07:09:02.756778 | controller | Ran 16 of 16 Specs in 526.126 seconds 2026-06-10 07:09:02.756782 | controller | SUCCESS! -- 16 Passed | 0 Failed | 0 Pending | 0 Skipped 2026-06-10 07:09:02.756831 | controller | --- PASS: TestSFOP (526.13s) 2026-06-10 07:09:02.756837 | controller | PASS 2026-06-10 07:09:02.761021 | controller | ok github.com/softwarefactory-project/sf-operator/tests 526.169s 2026-06-10 11:09:03.057152 | controller | ok: Runtime: 0:08:59.719715 2026-06-10 11:09:03.099416 | 2026-06-10 11:09:03.099595 | TASK [health-check/test-standalone-update : Create a temporary directory for generated CRs] 2026-06-10 11:09:03.469592 | controller | changed 2026-06-10 11:09:03.479229 | 2026-06-10 11:09:03.479396 | TASK [health-check/test-standalone-update : Generate initial CR] 2026-06-10 11:09:04.428785 | controller | changed 2026-06-10 11:09:04.438109 | 2026-06-10 11:09:04.438238 | TASK [Run standalone operator with initial config] 2026-06-10 11:09:04.475934 | controller | ok 2026-06-10 11:09:04.504974 | 2026-06-10 11:09:04.505059 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:09:04.784029 | controller | ok 2026-06-10 11:09:04.793827 | 2026-06-10 11:09:04.793957 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:09:05.094277 | controller | changed 2026-06-10 11:09:05.103678 | 2026-06-10 11:09:05.103812 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:09:05.748421 | controller | 2026-06-10T07:09:05-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:09:05.748497 | controller | 2026-06-10 07:09:05.757234 | controller | 2026-06-10T07:09:05-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:09:05.757259 | controller | 2026-06-10 07:09:05.770807 | controller | 2026-06-10T07:09:05-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:09:05.772444 | controller | 2026-06-10T07:09:05-04:00 INFO Nothing to clean up. 2026-06-10 07:09:05.778093 | controller | 2026-06-10T07:09:05-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-10 07:09:05.778134 | controller | 2026-06-10T07:09:05-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-10 07:09:05.786811 | controller | 2026-06-10T07:09:05-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 07:09:05.786856 | controller | 2026-06-10T07:09:05-04:00 INFO Updating object, name: git-server 2026-06-10 07:09:06.375087 | controller | 2026-06-10T07:09:06-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-10 07:09:06.375769 | controller | 2026-06-10T07:09:06-04:00 INFO Updating object, name: logserver 2026-06-10 07:09:06.397070 | controller | 2026-06-10T07:09:06-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:09:06.397203 | controller | 2026-06-10T07:09:06-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:09:06.438451 | controller | 2026-06-10T07:09:06-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:09:06.438601 | controller | 2026-06-10T07:09:06-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:09:06.791701 | controller | 2026-06-10T07:09:06-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 07:09:06.791777 | controller | 2026-06-10T07:09:06-04:00 INFO 2026-06-10 07:09:06.791786 | controller | - Config: Waiting ... 2026-06-10 07:09:06.791793 | controller | - Gateway: OK 2026-06-10 07:09:06.791798 | controller | - GitServer: Waiting ... 2026-06-10 07:09:06.791804 | controller | - HoundSearch: OK 2026-06-10 07:09:06.791809 | controller | - LogJuicer: OK 2026-06-10 07:09:06.791815 | controller | - Logserver: Waiting ... 2026-06-10 07:09:06.791820 | controller | - MariaDB: OK 2026-06-10 07:09:06.791827 | controller | - Zookeeper: OK 2026-06-10 07:09:06.791832 | controller | 2026-06-10 07:09:06.791838 | controller | 2026-06-10T07:09:06-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:09:11.800071 | controller | 2026-06-10T07:09:11-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:09:11.801904 | controller | 2026-06-10T07:09:11-04:00 INFO Nothing to clean up. 2026-06-10 07:09:11.825436 | controller | 2026-06-10T07:09:11-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 07:09:29.943057 | controller | 2026-06-10T07:09:29-04:00 INFO Waiting for statefulset, name: logserver 2026-06-10 07:10:00.054996 | controller | 2026-06-10T07:10:00-04:00 INFO Statefulset "logserver" not ready after 30s 2026-06-10 07:10:00.063159 | controller | 2026-06-10T07:10:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:10:00.063254 | controller | 2026-06-10T07:10:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:10:00.083937 | controller | 2026-06-10T07:10:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:10:00.084032 | controller | 2026-06-10T07:10:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:10:00.110605 | controller | 2026-06-10T07:10:00-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:10:00.134335 | controller | 2026-06-10T07:10:00-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:10:00.134410 | controller | 2026-06-10T07:10:00-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:10:00.280302 | controller | 2026-06-10T07:10:00-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:10:00.280396 | controller | 2026-06-10T07:10:00-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:10:01.064200 | controller | 2026-06-10T07:10:01-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:10:01.064291 | controller | 2026-06-10T07:10:01-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:10:02.664028 | controller | 2026-06-10T07:10:02-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:10:02.664109 | controller | 2026-06-10T07:10:02-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:10:03.469104 | controller | 2026-06-10T07:10:03-04:00 INFO 2026-06-10 07:10:03.469194 | controller | - Config: Waiting ... 2026-06-10 07:10:03.469201 | controller | - Gateway: OK 2026-06-10 07:10:03.469205 | controller | - GitServer: OK 2026-06-10 07:10:03.469209 | controller | - HoundSearch: OK 2026-06-10 07:10:03.469213 | controller | - LogJuicer: OK 2026-06-10 07:10:03.469216 | controller | - Logserver: Waiting ... 2026-06-10 07:10:03.469220 | controller | - MariaDB: OK 2026-06-10 07:10:03.469223 | controller | - NodePoolBuilder: OK 2026-06-10 07:10:03.469227 | controller | - NodePoolLauncher: OK 2026-06-10 07:10:03.469230 | controller | - Zookeeper: OK 2026-06-10 07:10:03.469234 | controller | - Zuul: Waiting ... 2026-06-10 07:10:03.469239 | controller | - |_ Config Secrets: OK 2026-06-10 07:10:03.469242 | controller | - |_ Executor: OK 2026-06-10 07:10:03.469246 | controller | - |_ Merger: OK 2026-06-10 07:10:03.469249 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:10:03.469253 | controller | - |_ Web: Waiting ... 2026-06-10 07:10:03.469257 | controller | 2026-06-10 07:10:03.469260 | controller | 2026-06-10T07:10:03-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:10:08.477828 | controller | 2026-06-10T07:10:08-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:10:08.478994 | controller | 2026-06-10T07:10:08-04:00 INFO Nothing to clean up. 2026-06-10 07:10:09.081376 | controller | 2026-06-10T07:10:09-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:10:09.213536 | controller | 2026-06-10T07:10:09-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:10:09.220746 | controller | 2026-06-10T07:10:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:10:09.220795 | controller | 2026-06-10T07:10:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:10:09.238047 | controller | 2026-06-10T07:10:09-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:10:09.238114 | controller | 2026-06-10T07:10:09-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:10:09.502604 | controller | 2026-06-10T07:10:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:10:09.892500 | controller | 2026-06-10T07:10:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:10:09.892582 | controller | 2026-06-10T07:10:09-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:10:10.902618 | controller | 2026-06-10T07:10:10-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:10:41.016792 | controller | 2026-06-10T07:10:41-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:10:41.057463 | controller | 2026-06-10T07:10:41-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:10:41.057539 | controller | 2026-06-10T07:10:41-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:10:41.432652 | controller | 2026-06-10T07:10:41-04:00 INFO 2026-06-10 07:10:41.432745 | controller | - Config: Waiting ... 2026-06-10 07:10:41.432760 | controller | - Gateway: OK 2026-06-10 07:10:41.432772 | controller | - GitServer: OK 2026-06-10 07:10:41.432782 | controller | - HoundSearch: OK 2026-06-10 07:10:41.432795 | controller | - LogJuicer: OK 2026-06-10 07:10:41.432806 | controller | - Logserver: OK 2026-06-10 07:10:41.432818 | controller | - MariaDB: OK 2026-06-10 07:10:41.432829 | controller | - NodePoolBuilder: OK 2026-06-10 07:10:41.432840 | controller | - NodePoolLauncher: OK 2026-06-10 07:10:41.432851 | controller | - Zookeeper: OK 2026-06-10 07:10:41.432861 | controller | - Zuul: Waiting ... 2026-06-10 07:10:41.432872 | controller | - |_ Config Secrets: OK 2026-06-10 07:10:41.432883 | controller | - |_ Executor: OK 2026-06-10 07:10:41.432894 | controller | - |_ Merger: OK 2026-06-10 07:10:41.432906 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:10:41.432918 | controller | - |_ Web: Waiting ... 2026-06-10 07:10:41.432930 | controller | 2026-06-10 07:10:41.432953 | controller | 2026-06-10T07:10:41-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:10:46.441912 | controller | 2026-06-10T07:10:46-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:10:46.444179 | controller | 2026-06-10T07:10:46-04:00 INFO Nothing to clean up. 2026-06-10 07:10:47.054521 | controller | 2026-06-10T07:10:47-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:10:47.054574 | controller | 2026-06-10T07:10:47-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:10:47.074064 | controller | 2026-06-10T07:10:47-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:10:47.074137 | controller | 2026-06-10T07:10:47-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:10:47.461846 | controller | 2026-06-10T07:10:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:10:47.861838 | controller | 2026-06-10T07:10:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:10:47.861888 | controller | 2026-06-10T07:10:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:10:48.862421 | controller | 2026-06-10T07:10:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:11:04.940490 | controller | 2026-06-10T07:11:04-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:11:04.940582 | controller | 2026-06-10T07:11:04-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:11:04.973152 | controller | 2026-06-10T07:11:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:11:04.973284 | controller | 2026-06-10T07:11:04-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:11:05.754983 | controller | 2026-06-10T07:11:05-04:00 INFO 2026-06-10 07:11:05.755038 | controller | - Config: OK 2026-06-10 07:11:05.755050 | controller | - Gateway: OK 2026-06-10 07:11:05.755059 | controller | - GitServer: OK 2026-06-10 07:11:05.755068 | controller | - HoundSearch: OK 2026-06-10 07:11:05.755076 | controller | - LogJuicer: OK 2026-06-10 07:11:05.755085 | controller | - Logserver: OK 2026-06-10 07:11:05.755094 | controller | - MariaDB: OK 2026-06-10 07:11:05.755102 | controller | - NodePoolBuilder: OK 2026-06-10 07:11:05.755111 | controller | - NodePoolLauncher: OK 2026-06-10 07:11:05.755119 | controller | - Zookeeper: OK 2026-06-10 07:11:05.755128 | controller | - Zuul: OK 2026-06-10 07:11:05.755137 | controller | 2026-06-10 07:11:05.755245 | controller | 2026-06-10T07:11:05-04:00 INFO Updating controller configmap ... 2026-06-10 07:11:06.145548 | controller | 2026-06-10T07:11:06-04:00 INFO Standalone reconcile done. 2026-06-10 11:11:06.300800 | controller | ok: Runtime: 0:02:00.751880 2026-06-10 11:11:06.311831 | 2026-06-10 11:11:06.311958 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:11:06.857570 | controller | ok: Runtime: 0:00:00.025286 2026-06-10 11:11:06.874180 | 2026-06-10 11:11:06.874328 | TASK [health-check/test-standalone-update : Get standalone owner configmap after initial creation] 2026-06-10 07:11:07.272473 | controller | fqdn: test-initial.sf.io 2026-06-10 07:11:07.272614 | controller | fluentbitlogforwarding: null 2026-06-10 07:11:07.272631 | controller | storagedefault: 2026-06-10 07:11:07.272640 | controller | classname: "" 2026-06-10 07:11:07.272649 | controller | extraannotations: {} 2026-06-10 07:11:07.272658 | controller | extralabels: {} 2026-06-10 07:11:07.272666 | controller | configrepositorylocation: 2026-06-10 07:11:07.272678 | controller | name: config 2026-06-10 07:11:07.272688 | controller | branch: "" 2026-06-10 07:11:07.272699 | controller | zuulconnectionname: gerrit 2026-06-10 07:11:07.272707 | controller | clusterapiurl: "" 2026-06-10 07:11:07.272717 | controller | logserverhost: "" 2026-06-10 07:11:07.272726 | controller | zuul: 2026-06-10 07:11:07.272734 | controller | oidcauthenticators: [] 2026-06-10 07:11:07.272742 | controller | defaultauthenticator: "" 2026-06-10 07:11:07.272750 | controller | gerritconns: 2026-06-10 07:11:07.272758 | controller | - name: gerrit 2026-06-10 07:11:07.272772 | controller | hostname: gerrit-sshd.sf 2026-06-10 07:11:07.272781 | controller | port: 0 2026-06-10 07:11:07.272793 | controller | puburl: https://gerrit.sfop.me 2026-06-10 07:11:07.272801 | controller | username: zuul 2026-06-10 07:11:07.272809 | controller | canonicalhostname: "" 2026-06-10 07:11:07.272818 | controller | password: "" 2026-06-10 07:11:07.272826 | controller | sshkey: "" 2026-06-10 07:11:07.272834 | controller | gitoverssh: false 2026-06-10 07:11:07.272842 | controller | verifyssl: false 2026-06-10 07:11:07.272850 | controller | streamevents: null 2026-06-10 07:11:07.272858 | controller | githubconns: [] 2026-06-10 07:11:07.272866 | controller | gitlabconns: [] 2026-06-10 07:11:07.272874 | controller | gitconns: [] 2026-06-10 07:11:07.272882 | controller | pagureconns: [] 2026-06-10 07:11:07.272890 | controller | elasticsearchconns: [] 2026-06-10 07:11:07.272898 | controller | smtpconns: [] 2026-06-10 07:11:07.272906 | controller | executor: 2026-06-10 07:11:07.272914 | controller | storage: 2026-06-10 07:11:07.272922 | controller | size: 2026-06-10 07:11:07.272930 | controller | format: "" 2026-06-10 07:11:07.273004 | controller | classname: "" 2026-06-10 07:11:07.273013 | controller | loglevel: DEBUG 2026-06-10 07:11:07.273021 | controller | enabled: true 2026-06-10 07:11:07.273058 | controller | standalone: null 2026-06-10 07:11:07.273102 | controller | limits: null 2026-06-10 07:11:07.273113 | controller | disklimitperjob: 0 2026-06-10 07:11:07.273121 | controller | ansiblesetuptimeout: 0 2026-06-10 07:11:07.273134 | controller | terminationgraceperiodseconds: 0 2026-06-10 07:11:07.273143 | controller | scheduler: 2026-06-10 07:11:07.273151 | controller | storage: 2026-06-10 07:11:07.273159 | controller | size: 2026-06-10 07:11:07.273196 | controller | format: "" 2026-06-10 07:11:07.273204 | controller | classname: "" 2026-06-10 07:11:07.273212 | controller | statsdtarget: "" 2026-06-10 07:11:07.273220 | controller | loglevel: DEBUG 2026-06-10 07:11:07.273228 | controller | limits: null 2026-06-10 07:11:07.273237 | controller | defaultholdexpiration: null 2026-06-10 07:11:07.273245 | controller | maxholdexpiration: null 2026-06-10 07:11:07.273253 | controller | web: 2026-06-10 07:11:07.273261 | controller | loglevel: DEBUG 2026-06-10 07:11:07.273269 | controller | limits: null 2026-06-10 07:11:07.273276 | controller | merger: 2026-06-10 07:11:07.273284 | controller | gitusername: "" 2026-06-10 07:11:07.273292 | controller | gituseremail: "" 2026-06-10 07:11:07.273300 | controller | githttplowspeedlimit: 0 2026-06-10 07:11:07.273308 | controller | githttplowspeedtime: 0 2026-06-10 07:11:07.273316 | controller | gittimeout: 0 2026-06-10 07:11:07.273324 | controller | storage: 2026-06-10 07:11:07.273332 | controller | size: 2026-06-10 07:11:07.273339 | controller | format: "" 2026-06-10 07:11:07.273347 | controller | classname: "" 2026-06-10 07:11:07.273355 | controller | loglevel: DEBUG 2026-06-10 07:11:07.273363 | controller | limits: null 2026-06-10 07:11:07.273371 | controller | nodepool: 2026-06-10 07:11:07.273379 | controller | launcher: 2026-06-10 07:11:07.273386 | controller | loglevel: "" 2026-06-10 07:11:07.273394 | controller | limits: null 2026-06-10 07:11:07.273402 | controller | builder: 2026-06-10 07:11:07.273410 | controller | storage: 2026-06-10 07:11:07.273418 | controller | size: 2026-06-10 07:11:07.273426 | controller | format: "" 2026-06-10 07:11:07.273434 | controller | classname: "" 2026-06-10 07:11:07.273442 | controller | loglevel: "" 2026-06-10 07:11:07.273461 | controller | limits: null 2026-06-10 07:11:07.273469 | controller | statsdtarget: "" 2026-06-10 07:11:07.273478 | controller | zookeeper: 2026-06-10 07:11:07.273486 | controller | storage: 2026-06-10 07:11:07.273494 | controller | size: 2026-06-10 07:11:07.273502 | controller | format: "" 2026-06-10 07:11:07.273509 | controller | classname: "" 2026-06-10 07:11:07.273517 | controller | limits: null 2026-06-10 07:11:07.273525 | controller | nodeantiaffinityenabled: null 2026-06-10 07:11:07.273533 | controller | logserver: 2026-06-10 07:11:07.273541 | controller | retentiondays: 30 2026-06-10 07:11:07.273549 | controller | loopdelay: 3600 2026-06-10 07:11:07.273557 | controller | storage: 2026-06-10 07:11:07.273565 | controller | size: 2026-06-10 07:11:07.273573 | controller | format: BinarySI 2026-06-10 07:11:07.273581 | controller | classname: "" 2026-06-10 07:11:07.273589 | controller | podannotations: {} 2026-06-10 07:11:07.273596 | controller | logjuicer: 2026-06-10 07:11:07.273604 | controller | size: 2026-06-10 07:11:07.273612 | controller | format: "" 2026-06-10 07:11:07.273620 | controller | classname: "" 2026-06-10 07:11:07.273628 | controller | mariadb: 2026-06-10 07:11:07.273636 | controller | dbstorage: 2026-06-10 07:11:07.273644 | controller | size: 2026-06-10 07:11:07.273652 | controller | format: "" 2026-06-10 07:11:07.273660 | controller | classname: "" 2026-06-10 07:11:07.273667 | controller | logstorage: 2026-06-10 07:11:07.273675 | controller | size: 2026-06-10 07:11:07.273683 | controller | format: "" 2026-06-10 07:11:07.273691 | controller | classname: "" 2026-06-10 07:11:07.273699 | controller | limits: null 2026-06-10 07:11:07.273707 | controller | gitserver: 2026-06-10 07:11:07.273715 | controller | storage: 2026-06-10 07:11:07.273722 | controller | size: 2026-06-10 07:11:07.273730 | controller | format: "" 2026-06-10 07:11:07.273738 | controller | classname: "" 2026-06-10 07:11:07.273746 | controller | codesearch: 2026-06-10 07:11:07.273754 | controller | storage: 2026-06-10 07:11:07.273762 | controller | size: 2026-06-10 07:11:07.273769 | controller | format: "" 2026-06-10 07:11:07.273777 | controller | classname: "" 2026-06-10 07:11:07.273785 | controller | limits: null 2026-06-10 07:11:07.273793 | controller | enabled: null 2026-06-10 07:11:07.273800 | controller | hostaliases: [] 2026-06-10 07:11:07.273808 | controller | gateway: null 2026-06-10 11:11:07.423319 | controller | ok: Runtime: 0:00:00.096073 2026-06-10 11:11:07.434226 | 2026-06-10 11:11:07.434390 | TASK [health-check/test-standalone-update : Verify initial spec in configmap] 2026-06-10 11:11:07.479185 | controller | ok: All assertions passed 2026-06-10 11:11:07.487429 | 2026-06-10 11:11:07.487558 | TASK [health-check/test-standalone-update : Get standalone owner configmap annotation after initial creation] 2026-06-10 11:11:08.030733 | controller | ok: Runtime: 0:00:00.096283 2026-06-10 11:11:08.037677 | 2026-06-10 11:11:08.037764 | TASK [health-check/test-standalone-update : Pause for 2 seconds to ensure timestamp difference] 2026-06-10 11:11:10.069532 | controller | ok 2026-06-10 11:11:10.078098 | 2026-06-10 11:11:10.078228 | TASK [health-check/test-standalone-update : Generate updated CR] 2026-06-10 11:11:10.863284 | controller | changed 2026-06-10 11:11:10.872158 | 2026-06-10 11:11:10.872282 | TASK [Run standalone operator with updated config] 2026-06-10 11:11:10.898035 | controller | ok 2026-06-10 11:11:10.932066 | 2026-06-10 11:11:10.932158 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:11:11.213423 | controller | ok 2026-06-10 11:11:11.219061 | 2026-06-10 11:11:11.219141 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:11:11.441404 | controller | changed 2026-06-10 11:11:11.446903 | 2026-06-10 11:11:11.446982 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:11:12.003934 | controller | 2026-06-10T07:11:12-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:11:12.003999 | controller | 2026-06-10 07:11:12.016033 | controller | 2026-06-10T07:11:12-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:11:12.016063 | controller | 2026-06-10 07:11:12.027987 | controller | 2026-06-10T07:11:12-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:11:12.029207 | controller | 2026-06-10T07:11:12-04:00 INFO Nothing to clean up. 2026-06-10 07:11:12.036319 | controller | 2026-06-10T07:11:12-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-10 07:11:12.036343 | controller | 2026-06-10T07:11:12-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-10 07:11:12.046497 | controller | 2026-06-10T07:11:12-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 07:11:12.046536 | controller | 2026-06-10T07:11:12-04:00 INFO Updating object, name: git-server 2026-06-10 07:11:12.631435 | controller | 2026-06-10T07:11:12-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-10 07:11:12.631498 | controller | 2026-06-10T07:11:12-04:00 INFO Updating object, name: logserver 2026-06-10 07:11:12.645851 | controller | 2026-06-10T07:11:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:11:12.645947 | controller | 2026-06-10T07:11:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:11:12.668840 | controller | 2026-06-10T07:11:12-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:11:12.668920 | controller | 2026-06-10T07:11:12-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:11:13.049395 | controller | 2026-06-10T07:11:13-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 07:11:13.049462 | controller | 2026-06-10T07:11:13-04:00 INFO 2026-06-10 07:11:13.049473 | controller | - Config: Waiting ... 2026-06-10 07:11:13.049481 | controller | - Gateway: OK 2026-06-10 07:11:13.049489 | controller | - GitServer: Waiting ... 2026-06-10 07:11:13.049497 | controller | - HoundSearch: OK 2026-06-10 07:11:13.049504 | controller | - LogJuicer: OK 2026-06-10 07:11:13.049512 | controller | - Logserver: Waiting ... 2026-06-10 07:11:13.049519 | controller | - MariaDB: OK 2026-06-10 07:11:13.049526 | controller | - Zookeeper: OK 2026-06-10 07:11:13.049533 | controller | 2026-06-10 07:11:13.049540 | controller | 2026-06-10T07:11:13-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:11:18.058234 | controller | 2026-06-10T07:11:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:11:18.059307 | controller | 2026-06-10T07:11:18-04:00 INFO Nothing to clean up. 2026-06-10 07:11:18.075742 | controller | 2026-06-10T07:11:18-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 07:11:36.202202 | controller | 2026-06-10T07:11:36-04:00 INFO Waiting for statefulset, name: logserver 2026-06-10 07:12:06.311987 | controller | 2026-06-10T07:12:06-04:00 INFO Statefulset "logserver" not ready after 30s 2026-06-10 07:12:06.319254 | controller | 2026-06-10T07:12:06-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:12:06.319325 | controller | 2026-06-10T07:12:06-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:12:06.341231 | controller | 2026-06-10T07:12:06-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:12:06.341301 | controller | 2026-06-10T07:12:06-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:12:06.372782 | controller | 2026-06-10T07:12:06-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:12:06.399040 | controller | 2026-06-10T07:12:06-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:12:06.399105 | controller | 2026-06-10T07:12:06-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:12:06.540041 | controller | 2026-06-10T07:12:06-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:12:06.540132 | controller | 2026-06-10T07:12:06-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:12:07.321721 | controller | 2026-06-10T07:12:07-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:12:07.321792 | controller | 2026-06-10T07:12:07-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:12:08.921576 | controller | 2026-06-10T07:12:08-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:12:08.921674 | controller | 2026-06-10T07:12:08-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:12:09.730884 | controller | 2026-06-10T07:12:09-04:00 INFO 2026-06-10 07:12:09.730949 | controller | - Config: Waiting ... 2026-06-10 07:12:09.730957 | controller | - Gateway: OK 2026-06-10 07:12:09.730963 | controller | - GitServer: OK 2026-06-10 07:12:09.730968 | controller | - HoundSearch: OK 2026-06-10 07:12:09.730974 | controller | - LogJuicer: OK 2026-06-10 07:12:09.730979 | controller | - Logserver: Waiting ... 2026-06-10 07:12:09.730985 | controller | - MariaDB: OK 2026-06-10 07:12:09.730990 | controller | - NodePoolBuilder: OK 2026-06-10 07:12:09.730995 | controller | - NodePoolLauncher: OK 2026-06-10 07:12:09.731001 | controller | - Zookeeper: OK 2026-06-10 07:12:09.731006 | controller | - Zuul: Waiting ... 2026-06-10 07:12:09.731013 | controller | - |_ Config Secrets: OK 2026-06-10 07:12:09.731019 | controller | - |_ Executor: OK 2026-06-10 07:12:09.731024 | controller | - |_ Merger: OK 2026-06-10 07:12:09.731030 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:12:09.731035 | controller | - |_ Web: Waiting ... 2026-06-10 07:12:09.731041 | controller | 2026-06-10 07:12:09.731046 | controller | 2026-06-10T07:12:09-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:12:14.740971 | controller | 2026-06-10T07:12:14-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:12:14.743664 | controller | 2026-06-10T07:12:14-04:00 INFO Nothing to clean up. 2026-06-10 07:12:15.351983 | controller | 2026-06-10T07:12:15-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:12:15.453977 | controller | 2026-06-10T07:12:15-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:12:15.462705 | controller | 2026-06-10T07:12:15-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:12:15.462764 | controller | 2026-06-10T07:12:15-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:12:15.478200 | controller | 2026-06-10T07:12:15-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:12:15.478262 | controller | 2026-06-10T07:12:15-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:12:15.774817 | controller | 2026-06-10T07:12:15-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:12:16.166912 | controller | 2026-06-10T07:12:16-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:12:16.166970 | controller | 2026-06-10T07:12:16-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:12:17.153863 | controller | 2026-06-10T07:12:17-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:12:47.262822 | controller | 2026-06-10T07:12:47-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:12:47.300438 | controller | 2026-06-10T07:12:47-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:12:47.300485 | controller | 2026-06-10T07:12:47-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:12:47.679050 | controller | 2026-06-10T07:12:47-04:00 INFO 2026-06-10 07:12:47.679104 | controller | - Config: Waiting ... 2026-06-10 07:12:47.679117 | controller | - Gateway: OK 2026-06-10 07:12:47.679126 | controller | - GitServer: OK 2026-06-10 07:12:47.679134 | controller | - HoundSearch: OK 2026-06-10 07:12:47.679144 | controller | - LogJuicer: OK 2026-06-10 07:12:47.679152 | controller | - Logserver: OK 2026-06-10 07:12:47.679160 | controller | - MariaDB: OK 2026-06-10 07:12:47.679204 | controller | - NodePoolBuilder: OK 2026-06-10 07:12:47.679213 | controller | - NodePoolLauncher: OK 2026-06-10 07:12:47.679223 | controller | - Zookeeper: OK 2026-06-10 07:12:47.679234 | controller | - Zuul: Waiting ... 2026-06-10 07:12:47.679246 | controller | - |_ Config Secrets: OK 2026-06-10 07:12:47.679258 | controller | - |_ Executor: OK 2026-06-10 07:12:47.679270 | controller | - |_ Merger: OK 2026-06-10 07:12:47.679282 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:12:47.679293 | controller | - |_ Web: Waiting ... 2026-06-10 07:12:47.679304 | controller | 2026-06-10 07:12:47.679316 | controller | 2026-06-10T07:12:47-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:12:52.683856 | controller | 2026-06-10T07:12:52-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:12:52.685297 | controller | 2026-06-10T07:12:52-04:00 INFO Nothing to clean up. 2026-06-10 07:12:53.296935 | controller | 2026-06-10T07:12:53-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:12:53.297000 | controller | 2026-06-10T07:12:53-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:12:53.311434 | controller | 2026-06-10T07:12:53-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:12:53.311519 | controller | 2026-06-10T07:12:53-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:12:53.698609 | controller | 2026-06-10T07:12:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:12:54.094322 | controller | 2026-06-10T07:12:54-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:12:54.094378 | controller | 2026-06-10T07:12:54-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:12:55.116353 | controller | 2026-06-10T07:12:55-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:13:13.196683 | controller | 2026-06-10T07:13:13-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:13:13.196777 | controller | 2026-06-10T07:13:13-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:13:13.233495 | controller | 2026-06-10T07:13:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:13:13.233573 | controller | 2026-06-10T07:13:13-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:13:14.016142 | controller | 2026-06-10T07:13:14-04:00 INFO 2026-06-10 07:13:14.016204 | controller | - Config: OK 2026-06-10 07:13:14.016209 | controller | - Gateway: OK 2026-06-10 07:13:14.016213 | controller | - GitServer: OK 2026-06-10 07:13:14.016217 | controller | - HoundSearch: OK 2026-06-10 07:13:14.016221 | controller | - LogJuicer: OK 2026-06-10 07:13:14.016224 | controller | - Logserver: OK 2026-06-10 07:13:14.016228 | controller | - MariaDB: OK 2026-06-10 07:13:14.016232 | controller | - NodePoolBuilder: OK 2026-06-10 07:13:14.016236 | controller | - NodePoolLauncher: OK 2026-06-10 07:13:14.016240 | controller | - Zookeeper: OK 2026-06-10 07:13:14.016243 | controller | - Zuul: OK 2026-06-10 07:13:14.016247 | controller | 2026-06-10 07:13:14.016252 | controller | 2026-06-10T07:13:14-04:00 INFO Updating controller configmap ... 2026-06-10 07:13:14.401351 | controller | 2026-06-10T07:13:14-04:00 INFO Standalone reconcile done. 2026-06-10 11:13:14.723521 | controller | ok: Runtime: 0:02:02.682894 2026-06-10 11:13:14.729708 | 2026-06-10 11:13:14.729793 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:13:15.261377 | controller | ok: Runtime: 0:00:00.008589 2026-06-10 11:13:15.277289 | 2026-06-10 11:13:15.277454 | TASK [health-check/test-standalone-update : Get standalone owner configmap after update] 2026-06-10 07:13:15.652302 | controller | fqdn: test-updated.sf.io 2026-06-10 07:13:15.652390 | controller | fluentbitlogforwarding: null 2026-06-10 07:13:15.652400 | controller | storagedefault: 2026-06-10 07:13:15.652406 | controller | classname: "" 2026-06-10 07:13:15.652411 | controller | extraannotations: {} 2026-06-10 07:13:15.652416 | controller | extralabels: {} 2026-06-10 07:13:15.652422 | controller | configrepositorylocation: 2026-06-10 07:13:15.652429 | controller | name: config 2026-06-10 07:13:15.652435 | controller | branch: "" 2026-06-10 07:13:15.652442 | controller | zuulconnectionname: gerrit 2026-06-10 07:13:15.652447 | controller | clusterapiurl: "" 2026-06-10 07:13:15.652453 | controller | logserverhost: "" 2026-06-10 07:13:15.652458 | controller | zuul: 2026-06-10 07:13:15.652463 | controller | oidcauthenticators: [] 2026-06-10 07:13:15.652468 | controller | defaultauthenticator: "" 2026-06-10 07:13:15.652472 | controller | gerritconns: 2026-06-10 07:13:15.652477 | controller | - name: gerrit 2026-06-10 07:13:15.652486 | controller | hostname: gerrit-sshd.sf 2026-06-10 07:13:15.652491 | controller | port: 0 2026-06-10 07:13:15.652499 | controller | puburl: https://gerrit.sfop.me 2026-06-10 07:13:15.652504 | controller | username: zuul 2026-06-10 07:13:15.652508 | controller | canonicalhostname: "" 2026-06-10 07:13:15.652514 | controller | password: "" 2026-06-10 07:13:15.652519 | controller | sshkey: "" 2026-06-10 07:13:15.652523 | controller | gitoverssh: false 2026-06-10 07:13:15.652528 | controller | verifyssl: false 2026-06-10 07:13:15.652532 | controller | streamevents: null 2026-06-10 07:13:15.652537 | controller | githubconns: [] 2026-06-10 07:13:15.652542 | controller | gitlabconns: [] 2026-06-10 07:13:15.652546 | controller | gitconns: [] 2026-06-10 07:13:15.652551 | controller | pagureconns: [] 2026-06-10 07:13:15.652555 | controller | elasticsearchconns: [] 2026-06-10 07:13:15.652560 | controller | smtpconns: [] 2026-06-10 07:13:15.652565 | controller | executor: 2026-06-10 07:13:15.652569 | controller | storage: 2026-06-10 07:13:15.652574 | controller | size: 2026-06-10 07:13:15.652578 | controller | format: "" 2026-06-10 07:13:15.652583 | controller | classname: "" 2026-06-10 07:13:15.652588 | controller | loglevel: DEBUG 2026-06-10 07:13:15.652592 | controller | enabled: true 2026-06-10 07:13:15.652597 | controller | standalone: null 2026-06-10 07:13:15.652601 | controller | limits: null 2026-06-10 07:13:15.652606 | controller | disklimitperjob: 0 2026-06-10 07:13:15.652639 | controller | ansiblesetuptimeout: 0 2026-06-10 07:13:15.652648 | controller | terminationgraceperiodseconds: 0 2026-06-10 07:13:15.652652 | controller | scheduler: 2026-06-10 07:13:15.652657 | controller | storage: 2026-06-10 07:13:15.652662 | controller | size: 2026-06-10 07:13:15.652666 | controller | format: "" 2026-06-10 07:13:15.652671 | controller | classname: "" 2026-06-10 07:13:15.652676 | controller | statsdtarget: "" 2026-06-10 07:13:15.652680 | controller | loglevel: DEBUG 2026-06-10 07:13:15.652685 | controller | limits: null 2026-06-10 07:13:15.652690 | controller | defaultholdexpiration: null 2026-06-10 07:13:15.652694 | controller | maxholdexpiration: null 2026-06-10 07:13:15.652699 | controller | web: 2026-06-10 07:13:15.652704 | controller | loglevel: DEBUG 2026-06-10 07:13:15.652708 | controller | limits: null 2026-06-10 07:13:15.652713 | controller | merger: 2026-06-10 07:13:15.652718 | controller | gitusername: "" 2026-06-10 07:13:15.652722 | controller | gituseremail: "" 2026-06-10 07:13:15.652727 | controller | githttplowspeedlimit: 0 2026-06-10 07:13:15.652732 | controller | githttplowspeedtime: 0 2026-06-10 07:13:15.652736 | controller | gittimeout: 0 2026-06-10 07:13:15.652741 | controller | storage: 2026-06-10 07:13:15.652746 | controller | size: 2026-06-10 07:13:15.652751 | controller | format: "" 2026-06-10 07:13:15.652756 | controller | classname: "" 2026-06-10 07:13:15.652760 | controller | loglevel: DEBUG 2026-06-10 07:13:15.652765 | controller | limits: null 2026-06-10 07:13:15.652770 | controller | nodepool: 2026-06-10 07:13:15.652774 | controller | launcher: 2026-06-10 07:13:15.652779 | controller | loglevel: "" 2026-06-10 07:13:15.652784 | controller | limits: null 2026-06-10 07:13:15.652788 | controller | builder: 2026-06-10 07:13:15.652793 | controller | storage: 2026-06-10 07:13:15.652798 | controller | size: 2026-06-10 07:13:15.652802 | controller | format: "" 2026-06-10 07:13:15.652807 | controller | classname: "" 2026-06-10 07:13:15.652812 | controller | loglevel: "" 2026-06-10 07:13:15.652823 | controller | limits: null 2026-06-10 07:13:15.652828 | controller | statsdtarget: "" 2026-06-10 07:13:15.652832 | controller | zookeeper: 2026-06-10 07:13:15.652837 | controller | storage: 2026-06-10 07:13:15.652842 | controller | size: 2026-06-10 07:13:15.652847 | controller | format: "" 2026-06-10 07:13:15.652851 | controller | classname: "" 2026-06-10 07:13:15.652856 | controller | limits: null 2026-06-10 07:13:15.652861 | controller | nodeantiaffinityenabled: null 2026-06-10 07:13:15.652865 | controller | logserver: 2026-06-10 07:13:15.652870 | controller | retentiondays: 30 2026-06-10 07:13:15.652875 | controller | loopdelay: 3600 2026-06-10 07:13:15.652879 | controller | storage: 2026-06-10 07:13:15.652884 | controller | size: 2026-06-10 07:13:15.652889 | controller | format: BinarySI 2026-06-10 07:13:15.652893 | controller | classname: "" 2026-06-10 07:13:15.652898 | controller | podannotations: {} 2026-06-10 07:13:15.652902 | controller | logjuicer: 2026-06-10 07:13:15.652907 | controller | size: 2026-06-10 07:13:15.652912 | controller | format: "" 2026-06-10 07:13:15.652916 | controller | classname: "" 2026-06-10 07:13:15.652921 | controller | mariadb: 2026-06-10 07:13:15.652926 | controller | dbstorage: 2026-06-10 07:13:15.652930 | controller | size: 2026-06-10 07:13:15.652935 | controller | format: "" 2026-06-10 07:13:15.652940 | controller | classname: "" 2026-06-10 07:13:15.652944 | controller | logstorage: 2026-06-10 07:13:15.652949 | controller | size: 2026-06-10 07:13:15.652954 | controller | format: "" 2026-06-10 07:13:15.652958 | controller | classname: "" 2026-06-10 07:13:15.652963 | controller | limits: null 2026-06-10 07:13:15.652967 | controller | gitserver: 2026-06-10 07:13:15.652972 | controller | storage: 2026-06-10 07:13:15.652977 | controller | size: 2026-06-10 07:13:15.652981 | controller | format: "" 2026-06-10 07:13:15.652986 | controller | classname: "" 2026-06-10 07:13:15.652991 | controller | codesearch: 2026-06-10 07:13:15.652995 | controller | storage: 2026-06-10 07:13:15.653000 | controller | size: 2026-06-10 07:13:15.653005 | controller | format: "" 2026-06-10 07:13:15.653009 | controller | classname: "" 2026-06-10 07:13:15.653014 | controller | limits: null 2026-06-10 07:13:15.653019 | controller | enabled: null 2026-06-10 07:13:15.653024 | controller | hostaliases: [] 2026-06-10 07:13:15.653028 | controller | gateway: null 2026-06-10 11:13:15.824755 | controller | ok: Runtime: 0:00:00.096432 2026-06-10 11:13:15.835045 | 2026-06-10 11:13:15.835181 | TASK [health-check/test-standalone-update : Verify updated spec in configmap] 2026-06-10 11:13:15.887205 | controller | ok: All assertions passed 2026-06-10 11:13:15.901408 | 2026-06-10 11:13:15.901498 | TASK [health-check/test-standalone-update : Get standalone owner configmap annotation after update] 2026-06-10 11:13:16.436128 | controller | ok: Runtime: 0:00:00.075975 2026-06-10 11:13:16.445872 | 2026-06-10 11:13:16.446013 | TASK [health-check/test-standalone-update : Verify that the last-reconcile annotation has changed] 2026-06-10 11:13:16.491906 | controller | ok: All assertions passed 2026-06-10 11:13:16.502777 | 2026-06-10 11:13:16.502920 | TASK [Restore original state] 2026-06-10 11:13:16.549477 | controller | ok 2026-06-10 11:13:16.581219 | 2026-06-10 11:13:16.581281 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:13:16.827183 | controller | ok 2026-06-10 11:13:16.836313 | 2026-06-10 11:13:16.836450 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:13:17.065425 | controller | changed 2026-06-10 11:13:17.074072 | 2026-06-10 11:13:17.074203 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:13:17.698335 | controller | 2026-06-10T07:13:17-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:13:17.698414 | controller | 2026-06-10 07:13:17.709937 | controller | 2026-06-10T07:13:17-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:13:17.709974 | controller | 2026-06-10 07:13:17.720785 | controller | 2026-06-10T07:13:17-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:13:17.722242 | controller | 2026-06-10T07:13:17-04:00 INFO Nothing to clean up. 2026-06-10 07:13:17.728050 | controller | 2026-06-10T07:13:17-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-10 07:13:17.728192 | controller | 2026-06-10T07:13:17-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-10 07:13:17.737215 | controller | 2026-06-10T07:13:17-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 07:13:17.737261 | controller | 2026-06-10T07:13:17-04:00 INFO Updating object, name: git-server 2026-06-10 07:13:18.324499 | controller | 2026-06-10T07:13:18-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-10 07:13:18.324564 | controller | 2026-06-10T07:13:18-04:00 INFO Updating object, name: logserver 2026-06-10 07:13:18.342596 | controller | 2026-06-10T07:13:18-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:13:18.342688 | controller | 2026-06-10T07:13:18-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:13:18.371907 | controller | 2026-06-10T07:13:18-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:13:18.371980 | controller | 2026-06-10T07:13:18-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:13:18.749082 | controller | 2026-06-10T07:13:18-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 07:13:18.749147 | controller | 2026-06-10T07:13:18-04:00 INFO 2026-06-10 07:13:18.749154 | controller | - Config: Waiting ... 2026-06-10 07:13:18.749160 | controller | - Gateway: OK 2026-06-10 07:13:18.749189 | controller | - GitServer: Waiting ... 2026-06-10 07:13:18.749195 | controller | - HoundSearch: OK 2026-06-10 07:13:18.749199 | controller | - LogJuicer: OK 2026-06-10 07:13:18.749205 | controller | - Logserver: Waiting ... 2026-06-10 07:13:18.749209 | controller | - MariaDB: OK 2026-06-10 07:13:18.749214 | controller | - Zookeeper: OK 2026-06-10 07:13:18.749219 | controller | 2026-06-10 07:13:18.749224 | controller | 2026-06-10T07:13:18-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:13:23.757242 | controller | 2026-06-10T07:13:23-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:13:23.758916 | controller | 2026-06-10T07:13:23-04:00 INFO Nothing to clean up. 2026-06-10 07:13:23.777717 | controller | 2026-06-10T07:13:23-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 07:13:39.875078 | controller | 2026-06-10T07:13:39-04:00 INFO Waiting for statefulset, name: logserver 2026-06-10 07:14:09.988510 | controller | 2026-06-10T07:14:09-04:00 INFO Statefulset "logserver" not ready after 30s 2026-06-10 07:14:09.997064 | controller | 2026-06-10T07:14:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:14:09.997133 | controller | 2026-06-10T07:14:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:14:10.020117 | controller | 2026-06-10T07:14:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:14:10.020248 | controller | 2026-06-10T07:14:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:14:10.051602 | controller | 2026-06-10T07:14:10-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:14:10.077097 | controller | 2026-06-10T07:14:10-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:14:10.077233 | controller | 2026-06-10T07:14:10-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:14:10.219456 | controller | 2026-06-10T07:14:10-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:14:10.219542 | controller | 2026-06-10T07:14:10-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:14:10.998807 | controller | 2026-06-10T07:14:10-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:14:10.998887 | controller | 2026-06-10T07:14:10-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:14:12.598988 | controller | 2026-06-10T07:14:12-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:14:12.599062 | controller | 2026-06-10T07:14:12-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:14:13.402546 | controller | 2026-06-10T07:14:13-04:00 INFO 2026-06-10 07:14:13.402610 | controller | - Config: Waiting ... 2026-06-10 07:14:13.402617 | controller | - Gateway: OK 2026-06-10 07:14:13.402624 | controller | - GitServer: OK 2026-06-10 07:14:13.402629 | controller | - HoundSearch: OK 2026-06-10 07:14:13.402635 | controller | - LogJuicer: OK 2026-06-10 07:14:13.402640 | controller | - Logserver: Waiting ... 2026-06-10 07:14:13.402645 | controller | - MariaDB: OK 2026-06-10 07:14:13.402650 | controller | - NodePoolBuilder: OK 2026-06-10 07:14:13.402656 | controller | - NodePoolLauncher: OK 2026-06-10 07:14:13.402662 | controller | - Zookeeper: OK 2026-06-10 07:14:13.402669 | controller | - Zuul: Waiting ... 2026-06-10 07:14:13.402678 | controller | - |_ Config Secrets: OK 2026-06-10 07:14:13.402685 | controller | - |_ Executor: OK 2026-06-10 07:14:13.402693 | controller | - |_ Merger: OK 2026-06-10 07:14:13.402700 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:14:13.402707 | controller | - |_ Web: Waiting ... 2026-06-10 07:14:13.402715 | controller | 2026-06-10 07:14:13.402730 | controller | 2026-06-10T07:14:13-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:14:18.412610 | controller | 2026-06-10T07:14:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:14:18.415914 | controller | 2026-06-10T07:14:18-04:00 INFO Nothing to clean up. 2026-06-10 07:14:19.022332 | controller | 2026-06-10T07:14:19-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:14:19.134852 | controller | 2026-06-10T07:14:19-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:14:19.140885 | controller | 2026-06-10T07:14:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:14:19.140914 | controller | 2026-06-10T07:14:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:14:19.164035 | controller | 2026-06-10T07:14:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:14:19.164114 | controller | 2026-06-10T07:14:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:14:19.431537 | controller | 2026-06-10T07:14:19-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:14:19.827377 | controller | 2026-06-10T07:14:19-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:14:19.827441 | controller | 2026-06-10T07:14:19-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:14:20.834856 | controller | 2026-06-10T07:14:20-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:14:50.954924 | controller | 2026-06-10T07:14:50-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:14:50.989001 | controller | 2026-06-10T07:14:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:14:50.989145 | controller | 2026-06-10T07:14:50-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:14:51.366688 | controller | 2026-06-10T07:14:51-04:00 INFO 2026-06-10 07:14:51.366802 | controller | - Config: Waiting ... 2026-06-10 07:14:51.366808 | controller | - Gateway: OK 2026-06-10 07:14:51.366813 | controller | - GitServer: OK 2026-06-10 07:14:51.366817 | controller | - HoundSearch: OK 2026-06-10 07:14:51.366823 | controller | - LogJuicer: OK 2026-06-10 07:14:51.366827 | controller | - Logserver: OK 2026-06-10 07:14:51.366830 | controller | - MariaDB: OK 2026-06-10 07:14:51.366834 | controller | - NodePoolBuilder: OK 2026-06-10 07:14:51.366838 | controller | - NodePoolLauncher: OK 2026-06-10 07:14:51.366841 | controller | - Zookeeper: OK 2026-06-10 07:14:51.366845 | controller | - Zuul: Waiting ... 2026-06-10 07:14:51.366849 | controller | - |_ Config Secrets: OK 2026-06-10 07:14:51.366853 | controller | - |_ Executor: OK 2026-06-10 07:14:51.366856 | controller | - |_ Merger: OK 2026-06-10 07:14:51.366860 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:14:51.366865 | controller | - |_ Web: Waiting ... 2026-06-10 07:14:51.366870 | controller | 2026-06-10 07:14:51.366874 | controller | 2026-06-10T07:14:51-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:14:56.370233 | controller | 2026-06-10T07:14:56-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:14:56.371395 | controller | 2026-06-10T07:14:56-04:00 INFO Nothing to clean up. 2026-06-10 07:14:56.990691 | controller | 2026-06-10T07:14:56-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:14:56.990790 | controller | 2026-06-10T07:14:56-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:14:57.012273 | controller | 2026-06-10T07:14:57-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:14:57.012348 | controller | 2026-06-10T07:14:57-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:14:57.387103 | controller | 2026-06-10T07:14:57-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:14:57.785082 | controller | 2026-06-10T07:14:57-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:14:57.785149 | controller | 2026-06-10T07:14:57-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:14:58.801660 | controller | 2026-06-10T07:14:58-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:15:14.869129 | controller | 2026-06-10T07:15:14-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:15:14.869238 | controller | 2026-06-10T07:15:14-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:15:14.904404 | controller | 2026-06-10T07:15:14-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:15:14.904461 | controller | 2026-06-10T07:15:14-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:15:15.691404 | controller | 2026-06-10T07:15:15-04:00 INFO 2026-06-10 07:15:15.691474 | controller | - Config: OK 2026-06-10 07:15:15.691485 | controller | - Gateway: OK 2026-06-10 07:15:15.691494 | controller | - GitServer: OK 2026-06-10 07:15:15.691502 | controller | - HoundSearch: OK 2026-06-10 07:15:15.691510 | controller | - LogJuicer: OK 2026-06-10 07:15:15.691518 | controller | - Logserver: OK 2026-06-10 07:15:15.691527 | controller | - MariaDB: OK 2026-06-10 07:15:15.691535 | controller | - NodePoolBuilder: OK 2026-06-10 07:15:15.691544 | controller | - NodePoolLauncher: OK 2026-06-10 07:15:15.691552 | controller | - Zookeeper: OK 2026-06-10 07:15:15.691561 | controller | - Zuul: OK 2026-06-10 07:15:15.691569 | controller | 2026-06-10 07:15:15.691578 | controller | 2026-06-10T07:15:15-04:00 INFO Updating controller configmap ... 2026-06-10 07:15:16.075825 | controller | 2026-06-10T07:15:16-04:00 INFO Standalone reconcile done. 2026-06-10 11:15:16.262421 | controller | ok: Runtime: 0:01:58.690495 2026-06-10 11:15:16.272861 | 2026-06-10 11:15:16.272997 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:15:16.822222 | controller | ok: Runtime: 0:00:00.005578 2026-06-10 11:15:16.837030 | 2026-06-10 11:15:16.837176 | TASK [health-check/test-standalone-update : Remove temporary directory] 2026-06-10 11:15:17.106296 | controller | changed 2026-06-10 11:15:17.122848 | 2026-06-10 11:15:17.122981 | TASK [health-check/check-zuul-weeder : Get zuul-weeder status] 2026-06-10 11:15:17.938295 | controller | ok: OK (unknown bytes) 2026-06-10 11:15:17.989274 | 2026-06-10 11:15:17.989388 | TASK [scale down zuul components] 2026-06-10 11:15:18.009511 | controller | ok 2026-06-10 11:15:18.026282 | 2026-06-10 11:15:18.026394 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 11:15:18.051540 | controller | ok: 2026-06-10 11:15:18.051812 | controller | { 2026-06-10 11:15:18.051863 | controller | "kind": "sts", 2026-06-10 11:15:18.051905 | controller | "name": "zuul-executor", 2026-06-10 11:15:18.051942 | controller | "replicas": 0 2026-06-10 11:15:18.051978 | controller | } 2026-06-10 11:15:18.052035 | controller | changed: All items complete 2026-06-10 11:15:18.052073 | 2026-06-10 11:15:18.058926 | controller | ok: 2026-06-10 11:15:18.059016 | controller | { 2026-06-10 11:15:18.059061 | controller | "kind": "sts", 2026-06-10 11:15:18.059117 | controller | "name": "zuul-merger", 2026-06-10 11:15:18.059153 | controller | "replicas": 0 2026-06-10 11:15:18.059198 | controller | } 2026-06-10 11:15:18.065708 | controller | ok: 2026-06-10 11:15:18.065797 | controller | { 2026-06-10 11:15:18.065842 | controller | "kind": "sts", 2026-06-10 11:15:18.065878 | controller | "name": "zuul-scheduler", 2026-06-10 11:15:18.065914 | controller | "replicas": 0 2026-06-10 11:15:18.065949 | controller | } 2026-06-10 11:15:18.072264 | controller | ok: 2026-06-10 11:15:18.072374 | controller | { 2026-06-10 11:15:18.072427 | controller | "kind": "deployment", 2026-06-10 11:15:18.072465 | controller | "name": "zuul-web", 2026-06-10 11:15:18.072500 | controller | "replicas": 0 2026-06-10 11:15:18.072534 | controller | } 2026-06-10 11:15:18.090882 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:15:18.094769 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:15:18.098304 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:15:18.101809 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:15:18.165649 | 2026-06-10 11:15:18.165793 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 11:15:18.691579 | controller | ok: Runtime: 0:00:00.089457 2026-06-10 11:15:18.703760 | 2026-06-10 11:15:18.703896 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 07:15:19.045661 | controller | Defaulted container "zuul-executor" out of: zuul-executor, zuul-executor-nodeexporter 2026-06-10 11:15:19.251753 | controller | ok: Runtime: 0:00:00.095434 2026-06-10 11:15:19.263404 | 2026-06-10 11:15:19.263548 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-10 07:15:19.587637 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 11:15:19.809771 | controller | ok: Runtime: 0:00:00.084295 2026-06-10 11:15:19.821863 | 2026-06-10 11:15:19.822072 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 11:15:30.522603 | controller | ok: Runtime: 0:00:00.066572 2026-06-10 11:15:30.535018 | 2026-06-10 11:15:30.535185 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 11:15:31.077680 | controller | ok: Runtime: 0:00:00.091537 2026-06-10 11:15:31.088955 | 2026-06-10 11:15:31.089142 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 07:15:31.433518 | controller | Defaulted container "zuul-merger" out of: zuul-merger, zuul-merger-nodeexporter 2026-06-10 11:15:31.634979 | controller | ok: Runtime: 0:00:00.070483 2026-06-10 11:15:31.640584 | 2026-06-10 11:15:31.640663 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-10 07:15:31.923536 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 11:15:32.167628 | controller | ok: Runtime: 0:00:00.065917 2026-06-10 11:15:32.178325 | 2026-06-10 11:15:32.178493 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 11:15:42.835589 | controller | ok: Runtime: 0:00:00.073627 2026-06-10 11:15:42.846691 | 2026-06-10 11:15:42.846826 | TASK [health-check/scale-resources : Get initial replica count for zuul-scheduler] 2026-06-10 11:15:43.392341 | controller | ok: Runtime: 0:00:00.051458 2026-06-10 11:15:43.403764 | 2026-06-10 11:15:43.403907 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 07:15:43.815396 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 11:15:43.979670 | controller | ok: Runtime: 0:00:00.085276 2026-06-10 11:15:43.990825 | 2026-06-10 11:15:43.990982 | TASK [health-check/scale-resources : Scale on sts zuul-scheduler] 2026-06-10 07:15:44.369129 | controller | statefulset.apps/zuul-scheduler scaled 2026-06-10 11:15:44.588285 | controller | ok: Runtime: 0:00:00.066705 2026-06-10 11:15:44.600635 | 2026-06-10 11:15:44.600801 | TASK [health-check/scale-resources : Get replica count for zuul-scheduler] 2026-06-10 11:15:55.234816 | controller | ok: Runtime: 0:00:00.073376 2026-06-10 11:15:55.246220 | 2026-06-10 11:15:55.246412 | TASK [health-check/scale-resources : Get initial replica count for zuul-web] 2026-06-10 11:15:55.789941 | controller | ok: Runtime: 0:00:00.071329 2026-06-10 11:15:55.801248 | 2026-06-10 11:15:55.801420 | TASK [health-check/scale-resources : Collect deployment log before executing the scale command] 2026-06-10 11:15:56.346965 | controller | ok: Runtime: 0:00:00.106834 2026-06-10 11:15:56.359071 | 2026-06-10 11:15:56.359237 | TASK [health-check/scale-resources : Scale on deployment zuul-web] 2026-06-10 07:15:56.702391 | controller | deployment.apps/zuul-web scaled 2026-06-10 11:15:56.908270 | controller | ok: Runtime: 0:00:00.104311 2026-06-10 11:15:56.919949 | 2026-06-10 11:15:56.920090 | TASK [health-check/scale-resources : Get replica count for zuul-web] 2026-06-10 11:15:57.465785 | controller | ok: Runtime: 0:00:00.067883 2026-06-10 11:15:57.483176 | 2026-06-10 11:15:57.483310 | TASK [Enable log forwarding] 2026-06-10 11:15:57.519272 | controller | ok 2026-06-10 11:15:57.550166 | 2026-06-10 11:15:57.550246 | TASK [update-custom-resource : read current crd] 2026-06-10 11:15:57.951450 | controller | ok 2026-06-10 11:15:57.961172 | 2026-06-10 11:15:57.961383 | TASK [update-custom-resource : merge the new cr] 2026-06-10 11:15:58.852245 | controller | changed 2026-06-10 11:15:58.861460 | 2026-06-10 11:15:58.861592 | TASK [run the operator in standalone mode] 2026-06-10 11:15:58.898192 | controller | ok 2026-06-10 11:15:58.932191 | 2026-06-10 11:15:58.932284 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:15:58.957769 | controller | skipping: Conditional result was False 2026-06-10 11:15:58.969836 | 2026-06-10 11:15:58.969983 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:15:58.997512 | controller | skipping: Conditional result was False 2026-06-10 11:15:59.011272 | 2026-06-10 11:15:59.011446 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:15:59.585054 | controller | 2026-06-10T07:15:59-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:15:59.585369 | controller | 2026-06-10 07:15:59.594523 | controller | 2026-06-10T07:15:59-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:15:59.594569 | controller | 2026-06-10 07:15:59.605349 | controller | 2026-06-10T07:15:59-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:15:59.606640 | controller | 2026-06-10T07:15:59-04:00 INFO Nothing to clean up. 2026-06-10 07:15:59.636691 | controller | 2026-06-10T07:15:59-04:00 INFO Creating config map name: fluentbit-mariadb-cfg-config-map 2026-06-10 07:15:59.636840 | controller | 2026-06-10T07:15:59-04:00 INFO Creating object, name: fluentbit-mariadb-cfg-config-map 2026-06-10 07:15:59.642828 | controller | 2026-06-10T07:15:59-04:00 INFO Statefulset "mariadb" configuration changed, applying... 2026-06-10 07:15:59.642934 | controller | 2026-06-10T07:15:59-04:00 INFO Updating object, name: mariadb 2026-06-10 07:16:00.017045 | controller | 2026-06-10T07:16:00-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:16:00.107146 | controller | 2026-06-10T07:16:00-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:16:00.113793 | controller | 2026-06-10T07:16:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:16:00.113834 | controller | 2026-06-10T07:16:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:16:00.128962 | controller | 2026-06-10T07:16:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:16:00.128993 | controller | 2026-06-10T07:16:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:16:00.420778 | controller | 2026-06-10T07:16:00-04:00 INFO Creating config map name: fluentbit-zk-cfg-config-map 2026-06-10 07:16:00.420836 | controller | 2026-06-10T07:16:00-04:00 INFO Creating object, name: fluentbit-zk-cfg-config-map 2026-06-10 07:16:00.431182 | controller | 2026-06-10T07:16:00-04:00 INFO Statefulset "zookeeper" configuration changed, applying... 2026-06-10 07:16:00.431222 | controller | 2026-06-10T07:16:00-04:00 INFO Updating object, name: zookeeper 2026-06-10 07:16:00.438606 | controller | 2026-06-10T07:16:00-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-10 07:16:00.438662 | controller | 2026-06-10T07:16:00-04:00 INFO 2026-06-10 07:16:00.438671 | controller | - Config: Waiting ... 2026-06-10 07:16:00.438677 | controller | - Gateway: OK 2026-06-10 07:16:00.438683 | controller | - GitServer: OK 2026-06-10 07:16:00.438688 | controller | - HoundSearch: OK 2026-06-10 07:16:00.438693 | controller | - LogJuicer: OK 2026-06-10 07:16:00.438698 | controller | - Logserver: OK 2026-06-10 07:16:00.438703 | controller | - MariaDB: Waiting ... 2026-06-10 07:16:00.438708 | controller | - Zookeeper: Waiting ... 2026-06-10 07:16:00.438713 | controller | 2026-06-10 07:16:00.438719 | controller | 2026-06-10T07:16:00-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:16:05.447798 | controller | 2026-06-10T07:16:05-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:16:05.449063 | controller | 2026-06-10T07:16:05-04:00 INFO Nothing to clean up. 2026-06-10 07:16:05.484281 | controller | 2026-06-10T07:16:05-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-10 07:16:09.532763 | controller | 2026-06-10T07:16:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:16:09.532831 | controller | 2026-06-10T07:16:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:16:09.547410 | controller | 2026-06-10T07:16:09-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:16:09.547473 | controller | 2026-06-10T07:16:09-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:16:09.571905 | controller | 2026-06-10T07:16:09-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-10 07:16:59.773833 | controller | 2026-06-10T07:16:59-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:16:59.781717 | controller | 2026-06-10T07:16:59-04:00 INFO Updating configmap, name: nodepool-launcher-extra-config-config-map 2026-06-10 07:16:59.781790 | controller | 2026-06-10T07:16:59-04:00 INFO Updating object, name: nodepool-launcher-extra-config-config-map 2026-06-10 07:16:59.790322 | controller | 2026-06-10T07:16:59-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-10 07:16:59.790383 | controller | 2026-06-10T07:16:59-04:00 INFO Updating object, name: nodepool-launcher 2026-06-10 07:16:59.805012 | controller | 2026-06-10T07:16:59-04:00 INFO Updating configmap, name: nodepool-builder-extra-config-config-map 2026-06-10 07:16:59.805081 | controller | 2026-06-10T07:16:59-04:00 INFO Updating object, name: nodepool-builder-extra-config-config-map 2026-06-10 07:16:59.810872 | controller | 2026-06-10T07:16:59-04:00 INFO Creating config map name: fluentbit-dib-cfg-config-map 2026-06-10 07:16:59.810919 | controller | 2026-06-10T07:16:59-04:00 INFO Creating object, name: fluentbit-dib-cfg-config-map 2026-06-10 07:16:59.989801 | controller | 2026-06-10T07:16:59-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-10 07:16:59.989860 | controller | 2026-06-10T07:16:59-04:00 INFO Updating object, name: nodepool-builder 2026-06-10 07:17:00.781547 | controller | 2026-06-10T07:17:00-04:00 INFO Updating configmap, name: zuul-logging-config-map 2026-06-10 07:17:00.781626 | controller | 2026-06-10T07:17:00-04:00 INFO Updating object, name: zuul-logging-config-map 2026-06-10 07:17:00.985707 | controller | 2026-06-10T07:17:00-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:17:00.985778 | controller | 2026-06-10T07:17:00-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:17:01.785020 | controller | 2026-06-10T07:17:01-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 07:17:01.785091 | controller | 2026-06-10T07:17:01-04:00 INFO Updating object, name: zuul-executor 2026-06-10 07:17:02.584680 | controller | 2026-06-10T07:17:02-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:17:02.584740 | controller | 2026-06-10T07:17:02-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:17:03.385635 | controller | 2026-06-10T07:17:03-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-10 07:17:03.385698 | controller | 2026-06-10T07:17:03-04:00 INFO Updating object, name: zuul-merger 2026-06-10 07:17:03.395925 | controller | 2026-06-10T07:17:03-04:00 INFO 2026-06-10 07:17:03.395981 | controller | - Config: Waiting ... 2026-06-10 07:17:03.395995 | controller | - Gateway: OK 2026-06-10 07:17:03.396008 | controller | - GitServer: OK 2026-06-10 07:17:03.396019 | controller | - HoundSearch: OK 2026-06-10 07:17:03.396030 | controller | - LogJuicer: OK 2026-06-10 07:17:03.396042 | controller | - Logserver: OK 2026-06-10 07:17:03.396053 | controller | - MariaDB: OK 2026-06-10 07:17:03.396064 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:17:03.396075 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:17:03.396086 | controller | - Zookeeper: OK 2026-06-10 07:17:03.396097 | controller | - Zuul: Waiting ... 2026-06-10 07:17:03.396108 | controller | - |_ Config Secrets: OK 2026-06-10 07:17:03.396119 | controller | - |_ Executor: Waiting ... 2026-06-10 07:17:03.396132 | controller | - |_ Merger: Waiting ... 2026-06-10 07:17:03.396143 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:17:03.396153 | controller | - |_ Web: Waiting ... 2026-06-10 07:17:03.396172 | controller | 2026-06-10 07:17:03.396229 | controller | 2026-06-10T07:17:03-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:17:08.403073 | controller | 2026-06-10T07:17:08-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:17:08.406062 | controller | 2026-06-10T07:17:08-04:00 INFO Nothing to clean up. 2026-06-10 07:17:09.017442 | controller | 2026-06-10T07:17:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:17:09.017492 | controller | 2026-06-10T07:17:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:17:09.033404 | controller | 2026-06-10T07:17:09-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:17:09.033501 | controller | 2026-06-10T07:17:09-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:17:09.426636 | controller | 2026-06-10T07:17:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:17:09.817510 | controller | 2026-06-10T07:17:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:17:09.817878 | controller | 2026-06-10T07:17:09-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 07:17:10.836903 | controller | 2026-06-10T07:17:10-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:17:10.836991 | controller | 2026-06-10T07:17:10-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:17:12.407134 | controller | 2026-06-10T07:17:12-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:17:13.614488 | controller | 2026-06-10T07:17:13-04:00 INFO 2026-06-10 07:17:13.614580 | controller | - Config: OK 2026-06-10 07:17:13.614592 | controller | - Gateway: OK 2026-06-10 07:17:13.614602 | controller | - GitServer: OK 2026-06-10 07:17:13.614610 | controller | - HoundSearch: OK 2026-06-10 07:17:13.614619 | controller | - LogJuicer: OK 2026-06-10 07:17:13.614628 | controller | - Logserver: OK 2026-06-10 07:17:13.614636 | controller | - MariaDB: OK 2026-06-10 07:17:13.614645 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:17:13.614654 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:17:13.614663 | controller | - Zookeeper: OK 2026-06-10 07:17:13.614672 | controller | - Zuul: OK 2026-06-10 07:17:13.614681 | controller | 2026-06-10 07:17:13.614703 | controller | 2026-06-10T07:17:13-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:17:18.619803 | controller | 2026-06-10T07:17:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:17:18.621060 | controller | 2026-06-10T07:17:18-04:00 INFO Nothing to clean up. 2026-06-10 07:17:19.231300 | controller | 2026-06-10T07:17:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:17:19.231348 | controller | 2026-06-10T07:17:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:17:19.249220 | controller | 2026-06-10T07:17:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:17:19.249301 | controller | 2026-06-10T07:17:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:17:19.636873 | controller | 2026-06-10T07:17:19-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:17:20.044762 | controller | 2026-06-10T07:17:20-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:17:20.044845 | controller | 2026-06-10T07:17:20-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 07:17:21.035992 | controller | 2026-06-10T07:17:21-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:17:21.036046 | controller | 2026-06-10T07:17:21-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:17:22.624861 | controller | 2026-06-10T07:17:22-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:17:23.830002 | controller | 2026-06-10T07:17:23-04:00 INFO 2026-06-10 07:17:23.830053 | controller | - Config: OK 2026-06-10 07:17:23.830058 | controller | - Gateway: OK 2026-06-10 07:17:23.830062 | controller | - GitServer: OK 2026-06-10 07:17:23.830066 | controller | - HoundSearch: OK 2026-06-10 07:17:23.830070 | controller | - LogJuicer: OK 2026-06-10 07:17:23.830073 | controller | - Logserver: OK 2026-06-10 07:17:23.830077 | controller | - MariaDB: OK 2026-06-10 07:17:23.830080 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:17:23.830084 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:17:23.830088 | controller | - Zookeeper: OK 2026-06-10 07:17:23.830091 | controller | - Zuul: OK 2026-06-10 07:17:23.830095 | controller | 2026-06-10 07:17:23.830103 | controller | 2026-06-10T07:17:23-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 07:17:28.836612 | controller | 2026-06-10T07:17:28-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:17:28.839424 | controller | 2026-06-10T07:17:28-04:00 INFO Nothing to clean up. 2026-06-10 07:17:29.453271 | controller | 2026-06-10T07:17:29-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:17:29.453316 | controller | 2026-06-10T07:17:29-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:17:29.472008 | controller | 2026-06-10T07:17:29-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:17:29.472080 | controller | 2026-06-10T07:17:29-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:17:29.863017 | controller | 2026-06-10T07:17:29-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:17:30.246979 | controller | 2026-06-10T07:17:30-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:17:30.247033 | controller | 2026-06-10T07:17:30-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 07:17:31.266005 | controller | 2026-06-10T07:17:31-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:17:31.266078 | controller | 2026-06-10T07:17:31-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:17:32.841680 | controller | 2026-06-10T07:17:32-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:17:34.045548 | controller | 2026-06-10T07:17:34-04:00 INFO 2026-06-10 07:17:34.045610 | controller | - Config: OK 2026-06-10 07:17:34.045622 | controller | - Gateway: OK 2026-06-10 07:17:34.045634 | controller | - GitServer: OK 2026-06-10 07:17:34.045645 | controller | - HoundSearch: OK 2026-06-10 07:17:34.045678 | controller | - LogJuicer: OK 2026-06-10 07:17:34.045690 | controller | - Logserver: OK 2026-06-10 07:17:34.045702 | controller | - MariaDB: OK 2026-06-10 07:17:34.045713 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:17:34.045724 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:17:34.045766 | controller | - Zookeeper: OK 2026-06-10 07:17:34.045779 | controller | - Zuul: OK 2026-06-10 07:17:34.045791 | controller | 2026-06-10 07:17:34.045804 | controller | 2026-06-10T07:17:34-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 07:17:39.049888 | controller | 2026-06-10T07:17:39-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:17:39.051612 | controller | 2026-06-10T07:17:39-04:00 INFO Nothing to clean up. 2026-06-10 07:17:39.660332 | controller | 2026-06-10T07:17:39-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:17:39.660420 | controller | 2026-06-10T07:17:39-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:17:39.673701 | controller | 2026-06-10T07:17:39-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:17:39.673754 | controller | 2026-06-10T07:17:39-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:17:40.069268 | controller | 2026-06-10T07:17:40-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:17:40.461255 | controller | 2026-06-10T07:17:40-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:17:40.461334 | controller | 2026-06-10T07:17:40-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:17:41.473156 | controller | 2026-06-10T07:17:41-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:17:41.473305 | controller | 2026-06-10T07:17:41-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:17:43.056872 | controller | 2026-06-10T07:17:43-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:17:44.263672 | controller | 2026-06-10T07:17:44-04:00 INFO 2026-06-10 07:17:44.263788 | controller | - Config: OK 2026-06-10 07:17:44.263794 | controller | - Gateway: OK 2026-06-10 07:17:44.263798 | controller | - GitServer: OK 2026-06-10 07:17:44.263802 | controller | - HoundSearch: OK 2026-06-10 07:17:44.263806 | controller | - LogJuicer: OK 2026-06-10 07:17:44.263810 | controller | - Logserver: OK 2026-06-10 07:17:44.263813 | controller | - MariaDB: OK 2026-06-10 07:17:44.263817 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:17:44.263821 | controller | - NodePoolLauncher: OK 2026-06-10 07:17:44.263825 | controller | - Zookeeper: OK 2026-06-10 07:17:44.263829 | controller | - Zuul: OK 2026-06-10 07:17:44.263833 | controller | 2026-06-10 07:17:44.263845 | controller | 2026-06-10T07:17:44-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 07:17:49.272280 | controller | 2026-06-10T07:17:49-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:17:49.273679 | controller | 2026-06-10T07:17:49-04:00 INFO Nothing to clean up. 2026-06-10 07:17:49.890589 | controller | 2026-06-10T07:17:49-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:17:49.890678 | controller | 2026-06-10T07:17:49-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:17:49.910562 | controller | 2026-06-10T07:17:49-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:17:49.910646 | controller | 2026-06-10T07:17:49-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:17:50.296261 | controller | 2026-06-10T07:17:50-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:17:50.686807 | controller | 2026-06-10T07:17:50-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:17:50.686852 | controller | 2026-06-10T07:17:50-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:17:51.702371 | controller | 2026-06-10T07:17:51-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:17:51.702436 | controller | 2026-06-10T07:17:51-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:17:53.275283 | controller | 2026-06-10T07:17:53-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:17:54.485551 | controller | 2026-06-10T07:17:54-04:00 INFO 2026-06-10 07:17:54.485636 | controller | - Config: OK 2026-06-10 07:17:54.485651 | controller | - Gateway: OK 2026-06-10 07:17:54.485663 | controller | - GitServer: OK 2026-06-10 07:17:54.485673 | controller | - HoundSearch: OK 2026-06-10 07:17:54.485684 | controller | - LogJuicer: OK 2026-06-10 07:17:54.485694 | controller | - Logserver: OK 2026-06-10 07:17:54.485705 | controller | - MariaDB: OK 2026-06-10 07:17:54.485741 | controller | - NodePoolBuilder: OK 2026-06-10 07:17:54.485753 | controller | - NodePoolLauncher: OK 2026-06-10 07:17:54.485764 | controller | - Zookeeper: OK 2026-06-10 07:17:54.485775 | controller | - Zuul: OK 2026-06-10 07:17:54.485787 | controller | 2026-06-10 07:17:54.485809 | controller | 2026-06-10T07:17:54-04:00 INFO Updating controller configmap ... 2026-06-10 07:17:54.874801 | controller | 2026-06-10T07:17:54-04:00 INFO Standalone reconcile done. 2026-06-10 11:17:55.202845 | controller | ok: Runtime: 0:01:55.598739 2026-06-10 11:17:55.215817 | 2026-06-10 11:17:55.215988 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:17:55.757392 | controller | skipping: Conditional result was False 2026-06-10 11:17:55.783084 | 2026-06-10 11:17:55.783228 | TASK [scale zuul components back up to defaults] 2026-06-10 11:17:55.817936 | controller | ok 2026-06-10 11:17:55.849109 | 2026-06-10 11:17:55.849213 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 11:17:55.883869 | controller | ok: 2026-06-10 11:17:55.884524 | controller | { 2026-06-10 11:17:55.884592 | controller | "kind": "sts", 2026-06-10 11:17:55.884632 | controller | "name": "zuul-executor", 2026-06-10 11:17:55.884669 | controller | "replicas": 1 2026-06-10 11:17:55.884703 | controller | } 2026-06-10 11:17:55.891757 | controller | ok: 2026-06-10 11:17:55.891857 | controller | { 2026-06-10 11:17:55.891904 | controller | "kind": "sts", 2026-06-10 11:17:55.891941 | controller | "name": "zuul-merger", 2026-06-10 11:17:55.891977 | controller | "replicas": 1 2026-06-10 11:17:55.892012 | controller | } 2026-06-10 11:17:55.902315 | controller | ok: 2026-06-10 11:17:55.902556 | controller | { 2026-06-10 11:17:55.902633 | controller | "kind": "sts", 2026-06-10 11:17:55.902673 | controller | "name": "zuul-scheduler", 2026-06-10 11:17:55.902707 | controller | "replicas": 1 2026-06-10 11:17:55.902740 | controller | } 2026-06-10 11:17:55.973079 | controller | ok: 2026-06-10 11:17:55.973206 | controller | { 2026-06-10 11:17:55.973227 | controller | "kind": "deployment", 2026-06-10 11:17:55.973243 | controller | "name": "zuul-web", 2026-06-10 11:17:55.973259 | controller | "replicas": 1 2026-06-10 11:17:55.973275 | controller | } 2026-06-10 11:17:55.995896 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:17:56.001638 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:17:56.007650 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:17:56.013179 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:17:56.023482 | 2026-06-10 11:17:56.023601 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 11:17:56.561591 | controller | ok: Runtime: 0:00:00.076134 2026-06-10 11:17:56.572824 | 2026-06-10 11:17:56.572961 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 11:17:57.114837 | controller | skipping: Conditional result was False 2026-06-10 11:17:57.128217 | 2026-06-10 11:17:57.128440 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-10 07:17:57.493657 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 11:17:57.675026 | controller | ok: Runtime: 0:00:00.080153 2026-06-10 11:17:57.686228 | 2026-06-10 11:17:57.686395 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 11:18:18.675019 | controller | ok: Runtime: 0:00:00.072043 2026-06-10 11:18:18.686232 | 2026-06-10 11:18:18.686405 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 11:18:19.230442 | controller | ok: Runtime: 0:00:00.069463 2026-06-10 11:18:19.241716 | 2026-06-10 11:18:19.241857 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 11:18:19.782647 | controller | skipping: Conditional result was False 2026-06-10 11:18:19.795480 | 2026-06-10 11:18:19.795636 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-10 07:18:20.120316 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 11:18:20.341868 | controller | ok: Runtime: 0:00:00.100323 2026-06-10 11:18:20.352654 | 2026-06-10 11:18:20.352788 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 11:18:41.316581 | controller | ok: Runtime: 0:00:00.066062 2026-06-10 11:18:41.322052 | 2026-06-10 11:18:41.322115 | TASK [health-check/scale-resources : Get initial replica count for zuul-scheduler] 2026-06-10 11:18:41.852324 | controller | ok: Runtime: 0:00:00.066383 2026-06-10 11:18:41.863945 | 2026-06-10 11:18:41.864091 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 11:18:42.400789 | controller | skipping: Conditional result was False 2026-06-10 11:18:42.413917 | 2026-06-10 11:18:42.414061 | TASK [health-check/scale-resources : Scale on sts zuul-scheduler] 2026-06-10 07:18:42.751343 | controller | statefulset.apps/zuul-scheduler scaled 2026-06-10 11:18:42.959927 | controller | ok: Runtime: 0:00:00.096404 2026-06-10 11:18:42.971058 | 2026-06-10 11:18:42.971203 | TASK [health-check/scale-resources : Get replica count for zuul-scheduler] 2026-06-10 11:19:45.069177 | controller | ok: Runtime: 0:00:00.060697 2026-06-10 11:19:45.080886 | 2026-06-10 11:19:45.081027 | TASK [health-check/scale-resources : Get initial replica count for zuul-web] 2026-06-10 11:19:45.684334 | controller | ok: Runtime: 0:00:00.059380 2026-06-10 11:19:45.696767 | 2026-06-10 11:19:45.696910 | TASK [health-check/scale-resources : Collect deployment log before executing the scale command] 2026-06-10 11:19:46.237187 | controller | skipping: Conditional result was False 2026-06-10 11:19:46.250026 | 2026-06-10 11:19:46.250178 | TASK [health-check/scale-resources : Scale on deployment zuul-web] 2026-06-10 07:19:46.540676 | controller | deployment.apps/zuul-web scaled 2026-06-10 11:19:46.795233 | controller | ok: Runtime: 0:00:00.064635 2026-06-10 11:19:46.806223 | 2026-06-10 11:19:46.806383 | TASK [health-check/scale-resources : Get replica count for zuul-web] 2026-06-10 11:20:48.863649 | controller | ok: Runtime: 0:00:00.059965 2026-06-10 11:20:48.887053 | 2026-06-10 11:20:48.887199 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 11:20:48.923716 | controller | ok: 2026-06-10 11:20:48.924026 | controller | { 2026-06-10 11:20:48.924080 | controller | "kind": "sts", 2026-06-10 11:20:48.924119 | controller | "name": "zuul-executor", 2026-06-10 11:20:48.924154 | controller | "replicas": 2 2026-06-10 11:20:48.924188 | controller | } 2026-06-10 11:20:48.924244 | controller | ok: All items complete 2026-06-10 11:20:48.924282 | 2026-06-10 11:20:48.931787 | controller | ok: 2026-06-10 11:20:48.931883 | controller | { 2026-06-10 11:20:48.931929 | controller | "kind": "sts", 2026-06-10 11:20:48.932120 | controller | "name": "zuul-merger", 2026-06-10 11:20:48.932172 | controller | "replicas": 2 2026-06-10 11:20:48.932207 | controller | } 2026-06-10 11:20:48.956307 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:20:48.961992 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 11:20:48.972084 | 2026-06-10 11:20:48.972208 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 11:20:49.507464 | controller | ok: Runtime: 0:00:00.058110 2026-06-10 11:20:49.518299 | 2026-06-10 11:20:49.518462 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 07:20:49.875609 | controller | Defaulted container "zuul-executor" out of: zuul-executor, zuul-executor-nodeexporter 2026-06-10 11:20:50.065119 | controller | ok: Runtime: 0:00:00.073646 2026-06-10 11:20:50.076340 | 2026-06-10 11:20:50.076511 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-10 07:20:50.414980 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 11:20:50.622659 | controller | ok: Runtime: 0:00:00.069631 2026-06-10 11:20:50.633703 | 2026-06-10 11:20:50.633844 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 11:21:11.567707 | controller | ok: Runtime: 0:00:00.092825 2026-06-10 11:21:11.579286 | 2026-06-10 11:21:11.579498 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 11:21:12.120947 | controller | ok: Runtime: 0:00:00.083696 2026-06-10 11:21:12.128557 | 2026-06-10 11:21:12.128644 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 07:21:12.497629 | controller | Defaulted container "zuul-merger" out of: zuul-merger, zuul-merger-nodeexporter 2026-06-10 11:21:12.663528 | controller | ok: Runtime: 0:00:00.107387 2026-06-10 11:21:12.674713 | 2026-06-10 11:21:12.674848 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-10 07:21:13.025805 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 11:21:13.221960 | controller | ok: Runtime: 0:00:00.094612 2026-06-10 11:21:13.232896 | 2026-06-10 11:21:13.233027 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 11:21:34.210208 | controller | ok: Runtime: 0:00:00.058672 2026-06-10 11:21:34.227005 | 2026-06-10 11:21:34.227144 | LOOP [health-check/zuul-components : Check zuul components sequentially] 2026-06-10 11:21:34.266167 | controller | ok: 2026-06-10 11:21:34.266567 | controller | { 2026-06-10 11:21:34.266623 | controller | "component": "web", 2026-06-10 11:21:34.266663 | controller | "kind": "deployment" 2026-06-10 11:21:34.266698 | controller | } 2026-06-10 11:21:34.273339 | controller | ok: 2026-06-10 11:21:34.273483 | controller | { 2026-06-10 11:21:34.273564 | controller | "component": "scheduler", 2026-06-10 11:21:34.273640 | controller | "kind": "statefulset" 2026-06-10 11:21:34.273706 | controller | } 2026-06-10 11:21:34.280867 | controller | ok: 2026-06-10 11:21:34.280960 | controller | { 2026-06-10 11:21:34.281006 | controller | "component": "merger", 2026-06-10 11:21:34.281065 | controller | "kind": "statefulset" 2026-06-10 11:21:34.281102 | controller | } 2026-06-10 11:21:34.287827 | controller | ok: 2026-06-10 11:21:34.287951 | controller | { 2026-06-10 11:21:34.288004 | controller | "component": "executor", 2026-06-10 11:21:34.288043 | controller | "kind": "statefulset" 2026-06-10 11:21:34.288077 | controller | } 2026-06-10 11:21:34.305565 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 11:21:34.310815 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 11:21:34.316125 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 11:21:34.321503 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 11:21:34.332038 | 2026-06-10 11:21:34.332165 | TASK [health-check/zuul-components : Get expected replica count for zuul-web] 2026-06-10 11:21:34.866690 | controller | ok: Runtime: 0:00:00.070224 2026-06-10 11:21:34.877965 | 2026-06-10 11:21:34.878098 | TASK [health-check/zuul-components : Wait until all of zuul-web replicas are ready] 2026-06-10 11:21:35.425785 | controller | ok: Runtime: 0:00:00.082225 2026-06-10 11:21:35.436764 | 2026-06-10 11:21:35.436834 | TASK [health-check/zuul-components : Validate zuul-web Component(s)] 2026-06-10 11:21:36.128431 | controller | ok: OK (799 bytes) 2026-06-10 11:21:36.140060 | 2026-06-10 11:21:36.140183 | TASK [health-check/zuul-components : Get expected replica count for zuul-scheduler] 2026-06-10 11:21:36.684210 | controller | ok: Runtime: 0:00:00.060887 2026-06-10 11:21:36.696053 | 2026-06-10 11:21:36.696187 | TASK [health-check/zuul-components : Wait until all of zuul-scheduler replicas are ready] 2026-06-10 11:21:37.244465 | controller | ok: Runtime: 0:00:00.081087 2026-06-10 11:21:37.255628 | 2026-06-10 11:21:37.255764 | TASK [health-check/zuul-components : Validate zuul-scheduler Component(s)] 2026-06-10 11:21:37.825559 | controller | ok: OK (799 bytes) 2026-06-10 11:21:37.835015 | 2026-06-10 11:21:37.835108 | TASK [health-check/zuul-components : Get expected replica count for zuul-merger] 2026-06-10 11:21:38.376253 | controller | ok: Runtime: 0:00:00.095964 2026-06-10 11:21:38.389651 | 2026-06-10 11:21:38.389787 | TASK [health-check/zuul-components : Wait until all of zuul-merger replicas are ready] 2026-06-10 11:21:38.935814 | controller | ok: Runtime: 0:00:00.065486 2026-06-10 11:21:38.949037 | 2026-06-10 11:21:38.949177 | TASK [health-check/zuul-components : Validate zuul-merger Component(s)] 2026-06-10 11:21:39.631667 | controller | ok: OK (799 bytes) 2026-06-10 11:21:39.646064 | 2026-06-10 11:21:39.646196 | TASK [health-check/zuul-components : Get expected replica count for zuul-executor] 2026-06-10 11:21:40.193719 | controller | ok: Runtime: 0:00:00.065028 2026-06-10 11:21:40.205742 | 2026-06-10 11:21:40.205875 | TASK [health-check/zuul-components : Wait until all of zuul-executor replicas are ready] 2026-06-10 11:21:40.832188 | controller | ok: Runtime: 0:00:00.083055 2026-06-10 11:21:40.837457 | 2026-06-10 11:21:40.837521 | TASK [health-check/zuul-components : Validate zuul-executor Component(s)] 2026-06-10 11:21:41.397051 | controller | ok: OK (799 bytes) 2026-06-10 11:21:41.414835 | 2026-06-10 11:21:41.414968 | TASK [health-check/zuul-connections : Setting Dummy Connections] 2026-06-10 11:21:41.461783 | controller | ok 2026-06-10 11:21:41.473121 | 2026-06-10 11:21:41.473259 | TASK [health-check/zuul-connections : Create SMTP Connection Secret] 2026-06-10 11:21:42.575343 | controller | changed 2026-06-10 11:21:42.585491 | 2026-06-10 11:21:42.585634 | TASK [health-check/zuul-connections : Create ElasticSearch Connection Secret] 2026-06-10 11:21:43.419246 | controller | changed 2026-06-10 11:21:43.429486 | 2026-06-10 11:21:43.429618 | TASK [health-check/zuul-connections : Create Gerrit Connection Secret] 2026-06-10 11:21:44.257727 | controller | changed 2026-06-10 11:21:44.268003 | 2026-06-10 11:21:44.268136 | TASK [health-check/zuul-connections : Create GitHub Connection Secret] 2026-06-10 11:21:45.098693 | controller | changed 2026-06-10 11:21:45.110250 | 2026-06-10 11:21:45.110503 | TASK [health-check/zuul-connections : Create GitLab Connection Secret] 2026-06-10 11:21:45.881906 | controller | changed 2026-06-10 11:21:45.892696 | 2026-06-10 11:21:45.892818 | TASK [health-check/zuul-connections : Create Pagure Connection Secret] 2026-06-10 11:21:46.713566 | controller | changed 2026-06-10 11:21:46.723705 | 2026-06-10 11:21:46.723839 | TASK [health-check/zuul-connections : set_fact] 2026-06-10 11:21:46.760794 | controller | ok 2026-06-10 11:21:46.772497 | 2026-06-10 11:21:46.772635 | TASK [Add dummy Zuul connections] 2026-06-10 11:21:46.807983 | controller | ok 2026-06-10 11:21:46.837411 | 2026-06-10 11:21:46.837500 | TASK [update-custom-resource : read current crd] 2026-06-10 11:21:47.079519 | controller | ok 2026-06-10 11:21:47.088189 | 2026-06-10 11:21:47.088326 | TASK [update-custom-resource : merge the new cr] 2026-06-10 11:21:47.908233 | controller | changed 2026-06-10 11:21:47.920156 | 2026-06-10 11:21:47.920283 | TASK [run the operator in standalone mode] 2026-06-10 11:21:47.955506 | controller | ok 2026-06-10 11:21:47.989422 | 2026-06-10 11:21:47.989583 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:21:48.025842 | controller | skipping: Conditional result was False 2026-06-10 11:21:48.032738 | 2026-06-10 11:21:48.032829 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:21:48.058413 | controller | skipping: Conditional result was False 2026-06-10 11:21:48.067509 | 2026-06-10 11:21:48.067598 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:21:48.603745 | controller | 2026-06-10T07:21:48-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:21:48.603843 | controller | 2026-06-10 07:21:48.617390 | controller | 2026-06-10T07:21:48-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:21:48.617458 | controller | 2026-06-10 07:21:48.629244 | controller | 2026-06-10T07:21:48-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:21:48.630744 | controller | 2026-06-10T07:21:48-04:00 INFO Nothing to clean up. 2026-06-10 07:21:48.644681 | controller | 2026-06-10T07:21:48-04:00 INFO Updating configmap, name: git-server-pi-config-map 2026-06-10 07:21:48.644748 | controller | 2026-06-10T07:21:48-04:00 INFO Updating object, name: git-server-pi-config-map 2026-06-10 07:21:48.651764 | controller | 2026-06-10T07:21:48-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 07:21:48.651838 | controller | 2026-06-10T07:21:48-04:00 INFO Updating object, name: git-server 2026-06-10 07:21:50.042955 | controller | 2026-06-10T07:21:50-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:21:50.124369 | controller | 2026-06-10T07:21:50-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:21:50.130216 | controller | 2026-06-10T07:21:50-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:21:50.130264 | controller | 2026-06-10T07:21:50-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:21:50.144585 | controller | 2026-06-10T07:21:50-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:21:50.144644 | controller | 2026-06-10T07:21:50-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:21:50.452560 | controller | 2026-06-10T07:21:50-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 07:21:50.452625 | controller | 2026-06-10T07:21:50-04:00 INFO 2026-06-10 07:21:50.452634 | controller | - Config: Waiting ... 2026-06-10 07:21:50.452639 | controller | - Gateway: OK 2026-06-10 07:21:50.452642 | controller | - GitServer: Waiting ... 2026-06-10 07:21:50.452646 | controller | - HoundSearch: OK 2026-06-10 07:21:50.452650 | controller | - LogJuicer: OK 2026-06-10 07:21:50.452653 | controller | - Logserver: OK 2026-06-10 07:21:50.452657 | controller | - MariaDB: OK 2026-06-10 07:21:50.452660 | controller | - Zookeeper: OK 2026-06-10 07:21:50.452664 | controller | 2026-06-10 07:21:50.452668 | controller | 2026-06-10T07:21:50-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:21:55.462431 | controller | 2026-06-10T07:21:55-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:21:55.464802 | controller | 2026-06-10T07:21:55-04:00 INFO Nothing to clean up. 2026-06-10 07:21:55.502306 | controller | 2026-06-10T07:21:55-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 07:22:11.601521 | controller | 2026-06-10T07:22:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:22:11.601634 | controller | 2026-06-10T07:22:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:22:11.617544 | controller | 2026-06-10T07:22:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:22:11.617630 | controller | 2026-06-10T07:22:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:22:11.764899 | controller | 2026-06-10T07:22:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:22:11.983736 | controller | 2026-06-10T07:22:11-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:22:11.983813 | controller | 2026-06-10T07:22:11-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:22:12.577767 | controller | 2026-06-10T07:22:12-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:22:12.580661 | controller | 2026-06-10T07:22:12-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:22:14.965078 | controller | 2026-06-10T07:22:14-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:22:14.965154 | controller | 2026-06-10T07:22:14-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:22:15.191622 | controller | 2026-06-10T07:22:15-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:22:15.191740 | controller | 2026-06-10T07:22:15-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:22:15.215892 | controller | 2026-06-10T07:22:15-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 07:22:15.216048 | controller | 2026-06-10T07:22:15-04:00 INFO Updating object, name: zuul-executor 2026-06-10 07:22:15.580215 | controller | 2026-06-10T07:22:15-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:22:15.580329 | controller | 2026-06-10T07:22:15-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:22:16.380323 | controller | 2026-06-10T07:22:16-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-10 07:22:16.380373 | controller | 2026-06-10T07:22:16-04:00 INFO Updating object, name: zuul-merger 2026-06-10 07:22:16.386662 | controller | 2026-06-10T07:22:16-04:00 INFO 2026-06-10 07:22:16.386696 | controller | - Config: Waiting ... 2026-06-10 07:22:16.386702 | controller | - Gateway: OK 2026-06-10 07:22:16.386706 | controller | - GitServer: OK 2026-06-10 07:22:16.386711 | controller | - HoundSearch: OK 2026-06-10 07:22:16.386714 | controller | - LogJuicer: OK 2026-06-10 07:22:16.386718 | controller | - Logserver: OK 2026-06-10 07:22:16.386722 | controller | - MariaDB: OK 2026-06-10 07:22:16.386727 | controller | - NodePoolBuilder: OK 2026-06-10 07:22:16.386731 | controller | - NodePoolLauncher: OK 2026-06-10 07:22:16.386735 | controller | - Zookeeper: OK 2026-06-10 07:22:16.386738 | controller | - Zuul: Waiting ... 2026-06-10 07:22:16.386742 | controller | - |_ Config Secrets: OK 2026-06-10 07:22:16.386746 | controller | - |_ Executor: Waiting ... 2026-06-10 07:22:16.386751 | controller | - |_ Merger: Waiting ... 2026-06-10 07:22:16.386755 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:22:16.386759 | controller | - |_ Web: Waiting ... 2026-06-10 07:22:16.386763 | controller | 2026-06-10 07:22:16.386772 | controller | 2026-06-10T07:22:16-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:22:21.394888 | controller | 2026-06-10T07:22:21-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:22:21.396837 | controller | 2026-06-10T07:22:21-04:00 INFO Nothing to clean up. 2026-06-10 07:22:22.814149 | controller | 2026-06-10T07:22:22-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:22:22.814254 | controller | 2026-06-10T07:22:22-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:22:22.829918 | controller | 2026-06-10T07:22:22-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:22:22.829996 | controller | 2026-06-10T07:22:22-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:22:23.211930 | controller | 2026-06-10T07:22:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:22:23.606000 | controller | 2026-06-10T07:22:23-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:22:23.606046 | controller | 2026-06-10T07:22:23-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:22:23.811196 | controller | 2026-06-10T07:22:23-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:22:24.193535 | controller | 2026-06-10T07:22:24-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:22:26.615682 | controller | 2026-06-10T07:22:26-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:22:56.727754 | controller | 2026-06-10T07:22:56-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:22:56.760489 | controller | 2026-06-10T07:22:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:22:56.760545 | controller | 2026-06-10T07:22:56-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:22:57.144721 | controller | 2026-06-10T07:22:57-04:00 INFO 2026-06-10 07:22:57.144772 | controller | - Config: Waiting ... 2026-06-10 07:22:57.144779 | controller | - Gateway: OK 2026-06-10 07:22:57.144785 | controller | - GitServer: OK 2026-06-10 07:22:57.144790 | controller | - HoundSearch: OK 2026-06-10 07:22:57.144795 | controller | - LogJuicer: OK 2026-06-10 07:22:57.144800 | controller | - Logserver: OK 2026-06-10 07:22:57.144805 | controller | - MariaDB: OK 2026-06-10 07:22:57.144810 | controller | - NodePoolBuilder: OK 2026-06-10 07:22:57.144815 | controller | - NodePoolLauncher: OK 2026-06-10 07:22:57.144820 | controller | - Zookeeper: OK 2026-06-10 07:22:57.144824 | controller | - Zuul: Waiting ... 2026-06-10 07:22:57.144829 | controller | - |_ Config Secrets: OK 2026-06-10 07:22:57.144834 | controller | - |_ Executor: OK 2026-06-10 07:22:57.144839 | controller | - |_ Merger: OK 2026-06-10 07:22:57.144844 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:22:57.144849 | controller | - |_ Web: Waiting ... 2026-06-10 07:22:57.144869 | controller | 2026-06-10 07:22:57.144880 | controller | 2026-06-10T07:22:57-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:23:02.155759 | controller | 2026-06-10T07:23:02-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:23:02.158950 | controller | 2026-06-10T07:23:02-04:00 INFO Nothing to clean up. 2026-06-10 07:23:03.571335 | controller | 2026-06-10T07:23:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:23:03.571400 | controller | 2026-06-10T07:23:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:23:03.587880 | controller | 2026-06-10T07:23:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:23:03.587922 | controller | 2026-06-10T07:23:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:23:03.979931 | controller | 2026-06-10T07:23:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:23:04.375021 | controller | 2026-06-10T07:23:04-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:23:04.375088 | controller | 2026-06-10T07:23:04-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:23:04.573568 | controller | 2026-06-10T07:23:04-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:23:04.951931 | controller | 2026-06-10T07:23:04-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:23:07.379396 | controller | 2026-06-10T07:23:07-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:23:21.437951 | controller | 2026-06-10T07:23:21-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-10 07:23:21.438016 | controller | 2026-06-10T07:23:21-04:00 INFO Updating object, name: zuul-weeder 2026-06-10 07:23:21.485678 | controller | 2026-06-10T07:23:21-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:23:21.485718 | controller | 2026-06-10T07:23:21-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:23:21.853860 | controller | 2026-06-10T07:23:21-04:00 INFO 2026-06-10 07:23:21.853912 | controller | - Config: Waiting ... 2026-06-10 07:23:21.853922 | controller | - Gateway: OK 2026-06-10 07:23:21.853930 | controller | - GitServer: OK 2026-06-10 07:23:21.853938 | controller | - HoundSearch: OK 2026-06-10 07:23:21.853945 | controller | - LogJuicer: OK 2026-06-10 07:23:21.853953 | controller | - Logserver: OK 2026-06-10 07:23:21.853961 | controller | - MariaDB: OK 2026-06-10 07:23:21.853968 | controller | - NodePoolBuilder: OK 2026-06-10 07:23:21.853976 | controller | - NodePoolLauncher: OK 2026-06-10 07:23:21.853983 | controller | - Zookeeper: OK 2026-06-10 07:23:21.853990 | controller | - Zuul: Waiting ... 2026-06-10 07:23:21.853999 | controller | - |_ Config Secrets: OK 2026-06-10 07:23:21.854007 | controller | - |_ Executor: OK 2026-06-10 07:23:21.854014 | controller | - |_ Merger: OK 2026-06-10 07:23:21.854022 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:23:21.854029 | controller | - |_ Web: OK 2026-06-10 07:23:21.854037 | controller | 2026-06-10 07:23:21.854045 | controller | 2026-06-10T07:23:21-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 07:23:26.864540 | controller | 2026-06-10T07:23:26-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:23:26.866557 | controller | 2026-06-10T07:23:26-04:00 INFO Nothing to clean up. 2026-06-10 07:23:28.280674 | controller | 2026-06-10T07:23:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:23:28.280726 | controller | 2026-06-10T07:23:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:23:28.293966 | controller | 2026-06-10T07:23:28-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:23:28.294016 | controller | 2026-06-10T07:23:28-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:23:28.687648 | controller | 2026-06-10T07:23:28-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:23:29.086428 | controller | 2026-06-10T07:23:29-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:23:29.086483 | controller | 2026-06-10T07:23:29-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:23:29.286868 | controller | 2026-06-10T07:23:29-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:23:29.661221 | controller | 2026-06-10T07:23:29-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:23:32.089503 | controller | 2026-06-10T07:23:32-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:23:32.089580 | controller | 2026-06-10T07:23:32-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:23:32.473228 | controller | 2026-06-10T07:23:32-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:23:32.473314 | controller | 2026-06-10T07:23:32-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:23:33.678820 | controller | 2026-06-10T07:23:33-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 07:23:33.868989 | controller | 2026-06-10T07:23:33-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 07:23:39.083955 | controller | 2026-06-10T07:23:39-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:23:39.088755 | controller | 2026-06-10T07:23:39-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-10 07:23:39.091691 | controller | 2026-06-10T07:23:39-04:00 INFO 2026-06-10 07:23:39.091721 | controller | - Config: Waiting ... 2026-06-10 07:23:39.091729 | controller | - Gateway: OK 2026-06-10 07:23:39.091746 | controller | - GitServer: OK 2026-06-10 07:23:39.091752 | controller | - HoundSearch: OK 2026-06-10 07:23:39.091757 | controller | - LogJuicer: OK 2026-06-10 07:23:39.091763 | controller | - Logserver: OK 2026-06-10 07:23:39.091768 | controller | - MariaDB: OK 2026-06-10 07:23:39.091773 | controller | - NodePoolBuilder: OK 2026-06-10 07:23:39.091800 | controller | - NodePoolLauncher: OK 2026-06-10 07:23:39.091806 | controller | - Zookeeper: OK 2026-06-10 07:23:39.091811 | controller | - Zuul: OK 2026-06-10 07:23:39.091817 | controller | 2026-06-10 07:23:39.091826 | controller | 2026-06-10T07:23:39-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 07:23:44.099399 | controller | 2026-06-10T07:23:44-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:23:44.100982 | controller | 2026-06-10T07:23:44-04:00 INFO Nothing to clean up. 2026-06-10 07:23:45.518556 | controller | 2026-06-10T07:23:45-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:23:45.518650 | controller | 2026-06-10T07:23:45-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:23:45.537366 | controller | 2026-06-10T07:23:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:23:45.537436 | controller | 2026-06-10T07:23:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:23:45.924402 | controller | 2026-06-10T07:23:45-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:23:46.307754 | controller | 2026-06-10T07:23:46-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:23:46.307793 | controller | 2026-06-10T07:23:46-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:23:46.513522 | controller | 2026-06-10T07:23:46-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:23:46.898399 | controller | 2026-06-10T07:23:46-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:23:49.329824 | controller | 2026-06-10T07:23:49-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:23:49.329893 | controller | 2026-06-10T07:23:49-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:23:49.707080 | controller | 2026-06-10T07:23:49-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:23:50.518198 | controller | 2026-06-10T07:23:50-04:00 INFO 2026-06-10 07:23:50.518241 | controller | - Config: Waiting ... 2026-06-10 07:23:50.518246 | controller | - Gateway: OK 2026-06-10 07:23:50.518250 | controller | - GitServer: OK 2026-06-10 07:23:50.518253 | controller | - HoundSearch: OK 2026-06-10 07:23:50.518257 | controller | - LogJuicer: OK 2026-06-10 07:23:50.518260 | controller | - Logserver: OK 2026-06-10 07:23:50.518264 | controller | - MariaDB: OK 2026-06-10 07:23:50.518267 | controller | - NodePoolBuilder: OK 2026-06-10 07:23:50.518271 | controller | - NodePoolLauncher: OK 2026-06-10 07:23:50.518274 | controller | - Zookeeper: OK 2026-06-10 07:23:50.518278 | controller | - Zuul: Waiting ... 2026-06-10 07:23:50.518281 | controller | - |_ Config Secrets: OK 2026-06-10 07:23:50.518285 | controller | - |_ Executor: OK 2026-06-10 07:23:50.518288 | controller | - |_ Merger: OK 2026-06-10 07:23:50.518292 | controller | - |_ Scheduler: OK 2026-06-10 07:23:50.518295 | controller | - |_ Web: Waiting ... 2026-06-10 07:23:50.518299 | controller | 2026-06-10 07:23:50.518316 | controller | 2026-06-10T07:23:50-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 07:23:55.526864 | controller | 2026-06-10T07:23:55-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:23:55.529889 | controller | 2026-06-10T07:23:55-04:00 INFO Nothing to clean up. 2026-06-10 07:23:56.938611 | controller | 2026-06-10T07:23:56-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:23:56.938688 | controller | 2026-06-10T07:23:56-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:23:56.959341 | controller | 2026-06-10T07:23:56-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:23:56.959403 | controller | 2026-06-10T07:23:56-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:23:57.377269 | controller | 2026-06-10T07:23:57-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:23:57.744338 | controller | 2026-06-10T07:23:57-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:23:57.744426 | controller | 2026-06-10T07:23:57-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:23:57.942562 | controller | 2026-06-10T07:23:57-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:23:58.325437 | controller | 2026-06-10T07:23:58-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:00.749135 | controller | 2026-06-10T07:24:00-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:24:00.749226 | controller | 2026-06-10T07:24:00-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:24:01.135044 | controller | 2026-06-10T07:24:01-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:24:01.135094 | controller | 2026-06-10T07:24:01-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:24:01.942558 | controller | 2026-06-10T07:24:01-04:00 INFO 2026-06-10 07:24:01.942617 | controller | - Config: Waiting ... 2026-06-10 07:24:01.942623 | controller | - Gateway: OK 2026-06-10 07:24:01.942628 | controller | - GitServer: OK 2026-06-10 07:24:01.942631 | controller | - HoundSearch: OK 2026-06-10 07:24:01.942635 | controller | - LogJuicer: OK 2026-06-10 07:24:01.942638 | controller | - Logserver: OK 2026-06-10 07:24:01.942642 | controller | - MariaDB: OK 2026-06-10 07:24:01.942645 | controller | - NodePoolBuilder: OK 2026-06-10 07:24:01.942649 | controller | - NodePoolLauncher: OK 2026-06-10 07:24:01.942652 | controller | - Zookeeper: OK 2026-06-10 07:24:01.942656 | controller | - Zuul: Waiting ... 2026-06-10 07:24:01.942659 | controller | - |_ Config Secrets: OK 2026-06-10 07:24:01.942663 | controller | - |_ Executor: OK 2026-06-10 07:24:01.942666 | controller | - |_ Merger: OK 2026-06-10 07:24:01.942670 | controller | - |_ Scheduler: OK 2026-06-10 07:24:01.942673 | controller | - |_ Web: Waiting ... 2026-06-10 07:24:01.942677 | controller | 2026-06-10 07:24:01.942681 | controller | 2026-06-10T07:24:01-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-10 07:24:06.951812 | controller | 2026-06-10T07:24:06-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:24:06.955154 | controller | 2026-06-10T07:24:06-04:00 INFO Nothing to clean up. 2026-06-10 07:24:08.367249 | controller | 2026-06-10T07:24:08-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:24:08.367290 | controller | 2026-06-10T07:24:08-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:24:08.384149 | controller | 2026-06-10T07:24:08-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:24:08.384241 | controller | 2026-06-10T07:24:08-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:24:08.781385 | controller | 2026-06-10T07:24:08-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:24:09.163770 | controller | 2026-06-10T07:24:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:24:09.163834 | controller | 2026-06-10T07:24:09-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:24:09.369702 | controller | 2026-06-10T07:24:09-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:09.749038 | controller | 2026-06-10T07:24:09-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:12.177559 | controller | 2026-06-10T07:24:12-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:24:12.177660 | controller | 2026-06-10T07:24:12-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:24:12.559461 | controller | 2026-06-10T07:24:12-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:24:12.559510 | controller | 2026-06-10T07:24:12-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:24:13.370554 | controller | 2026-06-10T07:24:13-04:00 INFO 2026-06-10 07:24:13.370594 | controller | - Config: Waiting ... 2026-06-10 07:24:13.370599 | controller | - Gateway: OK 2026-06-10 07:24:13.370603 | controller | - GitServer: OK 2026-06-10 07:24:13.370607 | controller | - HoundSearch: OK 2026-06-10 07:24:13.370611 | controller | - LogJuicer: OK 2026-06-10 07:24:13.370614 | controller | - Logserver: OK 2026-06-10 07:24:13.370618 | controller | - MariaDB: OK 2026-06-10 07:24:13.370621 | controller | - NodePoolBuilder: OK 2026-06-10 07:24:13.370624 | controller | - NodePoolLauncher: OK 2026-06-10 07:24:13.370628 | controller | - Zookeeper: OK 2026-06-10 07:24:13.370631 | controller | - Zuul: Waiting ... 2026-06-10 07:24:13.370635 | controller | - |_ Config Secrets: OK 2026-06-10 07:24:13.370638 | controller | - |_ Executor: OK 2026-06-10 07:24:13.370642 | controller | - |_ Merger: OK 2026-06-10 07:24:13.370645 | controller | - |_ Scheduler: OK 2026-06-10 07:24:13.370648 | controller | - |_ Web: Waiting ... 2026-06-10 07:24:13.370652 | controller | 2026-06-10 07:24:13.370656 | controller | 2026-06-10T07:24:13-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-10 07:24:18.378906 | controller | 2026-06-10T07:24:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:24:18.380786 | controller | 2026-06-10T07:24:18-04:00 INFO Nothing to clean up. 2026-06-10 07:24:19.800162 | controller | 2026-06-10T07:24:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:24:19.800302 | controller | 2026-06-10T07:24:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:24:19.818986 | controller | 2026-06-10T07:24:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:24:19.819040 | controller | 2026-06-10T07:24:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:24:20.195152 | controller | 2026-06-10T07:24:20-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:24:20.591426 | controller | 2026-06-10T07:24:20-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:24:20.591484 | controller | 2026-06-10T07:24:20-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:24:20.789599 | controller | 2026-06-10T07:24:20-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:21.177460 | controller | 2026-06-10T07:24:21-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:23.600470 | controller | 2026-06-10T07:24:23-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:24:23.600526 | controller | 2026-06-10T07:24:23-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:24:23.988144 | controller | 2026-06-10T07:24:23-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:24:23.988293 | controller | 2026-06-10T07:24:23-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:24:24.792968 | controller | 2026-06-10T07:24:24-04:00 INFO 2026-06-10 07:24:24.793006 | controller | - Config: Waiting ... 2026-06-10 07:24:24.793013 | controller | - Gateway: OK 2026-06-10 07:24:24.793018 | controller | - GitServer: OK 2026-06-10 07:24:24.793023 | controller | - HoundSearch: OK 2026-06-10 07:24:24.793028 | controller | - LogJuicer: OK 2026-06-10 07:24:24.793033 | controller | - Logserver: OK 2026-06-10 07:24:24.793038 | controller | - MariaDB: OK 2026-06-10 07:24:24.793041 | controller | - NodePoolBuilder: OK 2026-06-10 07:24:24.793045 | controller | - NodePoolLauncher: OK 2026-06-10 07:24:24.793048 | controller | - Zookeeper: OK 2026-06-10 07:24:24.793052 | controller | - Zuul: Waiting ... 2026-06-10 07:24:24.793055 | controller | - |_ Config Secrets: OK 2026-06-10 07:24:24.793059 | controller | - |_ Executor: OK 2026-06-10 07:24:24.793062 | controller | - |_ Merger: OK 2026-06-10 07:24:24.793065 | controller | - |_ Scheduler: OK 2026-06-10 07:24:24.793069 | controller | - |_ Web: Waiting ... 2026-06-10 07:24:24.793072 | controller | 2026-06-10 07:24:24.793104 | controller | 2026-06-10T07:24:24-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-10 07:24:29.802045 | controller | 2026-06-10T07:24:29-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:24:29.803433 | controller | 2026-06-10T07:24:29-04:00 INFO Nothing to clean up. 2026-06-10 07:24:31.220452 | controller | 2026-06-10T07:24:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:24:31.220870 | controller | 2026-06-10T07:24:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:24:31.235433 | controller | 2026-06-10T07:24:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:24:31.235476 | controller | 2026-06-10T07:24:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:24:31.630485 | controller | 2026-06-10T07:24:31-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:24:32.012781 | controller | 2026-06-10T07:24:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:24:32.012826 | controller | 2026-06-10T07:24:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:24:32.217917 | controller | 2026-06-10T07:24:32-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:32.600756 | controller | 2026-06-10T07:24:32-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:35.031451 | controller | 2026-06-10T07:24:35-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:24:35.031514 | controller | 2026-06-10T07:24:35-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:24:35.410692 | controller | 2026-06-10T07:24:35-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:24:35.410786 | controller | 2026-06-10T07:24:35-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:24:36.216970 | controller | 2026-06-10T07:24:36-04:00 INFO 2026-06-10 07:24:36.217009 | controller | - Config: Waiting ... 2026-06-10 07:24:36.217015 | controller | - Gateway: OK 2026-06-10 07:24:36.217019 | controller | - GitServer: OK 2026-06-10 07:24:36.217023 | controller | - HoundSearch: OK 2026-06-10 07:24:36.217026 | controller | - LogJuicer: OK 2026-06-10 07:24:36.217030 | controller | - Logserver: OK 2026-06-10 07:24:36.217033 | controller | - MariaDB: OK 2026-06-10 07:24:36.217037 | controller | - NodePoolBuilder: OK 2026-06-10 07:24:36.217041 | controller | - NodePoolLauncher: OK 2026-06-10 07:24:36.217045 | controller | - Zookeeper: OK 2026-06-10 07:24:36.217049 | controller | - Zuul: Waiting ... 2026-06-10 07:24:36.217053 | controller | - |_ Config Secrets: OK 2026-06-10 07:24:36.217057 | controller | - |_ Executor: OK 2026-06-10 07:24:36.217061 | controller | - |_ Merger: OK 2026-06-10 07:24:36.217064 | controller | - |_ Scheduler: OK 2026-06-10 07:24:36.217068 | controller | - |_ Web: Waiting ... 2026-06-10 07:24:36.217073 | controller | 2026-06-10 07:24:36.217102 | controller | 2026-06-10T07:24:36-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-10 07:24:41.225226 | controller | 2026-06-10T07:24:41-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:24:41.226870 | controller | 2026-06-10T07:24:41-04:00 INFO Nothing to clean up. 2026-06-10 07:24:42.642553 | controller | 2026-06-10T07:24:42-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:24:42.642607 | controller | 2026-06-10T07:24:42-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:24:42.659076 | controller | 2026-06-10T07:24:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:24:42.659148 | controller | 2026-06-10T07:24:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:24:43.057405 | controller | 2026-06-10T07:24:43-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:24:43.437460 | controller | 2026-06-10T07:24:43-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:24:43.437509 | controller | 2026-06-10T07:24:43-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:24:43.643733 | controller | 2026-06-10T07:24:43-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:44.024692 | controller | 2026-06-10T07:24:44-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:46.449096 | controller | 2026-06-10T07:24:46-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:24:46.449159 | controller | 2026-06-10T07:24:46-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:24:46.833879 | controller | 2026-06-10T07:24:46-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:24:46.833945 | controller | 2026-06-10T07:24:46-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:24:47.642715 | controller | 2026-06-10T07:24:47-04:00 INFO 2026-06-10 07:24:47.642764 | controller | - Config: Waiting ... 2026-06-10 07:24:47.642775 | controller | - Gateway: OK 2026-06-10 07:24:47.642783 | controller | - GitServer: OK 2026-06-10 07:24:47.642791 | controller | - HoundSearch: OK 2026-06-10 07:24:47.642799 | controller | - LogJuicer: OK 2026-06-10 07:24:47.642807 | controller | - Logserver: OK 2026-06-10 07:24:47.642815 | controller | - MariaDB: OK 2026-06-10 07:24:47.642823 | controller | - NodePoolBuilder: OK 2026-06-10 07:24:47.642831 | controller | - NodePoolLauncher: OK 2026-06-10 07:24:47.642838 | controller | - Zookeeper: OK 2026-06-10 07:24:47.642846 | controller | - Zuul: Waiting ... 2026-06-10 07:24:47.642854 | controller | - |_ Config Secrets: OK 2026-06-10 07:24:47.642862 | controller | - |_ Executor: OK 2026-06-10 07:24:47.642870 | controller | - |_ Merger: OK 2026-06-10 07:24:47.642877 | controller | - |_ Scheduler: OK 2026-06-10 07:24:47.642885 | controller | - |_ Web: Waiting ... 2026-06-10 07:24:47.642893 | controller | 2026-06-10 07:24:47.642957 | controller | 2026-06-10T07:24:47-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-10 07:24:52.648504 | controller | 2026-06-10T07:24:52-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:24:52.651309 | controller | 2026-06-10T07:24:52-04:00 INFO Nothing to clean up. 2026-06-10 07:24:54.067082 | controller | 2026-06-10T07:24:54-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:24:54.067227 | controller | 2026-06-10T07:24:54-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:24:54.089921 | controller | 2026-06-10T07:24:54-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:24:54.090000 | controller | 2026-06-10T07:24:54-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:24:54.485624 | controller | 2026-06-10T07:24:54-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:24:54.869809 | controller | 2026-06-10T07:24:54-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:24:54.869887 | controller | 2026-06-10T07:24:54-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:24:55.071736 | controller | 2026-06-10T07:24:55-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:55.447365 | controller | 2026-06-10T07:24:55-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 07:24:57.883087 | controller | 2026-06-10T07:24:57-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:24:57.883157 | controller | 2026-06-10T07:24:57-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:24:58.256471 | controller | 2026-06-10T07:24:58-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:24:58.256541 | controller | 2026-06-10T07:24:58-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:24:59.462596 | controller | 2026-06-10T07:24:59-04:00 INFO 2026-06-10 07:24:59.462647 | controller | - Config: OK 2026-06-10 07:24:59.462654 | controller | - Gateway: OK 2026-06-10 07:24:59.462659 | controller | - GitServer: OK 2026-06-10 07:24:59.462664 | controller | - HoundSearch: OK 2026-06-10 07:24:59.462670 | controller | - LogJuicer: OK 2026-06-10 07:24:59.462675 | controller | - Logserver: OK 2026-06-10 07:24:59.462679 | controller | - MariaDB: OK 2026-06-10 07:24:59.462684 | controller | - NodePoolBuilder: OK 2026-06-10 07:24:59.462689 | controller | - NodePoolLauncher: OK 2026-06-10 07:24:59.462694 | controller | - Zookeeper: OK 2026-06-10 07:24:59.462699 | controller | - Zuul: OK 2026-06-10 07:24:59.462704 | controller | 2026-06-10 07:24:59.462710 | controller | 2026-06-10T07:24:59-04:00 INFO Updating controller configmap ... 2026-06-10 07:24:59.854696 | controller | 2026-06-10T07:24:59-04:00 INFO Standalone reconcile done. 2026-06-10 11:25:00.337914 | controller | ok: Runtime: 0:03:11.538099 2026-06-10 11:25:00.352735 | 2026-06-10 11:25:00.352902 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:25:00.897663 | controller | skipping: Conditional result was False 2026-06-10 11:25:00.922823 | 2026-06-10 11:25:00.922977 | TASK [health-check/zuul-connections : Wait for the new Zuul connections to appear in the Zuul API] 2026-06-10 11:25:01.407015 | controller | ok: OK (1399 bytes) 2026-06-10 11:25:01.419178 | 2026-06-10 11:25:01.419318 | TASK [health-check/zuul-connections : Ensure the new Zuul connections (non SourceInterface) exist in the scheduler's zuul.conf] 2026-06-10 07:25:01.783733 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 07:25:01.882330 | controller | [connection dummy-elasticsearch-conn] 2026-06-10 07:25:01.950802 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 07:25:02.033422 | controller | [connection dummy-smtp-conn] 2026-06-10 11:25:02.464673 | controller | ok: Runtime: 0:00:00.313959 2026-06-10 11:25:02.468972 | 2026-06-10 11:25:02.469034 | TASK [health-check/zuul-connections : Ensure ElasticSearch URI is configured with basic auth] 2026-06-10 07:25:02.894898 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 07:25:02.994285 | controller | uri = http://es-username:es-password@test:9200 2026-06-10 11:25:03.498142 | controller | ok: Runtime: 0:00:00.196984 2026-06-10 11:25:03.506833 | 2026-06-10 11:25:03.506962 | TASK [health-check/zuul-connections : Ensure SMTP password is present] 2026-06-10 07:25:04.171776 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 07:25:04.258989 | controller | password = smtp-password 2026-06-10 11:25:04.553629 | controller | ok: Runtime: 0:00:00.157113 2026-06-10 11:25:04.562437 | 2026-06-10 11:25:04.562570 | TASK [health-check/zuul-connections : Ensure the new Zuul dummy gerrit secret exist in the scheduler's zuul.conf] 2026-06-10 07:25:05.099462 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 07:25:05.195078 | controller | sshkey = /var/lib/zuul-dummygerritsecret/..data/priv 2026-06-10 07:25:05.195200 | controller | sshkey = /var/lib/zuul-dummygerritsecret/..data/priv 2026-06-10 07:25:05.294276 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 11:25:05.673846 | controller | ok: Runtime: 0:00:00.359694 2026-06-10 11:25:05.683259 | 2026-06-10 11:25:05.683430 | TASK [Delete the dummy Zuul connections] 2026-06-10 11:25:05.719542 | controller | ok 2026-06-10 11:25:05.754937 | 2026-06-10 11:25:05.755026 | TASK [update-custom-resource : read current crd] 2026-06-10 11:25:06.184373 | controller | ok 2026-06-10 11:25:06.193041 | 2026-06-10 11:25:06.193175 | TASK [update-custom-resource : merge the new cr] 2026-06-10 11:25:07.321998 | controller | changed 2026-06-10 11:25:07.331324 | 2026-06-10 11:25:07.331479 | TASK [run the operator in standalone mode] 2026-06-10 11:25:07.356106 | controller | ok 2026-06-10 11:25:07.387608 | 2026-06-10 11:25:07.387716 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:25:07.412429 | controller | skipping: Conditional result was False 2026-06-10 11:25:07.420650 | 2026-06-10 11:25:07.420744 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:25:07.436326 | controller | skipping: Conditional result was False 2026-06-10 11:25:07.444681 | 2026-06-10 11:25:07.444776 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:25:07.956283 | controller | 2026-06-10T07:25:07-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:25:07.956360 | controller | 2026-06-10 07:25:07.967228 | controller | 2026-06-10T07:25:07-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:25:07.967279 | controller | 2026-06-10 07:25:07.980501 | controller | 2026-06-10T07:25:07-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:25:07.981647 | controller | 2026-06-10T07:25:07-04:00 INFO Nothing to clean up. 2026-06-10 07:25:07.988079 | controller | 2026-06-10T07:25:07-04:00 INFO Updating configmap, name: git-server-pi-config-map 2026-06-10 07:25:07.988139 | controller | 2026-06-10T07:25:07-04:00 INFO Updating object, name: git-server-pi-config-map 2026-06-10 07:25:07.996108 | controller | 2026-06-10T07:25:07-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 07:25:07.996192 | controller | 2026-06-10T07:25:07-04:00 INFO Updating object, name: git-server 2026-06-10 07:25:08.596596 | controller | 2026-06-10T07:25:08-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:25:08.676593 | controller | 2026-06-10T07:25:08-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:25:08.686369 | controller | 2026-06-10T07:25:08-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:25:08.686447 | controller | 2026-06-10T07:25:08-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:25:08.703569 | controller | 2026-06-10T07:25:08-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:25:08.703622 | controller | 2026-06-10T07:25:08-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:25:08.995507 | controller | 2026-06-10T07:25:08-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 07:25:08.995556 | controller | 2026-06-10T07:25:08-04:00 INFO 2026-06-10 07:25:08.995562 | controller | - Config: Waiting ... 2026-06-10 07:25:08.995566 | controller | - Gateway: OK 2026-06-10 07:25:08.995571 | controller | - GitServer: Waiting ... 2026-06-10 07:25:08.995575 | controller | - HoundSearch: OK 2026-06-10 07:25:08.995578 | controller | - LogJuicer: OK 2026-06-10 07:25:08.995582 | controller | - Logserver: OK 2026-06-10 07:25:08.995586 | controller | - MariaDB: OK 2026-06-10 07:25:08.995589 | controller | - Zookeeper: OK 2026-06-10 07:25:08.995593 | controller | 2026-06-10 07:25:08.995716 | controller | 2026-06-10T07:25:08-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:25:14.000326 | controller | 2026-06-10T07:25:14-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:25:14.001803 | controller | 2026-06-10T07:25:14-04:00 INFO Nothing to clean up. 2026-06-10 07:25:14.021896 | controller | 2026-06-10T07:25:14-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 07:25:32.138489 | controller | 2026-06-10T07:25:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:25:32.138575 | controller | 2026-06-10T07:25:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:25:32.155189 | controller | 2026-06-10T07:25:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:25:32.155246 | controller | 2026-06-10T07:25:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:25:32.282880 | controller | 2026-06-10T07:25:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:25:32.498534 | controller | 2026-06-10T07:25:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:25:32.498597 | controller | 2026-06-10T07:25:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:25:33.484953 | controller | 2026-06-10T07:25:33-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:25:33.485014 | controller | 2026-06-10T07:25:33-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:25:33.895809 | controller | 2026-06-10T07:25:33-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:25:33.895884 | controller | 2026-06-10T07:25:33-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:25:34.695435 | controller | 2026-06-10T07:25:34-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 07:25:34.695491 | controller | 2026-06-10T07:25:34-04:00 INFO Updating object, name: zuul-executor 2026-06-10 07:25:35.498062 | controller | 2026-06-10T07:25:35-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:25:35.498187 | controller | 2026-06-10T07:25:35-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:25:36.298090 | controller | 2026-06-10T07:25:36-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-10 07:25:36.298159 | controller | 2026-06-10T07:25:36-04:00 INFO Updating object, name: zuul-merger 2026-06-10 07:25:36.304225 | controller | 2026-06-10T07:25:36-04:00 INFO 2026-06-10 07:25:36.304272 | controller | - Config: Waiting ... 2026-06-10 07:25:36.304282 | controller | - Gateway: OK 2026-06-10 07:25:36.304290 | controller | - GitServer: OK 2026-06-10 07:25:36.304297 | controller | - HoundSearch: OK 2026-06-10 07:25:36.304304 | controller | - LogJuicer: OK 2026-06-10 07:25:36.304313 | controller | - Logserver: OK 2026-06-10 07:25:36.304320 | controller | - MariaDB: OK 2026-06-10 07:25:36.304327 | controller | - NodePoolBuilder: OK 2026-06-10 07:25:36.304334 | controller | - NodePoolLauncher: OK 2026-06-10 07:25:36.304341 | controller | - Zookeeper: OK 2026-06-10 07:25:36.304348 | controller | - Zuul: Waiting ... 2026-06-10 07:25:36.304355 | controller | - |_ Config Secrets: OK 2026-06-10 07:25:36.304362 | controller | - |_ Executor: Waiting ... 2026-06-10 07:25:36.304369 | controller | - |_ Merger: Waiting ... 2026-06-10 07:25:36.304376 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:25:36.304383 | controller | - |_ Web: Waiting ... 2026-06-10 07:25:36.304391 | controller | 2026-06-10 07:25:36.304405 | controller | 2026-06-10T07:25:36-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:25:41.312328 | controller | 2026-06-10T07:25:41-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:25:41.313788 | controller | 2026-06-10T07:25:41-04:00 INFO Nothing to clean up. 2026-06-10 07:25:41.935465 | controller | 2026-06-10T07:25:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:25:41.935511 | controller | 2026-06-10T07:25:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:25:41.972865 | controller | 2026-06-10T07:25:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:25:41.972903 | controller | 2026-06-10T07:25:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:25:42.332930 | controller | 2026-06-10T07:25:42-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:25:42.728092 | controller | 2026-06-10T07:25:42-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:25:42.728135 | controller | 2026-06-10T07:25:42-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:25:43.733846 | controller | 2026-06-10T07:25:43-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:26:13.843241 | controller | 2026-06-10T07:26:13-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:26:13.874714 | controller | 2026-06-10T07:26:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:26:13.874785 | controller | 2026-06-10T07:26:13-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:26:14.258598 | controller | 2026-06-10T07:26:14-04:00 INFO 2026-06-10 07:26:14.258712 | controller | - Config: Waiting ... 2026-06-10 07:26:14.258721 | controller | - Gateway: OK 2026-06-10 07:26:14.258728 | controller | - GitServer: OK 2026-06-10 07:26:14.258734 | controller | - HoundSearch: OK 2026-06-10 07:26:14.258739 | controller | - LogJuicer: OK 2026-06-10 07:26:14.258745 | controller | - Logserver: OK 2026-06-10 07:26:14.258750 | controller | - MariaDB: OK 2026-06-10 07:26:14.258755 | controller | - NodePoolBuilder: OK 2026-06-10 07:26:14.258761 | controller | - NodePoolLauncher: OK 2026-06-10 07:26:14.258787 | controller | - Zookeeper: OK 2026-06-10 07:26:14.258793 | controller | - Zuul: Waiting ... 2026-06-10 07:26:14.258799 | controller | - |_ Config Secrets: OK 2026-06-10 07:26:14.258804 | controller | - |_ Executor: OK 2026-06-10 07:26:14.258809 | controller | - |_ Merger: OK 2026-06-10 07:26:14.258815 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:26:14.258820 | controller | - |_ Web: Waiting ... 2026-06-10 07:26:14.258826 | controller | 2026-06-10 07:26:14.258833 | controller | 2026-06-10T07:26:14-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:26:19.263358 | controller | 2026-06-10T07:26:19-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:26:19.266568 | controller | 2026-06-10T07:26:19-04:00 INFO Nothing to clean up. 2026-06-10 07:26:19.883797 | controller | 2026-06-10T07:26:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:26:19.883850 | controller | 2026-06-10T07:26:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:26:19.906707 | controller | 2026-06-10T07:26:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:26:19.906784 | controller | 2026-06-10T07:26:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:26:20.289440 | controller | 2026-06-10T07:26:20-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:26:20.680750 | controller | 2026-06-10T07:26:20-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:26:20.680832 | controller | 2026-06-10T07:26:20-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:26:21.690789 | controller | 2026-06-10T07:26:21-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:26:39.773480 | controller | 2026-06-10T07:26:39-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-10 07:26:39.773617 | controller | 2026-06-10T07:26:39-04:00 INFO Updating object, name: zuul-weeder 2026-06-10 07:26:39.832662 | controller | 2026-06-10T07:26:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:26:39.832774 | controller | 2026-06-10T07:26:39-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:26:40.196431 | controller | 2026-06-10T07:26:40-04:00 INFO 2026-06-10 07:26:40.196545 | controller | - Config: Waiting ... 2026-06-10 07:26:40.196553 | controller | - Gateway: OK 2026-06-10 07:26:40.196559 | controller | - GitServer: OK 2026-06-10 07:26:40.196564 | controller | - HoundSearch: OK 2026-06-10 07:26:40.196569 | controller | - LogJuicer: OK 2026-06-10 07:26:40.196574 | controller | - Logserver: OK 2026-06-10 07:26:40.196579 | controller | - MariaDB: OK 2026-06-10 07:26:40.196584 | controller | - NodePoolBuilder: OK 2026-06-10 07:26:40.196589 | controller | - NodePoolLauncher: OK 2026-06-10 07:26:40.196594 | controller | - Zookeeper: OK 2026-06-10 07:26:40.196599 | controller | - Zuul: Waiting ... 2026-06-10 07:26:40.196604 | controller | - |_ Config Secrets: OK 2026-06-10 07:26:40.196609 | controller | - |_ Executor: OK 2026-06-10 07:26:40.196614 | controller | - |_ Merger: OK 2026-06-10 07:26:40.196619 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:26:40.196624 | controller | - |_ Web: OK 2026-06-10 07:26:40.196629 | controller | 2026-06-10 07:26:40.196645 | controller | 2026-06-10T07:26:40-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 07:26:45.204001 | controller | 2026-06-10T07:26:45-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:26:45.206250 | controller | 2026-06-10T07:26:45-04:00 INFO Nothing to clean up. 2026-06-10 07:26:45.823682 | controller | 2026-06-10T07:26:45-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:26:45.823746 | controller | 2026-06-10T07:26:45-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:26:45.843744 | controller | 2026-06-10T07:26:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:26:45.843812 | controller | 2026-06-10T07:26:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:26:46.226194 | controller | 2026-06-10T07:26:46-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:26:46.614629 | controller | 2026-06-10T07:26:46-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:26:46.614688 | controller | 2026-06-10T07:26:46-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:26:47.636232 | controller | 2026-06-10T07:26:47-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:26:47.636312 | controller | 2026-06-10T07:26:47-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:26:49.206641 | controller | 2026-06-10T07:26:49-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:26:49.206712 | controller | 2026-06-10T07:26:49-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:26:50.416341 | controller | 2026-06-10T07:26:50-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 07:26:50.603919 | controller | 2026-06-10T07:26:50-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 07:26:56.348307 | controller | 2026-06-10T07:26:56-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:26:56.352664 | controller | 2026-06-10T07:26:56-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-10 07:26:56.356691 | controller | 2026-06-10T07:26:56-04:00 INFO 2026-06-10 07:26:56.356716 | controller | - Config: Waiting ... 2026-06-10 07:26:56.356722 | controller | - Gateway: OK 2026-06-10 07:26:56.356727 | controller | - GitServer: OK 2026-06-10 07:26:56.356732 | controller | - HoundSearch: OK 2026-06-10 07:26:56.356737 | controller | - LogJuicer: OK 2026-06-10 07:26:56.356742 | controller | - Logserver: OK 2026-06-10 07:26:56.356753 | controller | - MariaDB: OK 2026-06-10 07:26:56.356758 | controller | - NodePoolBuilder: OK 2026-06-10 07:26:56.356763 | controller | - NodePoolLauncher: OK 2026-06-10 07:26:56.356768 | controller | - Zookeeper: OK 2026-06-10 07:26:56.356774 | controller | - Zuul: OK 2026-06-10 07:26:56.356779 | controller | 2026-06-10 07:26:56.356813 | controller | 2026-06-10T07:26:56-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 07:27:01.367555 | controller | 2026-06-10T07:27:01-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:27:01.369378 | controller | 2026-06-10T07:27:01-04:00 INFO Nothing to clean up. 2026-06-10 07:27:01.981951 | controller | 2026-06-10T07:27:01-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:27:01.982039 | controller | 2026-06-10T07:27:01-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:27:02.001264 | controller | 2026-06-10T07:27:02-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:27:02.001302 | controller | 2026-06-10T07:27:02-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:27:02.383104 | controller | 2026-06-10T07:27:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:27:02.780057 | controller | 2026-06-10T07:27:02-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:27:02.780243 | controller | 2026-06-10T07:27:02-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:27:03.798834 | controller | 2026-06-10T07:27:03-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:27:03.798924 | controller | 2026-06-10T07:27:03-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:27:05.370315 | controller | 2026-06-10T07:27:05-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:27:06.179202 | controller | 2026-06-10T07:27:06-04:00 INFO 2026-06-10 07:27:06.179276 | controller | - Config: Waiting ... 2026-06-10 07:27:06.179287 | controller | - Gateway: OK 2026-06-10 07:27:06.179296 | controller | - GitServer: OK 2026-06-10 07:27:06.179304 | controller | - HoundSearch: OK 2026-06-10 07:27:06.179312 | controller | - LogJuicer: OK 2026-06-10 07:27:06.179320 | controller | - Logserver: OK 2026-06-10 07:27:06.179327 | controller | - MariaDB: OK 2026-06-10 07:27:06.179335 | controller | - NodePoolBuilder: OK 2026-06-10 07:27:06.179343 | controller | - NodePoolLauncher: OK 2026-06-10 07:27:06.179350 | controller | - Zookeeper: OK 2026-06-10 07:27:06.179358 | controller | - Zuul: Waiting ... 2026-06-10 07:27:06.179366 | controller | - |_ Config Secrets: OK 2026-06-10 07:27:06.179374 | controller | - |_ Executor: OK 2026-06-10 07:27:06.179382 | controller | - |_ Merger: OK 2026-06-10 07:27:06.179389 | controller | - |_ Scheduler: OK 2026-06-10 07:27:06.179397 | controller | - |_ Web: Waiting ... 2026-06-10 07:27:06.179405 | controller | 2026-06-10 07:27:06.179420 | controller | 2026-06-10T07:27:06-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 07:27:11.186428 | controller | 2026-06-10T07:27:11-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:27:11.188514 | controller | 2026-06-10T07:27:11-04:00 INFO Nothing to clean up. 2026-06-10 07:27:11.806321 | controller | 2026-06-10T07:27:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:27:11.806418 | controller | 2026-06-10T07:27:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:27:11.831995 | controller | 2026-06-10T07:27:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:27:11.832074 | controller | 2026-06-10T07:27:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:27:12.204755 | controller | 2026-06-10T07:27:12-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:27:12.597533 | controller | 2026-06-10T07:27:12-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:27:12.597574 | controller | 2026-06-10T07:27:12-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:27:13.617082 | controller | 2026-06-10T07:27:13-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:27:13.617219 | controller | 2026-06-10T07:27:13-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:27:15.191382 | controller | 2026-06-10T07:27:15-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:27:15.191456 | controller | 2026-06-10T07:27:15-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:27:16.010002 | controller | 2026-06-10T07:27:16-04:00 INFO 2026-06-10 07:27:16.010104 | controller | - Config: Waiting ... 2026-06-10 07:27:16.010120 | controller | - Gateway: OK 2026-06-10 07:27:16.010148 | controller | - GitServer: OK 2026-06-10 07:27:16.010161 | controller | - HoundSearch: OK 2026-06-10 07:27:16.010200 | controller | - LogJuicer: OK 2026-06-10 07:27:16.010208 | controller | - Logserver: OK 2026-06-10 07:27:16.010216 | controller | - MariaDB: OK 2026-06-10 07:27:16.010224 | controller | - NodePoolBuilder: OK 2026-06-10 07:27:16.010231 | controller | - NodePoolLauncher: OK 2026-06-10 07:27:16.010239 | controller | - Zookeeper: OK 2026-06-10 07:27:16.010247 | controller | - Zuul: Waiting ... 2026-06-10 07:27:16.010255 | controller | - |_ Config Secrets: OK 2026-06-10 07:27:16.010263 | controller | - |_ Executor: OK 2026-06-10 07:27:16.010271 | controller | - |_ Merger: OK 2026-06-10 07:27:16.010278 | controller | - |_ Scheduler: OK 2026-06-10 07:27:16.010286 | controller | - |_ Web: Waiting ... 2026-06-10 07:27:16.010294 | controller | 2026-06-10 07:27:16.010312 | controller | 2026-06-10T07:27:16-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-10 07:27:21.015223 | controller | 2026-06-10T07:27:21-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:27:21.017944 | controller | 2026-06-10T07:27:21-04:00 INFO Nothing to clean up. 2026-06-10 07:27:21.630808 | controller | 2026-06-10T07:27:21-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:27:21.630886 | controller | 2026-06-10T07:27:21-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:27:21.650034 | controller | 2026-06-10T07:27:21-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:27:21.650115 | controller | 2026-06-10T07:27:21-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:27:22.037354 | controller | 2026-06-10T07:27:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:27:22.432822 | controller | 2026-06-10T07:27:22-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:27:22.432898 | controller | 2026-06-10T07:27:22-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:27:23.437661 | controller | 2026-06-10T07:27:23-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:27:23.437701 | controller | 2026-06-10T07:27:23-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:27:25.018422 | controller | 2026-06-10T07:27:25-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:27:25.018464 | controller | 2026-06-10T07:27:25-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:27:25.824845 | controller | 2026-06-10T07:27:25-04:00 INFO 2026-06-10 07:27:25.824896 | controller | - Config: Waiting ... 2026-06-10 07:27:25.824904 | controller | - Gateway: OK 2026-06-10 07:27:25.824910 | controller | - GitServer: OK 2026-06-10 07:27:25.824915 | controller | - HoundSearch: OK 2026-06-10 07:27:25.824921 | controller | - LogJuicer: OK 2026-06-10 07:27:25.824926 | controller | - Logserver: OK 2026-06-10 07:27:25.824931 | controller | - MariaDB: OK 2026-06-10 07:27:25.824936 | controller | - NodePoolBuilder: OK 2026-06-10 07:27:25.824941 | controller | - NodePoolLauncher: OK 2026-06-10 07:27:25.824947 | controller | - Zookeeper: OK 2026-06-10 07:27:25.824952 | controller | - Zuul: Waiting ... 2026-06-10 07:27:25.824957 | controller | - |_ Config Secrets: OK 2026-06-10 07:27:25.824962 | controller | - |_ Executor: OK 2026-06-10 07:27:25.824968 | controller | - |_ Merger: OK 2026-06-10 07:27:25.824973 | controller | - |_ Scheduler: OK 2026-06-10 07:27:25.824978 | controller | - |_ Web: Waiting ... 2026-06-10 07:27:25.824983 | controller | 2026-06-10 07:27:25.824994 | controller | 2026-06-10T07:27:25-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-10 07:27:30.832628 | controller | 2026-06-10T07:27:30-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:27:30.834159 | controller | 2026-06-10T07:27:30-04:00 INFO Nothing to clean up. 2026-06-10 07:27:31.458336 | controller | 2026-06-10T07:27:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:27:31.458467 | controller | 2026-06-10T07:27:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:27:31.484815 | controller | 2026-06-10T07:27:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:27:31.484876 | controller | 2026-06-10T07:27:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:27:31.869654 | controller | 2026-06-10T07:27:31-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:27:32.253886 | controller | 2026-06-10T07:27:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:27:32.253950 | controller | 2026-06-10T07:27:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:27:33.265868 | controller | 2026-06-10T07:27:33-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:27:33.266225 | controller | 2026-06-10T07:27:33-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:27:34.838597 | controller | 2026-06-10T07:27:34-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:27:34.838636 | controller | 2026-06-10T07:27:34-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:27:35.644457 | controller | 2026-06-10T07:27:35-04:00 INFO 2026-06-10 07:27:35.644559 | controller | - Config: Waiting ... 2026-06-10 07:27:35.644565 | controller | - Gateway: OK 2026-06-10 07:27:35.644569 | controller | - GitServer: OK 2026-06-10 07:27:35.644573 | controller | - HoundSearch: OK 2026-06-10 07:27:35.644577 | controller | - LogJuicer: OK 2026-06-10 07:27:35.644580 | controller | - Logserver: OK 2026-06-10 07:27:35.644584 | controller | - MariaDB: OK 2026-06-10 07:27:35.644587 | controller | - NodePoolBuilder: OK 2026-06-10 07:27:35.644591 | controller | - NodePoolLauncher: OK 2026-06-10 07:27:35.644594 | controller | - Zookeeper: OK 2026-06-10 07:27:35.644598 | controller | - Zuul: Waiting ... 2026-06-10 07:27:35.644602 | controller | - |_ Config Secrets: OK 2026-06-10 07:27:35.644605 | controller | - |_ Executor: OK 2026-06-10 07:27:35.644609 | controller | - |_ Merger: OK 2026-06-10 07:27:35.644612 | controller | - |_ Scheduler: OK 2026-06-10 07:27:35.644616 | controller | - |_ Web: Waiting ... 2026-06-10 07:27:35.644620 | controller | 2026-06-10 07:27:35.644632 | controller | 2026-06-10T07:27:35-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-10 07:27:40.650913 | controller | 2026-06-10T07:27:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:27:40.654427 | controller | 2026-06-10T07:27:40-04:00 INFO Nothing to clean up. 2026-06-10 07:27:41.265711 | controller | 2026-06-10T07:27:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:27:41.265750 | controller | 2026-06-10T07:27:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:27:41.281550 | controller | 2026-06-10T07:27:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:27:41.281596 | controller | 2026-06-10T07:27:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:27:41.679920 | controller | 2026-06-10T07:27:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:27:42.068270 | controller | 2026-06-10T07:27:42-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:27:42.068322 | controller | 2026-06-10T07:27:42-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:27:43.088086 | controller | 2026-06-10T07:27:43-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:27:43.088126 | controller | 2026-06-10T07:27:43-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:27:44.655023 | controller | 2026-06-10T07:27:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:27:44.655066 | controller | 2026-06-10T07:27:44-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:27:45.460544 | controller | 2026-06-10T07:27:45-04:00 INFO 2026-06-10 07:27:45.460616 | controller | - Config: Waiting ... 2026-06-10 07:27:45.460627 | controller | - Gateway: OK 2026-06-10 07:27:45.460636 | controller | - GitServer: OK 2026-06-10 07:27:45.460644 | controller | - HoundSearch: OK 2026-06-10 07:27:45.460652 | controller | - LogJuicer: OK 2026-06-10 07:27:45.460660 | controller | - Logserver: OK 2026-06-10 07:27:45.460667 | controller | - MariaDB: OK 2026-06-10 07:27:45.460675 | controller | - NodePoolBuilder: OK 2026-06-10 07:27:45.460683 | controller | - NodePoolLauncher: OK 2026-06-10 07:27:45.460691 | controller | - Zookeeper: OK 2026-06-10 07:27:45.460698 | controller | - Zuul: Waiting ... 2026-06-10 07:27:45.460706 | controller | - |_ Config Secrets: OK 2026-06-10 07:27:45.460714 | controller | - |_ Executor: OK 2026-06-10 07:27:45.460722 | controller | - |_ Merger: OK 2026-06-10 07:27:45.460730 | controller | - |_ Scheduler: OK 2026-06-10 07:27:45.460737 | controller | - |_ Web: Waiting ... 2026-06-10 07:27:45.460745 | controller | 2026-06-10 07:27:45.460761 | controller | 2026-06-10T07:27:45-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-10 07:27:50.469394 | controller | 2026-06-10T07:27:50-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:27:50.470948 | controller | 2026-06-10T07:27:50-04:00 INFO Nothing to clean up. 2026-06-10 07:27:51.084185 | controller | 2026-06-10T07:27:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:27:51.084230 | controller | 2026-06-10T07:27:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:27:51.104559 | controller | 2026-06-10T07:27:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:27:51.104606 | controller | 2026-06-10T07:27:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:27:51.491034 | controller | 2026-06-10T07:27:51-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:27:51.884330 | controller | 2026-06-10T07:27:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:27:51.884390 | controller | 2026-06-10T07:27:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:27:52.899140 | controller | 2026-06-10T07:27:52-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:27:52.899235 | controller | 2026-06-10T07:27:52-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:27:54.474714 | controller | 2026-06-10T07:27:54-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:27:54.474844 | controller | 2026-06-10T07:27:54-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:27:55.279525 | controller | 2026-06-10T07:27:55-04:00 INFO 2026-06-10 07:27:55.279590 | controller | - Config: Waiting ... 2026-06-10 07:27:55.279607 | controller | - Gateway: OK 2026-06-10 07:27:55.279621 | controller | - GitServer: OK 2026-06-10 07:27:55.279634 | controller | - HoundSearch: OK 2026-06-10 07:27:55.279647 | controller | - LogJuicer: OK 2026-06-10 07:27:55.279660 | controller | - Logserver: OK 2026-06-10 07:27:55.279672 | controller | - MariaDB: OK 2026-06-10 07:27:55.279684 | controller | - NodePoolBuilder: OK 2026-06-10 07:27:55.279697 | controller | - NodePoolLauncher: OK 2026-06-10 07:27:55.279710 | controller | - Zookeeper: OK 2026-06-10 07:27:55.279723 | controller | - Zuul: Waiting ... 2026-06-10 07:27:55.279736 | controller | - |_ Config Secrets: OK 2026-06-10 07:27:55.279749 | controller | - |_ Executor: OK 2026-06-10 07:27:55.279761 | controller | - |_ Merger: OK 2026-06-10 07:27:55.279774 | controller | - |_ Scheduler: OK 2026-06-10 07:27:55.279786 | controller | - |_ Web: Waiting ... 2026-06-10 07:27:55.279799 | controller | 2026-06-10 07:27:55.279889 | controller | 2026-06-10T07:27:55-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-10 07:28:00.286734 | controller | 2026-06-10T07:28:00-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:28:00.288641 | controller | 2026-06-10T07:28:00-04:00 INFO Nothing to clean up. 2026-06-10 07:28:00.901980 | controller | 2026-06-10T07:28:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:28:00.902050 | controller | 2026-06-10T07:28:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:28:00.918504 | controller | 2026-06-10T07:28:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:28:00.918579 | controller | 2026-06-10T07:28:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:28:01.301982 | controller | 2026-06-10T07:28:01-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:28:01.702254 | controller | 2026-06-10T07:28:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:28:01.702321 | controller | 2026-06-10T07:28:01-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:28:02.709637 | controller | 2026-06-10T07:28:02-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:28:02.709697 | controller | 2026-06-10T07:28:02-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:28:04.288460 | controller | 2026-06-10T07:28:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:28:04.288535 | controller | 2026-06-10T07:28:04-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:28:05.099218 | controller | 2026-06-10T07:28:05-04:00 INFO 2026-06-10 07:28:05.099291 | controller | - Config: Waiting ... 2026-06-10 07:28:05.099303 | controller | - Gateway: OK 2026-06-10 07:28:05.099311 | controller | - GitServer: OK 2026-06-10 07:28:05.099319 | controller | - HoundSearch: OK 2026-06-10 07:28:05.099327 | controller | - LogJuicer: OK 2026-06-10 07:28:05.099335 | controller | - Logserver: OK 2026-06-10 07:28:05.099343 | controller | - MariaDB: OK 2026-06-10 07:28:05.099351 | controller | - NodePoolBuilder: OK 2026-06-10 07:28:05.099359 | controller | - NodePoolLauncher: OK 2026-06-10 07:28:05.099366 | controller | - Zookeeper: OK 2026-06-10 07:28:05.099374 | controller | - Zuul: Waiting ... 2026-06-10 07:28:05.099382 | controller | - |_ Config Secrets: OK 2026-06-10 07:28:05.099390 | controller | - |_ Executor: OK 2026-06-10 07:28:05.099398 | controller | - |_ Merger: OK 2026-06-10 07:28:05.099406 | controller | - |_ Scheduler: OK 2026-06-10 07:28:05.099430 | controller | - |_ Web: Waiting ... 2026-06-10 07:28:05.099442 | controller | 2026-06-10 07:28:05.099455 | controller | 2026-06-10T07:28:05-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-10 07:28:10.108018 | controller | 2026-06-10T07:28:10-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:28:10.110211 | controller | 2026-06-10T07:28:10-04:00 INFO Nothing to clean up. 2026-06-10 07:28:10.722413 | controller | 2026-06-10T07:28:10-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:28:10.722553 | controller | 2026-06-10T07:28:10-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:28:10.759206 | controller | 2026-06-10T07:28:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:28:10.759257 | controller | 2026-06-10T07:28:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:28:11.134032 | controller | 2026-06-10T07:28:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:28:11.520527 | controller | 2026-06-10T07:28:11-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:28:11.520588 | controller | 2026-06-10T07:28:11-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:28:12.537481 | controller | 2026-06-10T07:28:12-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:28:12.537525 | controller | 2026-06-10T07:28:12-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:28:14.112792 | controller | 2026-06-10T07:28:14-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:28:14.112864 | controller | 2026-06-10T07:28:14-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:28:15.316389 | controller | 2026-06-10T07:28:15-04:00 INFO 2026-06-10 07:28:15.316506 | controller | - Config: OK 2026-06-10 07:28:15.316513 | controller | - Gateway: OK 2026-06-10 07:28:15.316518 | controller | - GitServer: OK 2026-06-10 07:28:15.316523 | controller | - HoundSearch: OK 2026-06-10 07:28:15.316527 | controller | - LogJuicer: OK 2026-06-10 07:28:15.316531 | controller | - Logserver: OK 2026-06-10 07:28:15.316535 | controller | - MariaDB: OK 2026-06-10 07:28:15.316540 | controller | - NodePoolBuilder: OK 2026-06-10 07:28:15.316544 | controller | - NodePoolLauncher: OK 2026-06-10 07:28:15.316548 | controller | - Zookeeper: OK 2026-06-10 07:28:15.316553 | controller | - Zuul: OK 2026-06-10 07:28:15.316557 | controller | 2026-06-10 07:28:15.316562 | controller | 2026-06-10T07:28:15-04:00 INFO Updating controller configmap ... 2026-06-10 07:28:15.710654 | controller | 2026-06-10T07:28:15-04:00 INFO Standalone reconcile done. 2026-06-10 11:28:15.770791 | controller | ok: Runtime: 0:03:08.032112 2026-06-10 11:28:15.780384 | 2026-06-10 11:28:15.780473 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:28:16.308468 | controller | skipping: Conditional result was False 2026-06-10 11:28:16.334562 | 2026-06-10 11:28:16.334727 | TASK [health-check/zuul-connections : Wait for the dummy Zuul connections to be removed from the API] 2026-06-10 11:28:16.766278 | controller | ok: OK (415 bytes) 2026-06-10 11:28:16.776182 | 2026-06-10 11:28:16.776310 | TASK [health-check/zuul-connections : Ensure the new Zuul connections (non SourceInterface) no longer exist in the scheduler's zuul.conf] 2026-06-10 07:28:17.108285 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 07:28:17.206641 | controller | command terminated with exit code 1 2026-06-10 07:28:17.281754 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 07:28:17.355876 | controller | command terminated with exit code 1 2026-06-10 11:28:17.826903 | controller | ok: Runtime: 0:00:00.323671 2026-06-10 11:28:17.837566 | 2026-06-10 11:28:17.837726 | TASK [health-check/zuul-connections : Delete GitHub Connection Secret] 2026-06-10 11:28:18.604855 | controller | changed 2026-06-10 11:28:18.615738 | 2026-06-10 11:28:18.615909 | TASK [health-check/zuul-connections : Delete GitLab Connection Secret] 2026-06-10 11:28:19.404958 | controller | changed 2026-06-10 11:28:19.416099 | 2026-06-10 11:28:19.416301 | TASK [health-check/zuul-connections : Delete Pagure Connection Secret] 2026-06-10 11:28:20.247869 | controller | changed 2026-06-10 11:28:20.265894 | 2026-06-10 11:28:20.266036 | TASK [health-check/zuul-authenticators : Initialize new authenticator config] 2026-06-10 11:28:20.310996 | controller | ok 2026-06-10 11:28:20.320850 | 2026-06-10 11:28:20.321064 | TASK [Add the authenticator to spec] 2026-06-10 11:28:20.345605 | controller | ok 2026-06-10 11:28:20.369320 | 2026-06-10 11:28:20.369550 | TASK [update-custom-resource : read current crd] 2026-06-10 11:28:20.644814 | controller | ok 2026-06-10 11:28:20.651379 | 2026-06-10 11:28:20.651464 | TASK [update-custom-resource : merge the new cr] 2026-06-10 11:28:21.471154 | controller | changed 2026-06-10 11:28:21.481648 | 2026-06-10 11:28:21.481777 | TASK [run the operator in standalone mode] 2026-06-10 11:28:21.517295 | controller | ok 2026-06-10 11:28:21.547329 | 2026-06-10 11:28:21.547440 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:28:21.573567 | controller | skipping: Conditional result was False 2026-06-10 11:28:21.581435 | 2026-06-10 11:28:21.581529 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:28:21.617544 | controller | skipping: Conditional result was False 2026-06-10 11:28:21.624457 | 2026-06-10 11:28:21.624553 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:28:22.142043 | controller | 2026-06-10T07:28:22-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:28:22.142126 | controller | 2026-06-10 07:28:22.152446 | controller | 2026-06-10T07:28:22-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:28:22.152499 | controller | 2026-06-10 07:28:22.162892 | controller | 2026-06-10T07:28:22-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:28:22.164249 | controller | 2026-06-10T07:28:22-04:00 INFO Nothing to clean up. 2026-06-10 07:28:22.776632 | controller | 2026-06-10T07:28:22-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:28:22.869475 | controller | 2026-06-10T07:28:22-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:28:22.875949 | controller | 2026-06-10T07:28:22-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:28:22.875985 | controller | 2026-06-10T07:28:22-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:28:22.890206 | controller | 2026-06-10T07:28:22-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:28:22.890239 | controller | 2026-06-10T07:28:22-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:28:23.186680 | controller | 2026-06-10T07:28:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:28:23.582295 | controller | 2026-06-10T07:28:23-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:28:23.582393 | controller | 2026-06-10T07:28:23-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:28:24.563232 | controller | 2026-06-10T07:28:24-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:28:24.563297 | controller | 2026-06-10T07:28:24-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:28:24.788913 | controller | 2026-06-10T07:28:24-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:28:24.788988 | controller | 2026-06-10T07:28:24-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:28:26.363050 | controller | 2026-06-10T07:28:26-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:28:26.363109 | controller | 2026-06-10T07:28:26-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:28:27.169273 | controller | 2026-06-10T07:28:27-04:00 INFO 2026-06-10 07:28:27.169346 | controller | - Config: Waiting ... 2026-06-10 07:28:27.169355 | controller | - Gateway: OK 2026-06-10 07:28:27.169363 | controller | - GitServer: OK 2026-06-10 07:28:27.169370 | controller | - HoundSearch: OK 2026-06-10 07:28:27.169376 | controller | - LogJuicer: OK 2026-06-10 07:28:27.169381 | controller | - Logserver: OK 2026-06-10 07:28:27.169386 | controller | - MariaDB: OK 2026-06-10 07:28:27.169391 | controller | - NodePoolBuilder: OK 2026-06-10 07:28:27.169398 | controller | - NodePoolLauncher: OK 2026-06-10 07:28:27.169403 | controller | - Zookeeper: OK 2026-06-10 07:28:27.169408 | controller | - Zuul: Waiting ... 2026-06-10 07:28:27.169414 | controller | - |_ Config Secrets: OK 2026-06-10 07:28:27.169419 | controller | - |_ Executor: OK 2026-06-10 07:28:27.169424 | controller | - |_ Merger: OK 2026-06-10 07:28:27.169429 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:28:27.169435 | controller | - |_ Web: Waiting ... 2026-06-10 07:28:27.169440 | controller | 2026-06-10 07:28:27.169493 | controller | 2026-06-10T07:28:27-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:28:32.174840 | controller | 2026-06-10T07:28:32-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:28:32.177124 | controller | 2026-06-10T07:28:32-04:00 INFO Nothing to clean up. 2026-06-10 07:28:32.789282 | controller | 2026-06-10T07:28:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:28:32.789355 | controller | 2026-06-10T07:28:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:28:32.806428 | controller | 2026-06-10T07:28:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:28:32.806486 | controller | 2026-06-10T07:28:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:28:33.196888 | controller | 2026-06-10T07:28:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:28:33.599322 | controller | 2026-06-10T07:28:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:28:33.599393 | controller | 2026-06-10T07:28:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:28:34.594207 | controller | 2026-06-10T07:28:34-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:29:04.734997 | controller | 2026-06-10T07:29:04-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:29:04.764887 | controller | 2026-06-10T07:29:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:29:04.764944 | controller | 2026-06-10T07:29:04-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:29:05.145081 | controller | 2026-06-10T07:29:05-04:00 INFO 2026-06-10 07:29:05.145151 | controller | - Config: Waiting ... 2026-06-10 07:29:05.145158 | controller | - Gateway: OK 2026-06-10 07:29:05.145196 | controller | - GitServer: OK 2026-06-10 07:29:05.145204 | controller | - HoundSearch: OK 2026-06-10 07:29:05.145209 | controller | - LogJuicer: OK 2026-06-10 07:29:05.145214 | controller | - Logserver: OK 2026-06-10 07:29:05.145219 | controller | - MariaDB: OK 2026-06-10 07:29:05.145223 | controller | - NodePoolBuilder: OK 2026-06-10 07:29:05.145228 | controller | - NodePoolLauncher: OK 2026-06-10 07:29:05.145233 | controller | - Zookeeper: OK 2026-06-10 07:29:05.145238 | controller | - Zuul: Waiting ... 2026-06-10 07:29:05.145243 | controller | - |_ Config Secrets: OK 2026-06-10 07:29:05.145249 | controller | - |_ Executor: OK 2026-06-10 07:29:05.145254 | controller | - |_ Merger: OK 2026-06-10 07:29:05.145260 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:29:05.145265 | controller | - |_ Web: Waiting ... 2026-06-10 07:29:05.145270 | controller | 2026-06-10 07:29:05.145276 | controller | 2026-06-10T07:29:05-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:29:10.151900 | controller | 2026-06-10T07:29:10-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:29:10.154636 | controller | 2026-06-10T07:29:10-04:00 INFO Nothing to clean up. 2026-06-10 07:29:10.766383 | controller | 2026-06-10T07:29:10-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:29:10.766439 | controller | 2026-06-10T07:29:10-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:29:10.785981 | controller | 2026-06-10T07:29:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:29:10.786049 | controller | 2026-06-10T07:29:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:29:11.173728 | controller | 2026-06-10T07:29:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:29:11.569381 | controller | 2026-06-10T07:29:11-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:29:11.569436 | controller | 2026-06-10T07:29:11-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:29:12.574927 | controller | 2026-06-10T07:29:12-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:29:30.663293 | controller | 2026-06-10T07:29:30-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:29:30.663372 | controller | 2026-06-10T07:29:30-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:29:30.696905 | controller | 2026-06-10T07:29:30-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:29:30.696983 | controller | 2026-06-10T07:29:30-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:29:31.480058 | controller | 2026-06-10T07:29:31-04:00 INFO 2026-06-10 07:29:31.480121 | controller | - Config: OK 2026-06-10 07:29:31.480136 | controller | - Gateway: OK 2026-06-10 07:29:31.480149 | controller | - GitServer: OK 2026-06-10 07:29:31.480161 | controller | - HoundSearch: OK 2026-06-10 07:29:31.480235 | controller | - LogJuicer: OK 2026-06-10 07:29:31.480251 | controller | - Logserver: OK 2026-06-10 07:29:31.480264 | controller | - MariaDB: OK 2026-06-10 07:29:31.480276 | controller | - NodePoolBuilder: OK 2026-06-10 07:29:31.480288 | controller | - NodePoolLauncher: OK 2026-06-10 07:29:31.480303 | controller | - Zookeeper: OK 2026-06-10 07:29:31.480316 | controller | - Zuul: OK 2026-06-10 07:29:31.480329 | controller | 2026-06-10 07:29:31.480343 | controller | 2026-06-10T07:29:31-04:00 INFO Updating controller configmap ... 2026-06-10 07:29:31.870442 | controller | 2026-06-10T07:29:31-04:00 INFO Standalone reconcile done. 2026-06-10 11:29:33.056846 | controller | ok: Runtime: 0:01:10.013116 2026-06-10 11:29:33.062202 | 2026-06-10 11:29:33.062268 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:29:33.583518 | controller | skipping: Conditional result was False 2026-06-10 11:29:33.609020 | 2026-06-10 11:29:33.609164 | TASK [health-check/zuul-authenticators : Wait for the new authenticator to appear in the Zuul API] 2026-06-10 11:29:34.217482 | controller | ok: OK (589 bytes) 2026-06-10 11:29:34.222277 | 2026-06-10 11:29:34.222338 | TASK [Remove the test authenticator from the spec] 2026-06-10 11:29:34.248884 | controller | ok 2026-06-10 11:29:34.264504 | 2026-06-10 11:29:34.264566 | TASK [update-custom-resource : read current crd] 2026-06-10 11:29:34.511450 | controller | ok 2026-06-10 11:29:34.522146 | 2026-06-10 11:29:34.522274 | TASK [update-custom-resource : merge the new cr] 2026-06-10 11:29:35.259564 | controller | changed 2026-06-10 11:29:35.269035 | 2026-06-10 11:29:35.269164 | TASK [run the operator in standalone mode] 2026-06-10 11:29:35.304242 | controller | ok 2026-06-10 11:29:35.335729 | 2026-06-10 11:29:35.335864 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:29:35.361624 | controller | skipping: Conditional result was False 2026-06-10 11:29:35.369492 | 2026-06-10 11:29:35.369582 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:29:35.395736 | controller | skipping: Conditional result was False 2026-06-10 11:29:35.408094 | 2026-06-10 11:29:35.408236 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:29:36.055321 | controller | 2026-06-10T07:29:36-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:29:36.055511 | controller | 2026-06-10 07:29:36.065917 | controller | 2026-06-10T07:29:36-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:29:36.065973 | controller | 2026-06-10 07:29:36.081391 | controller | 2026-06-10T07:29:36-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:29:36.083102 | controller | 2026-06-10T07:29:36-04:00 INFO Nothing to clean up. 2026-06-10 07:29:36.699715 | controller | 2026-06-10T07:29:36-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:29:36.795330 | controller | 2026-06-10T07:29:36-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:29:36.802238 | controller | 2026-06-10T07:29:36-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:29:36.802273 | controller | 2026-06-10T07:29:36-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:29:36.820617 | controller | 2026-06-10T07:29:36-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:29:36.820670 | controller | 2026-06-10T07:29:36-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:29:37.110725 | controller | 2026-06-10T07:29:37-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:29:37.501431 | controller | 2026-06-10T07:29:37-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:29:37.501588 | controller | 2026-06-10T07:29:37-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:29:38.482031 | controller | 2026-06-10T07:29:38-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:29:38.482136 | controller | 2026-06-10T07:29:38-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:29:38.705767 | controller | 2026-06-10T07:29:38-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:29:38.705878 | controller | 2026-06-10T07:29:38-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:29:40.281272 | controller | 2026-06-10T07:29:40-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:29:40.281348 | controller | 2026-06-10T07:29:40-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:29:41.102726 | controller | 2026-06-10T07:29:41-04:00 INFO 2026-06-10 07:29:41.102822 | controller | - Config: Waiting ... 2026-06-10 07:29:41.102827 | controller | - Gateway: OK 2026-06-10 07:29:41.102831 | controller | - GitServer: OK 2026-06-10 07:29:41.102835 | controller | - HoundSearch: OK 2026-06-10 07:29:41.102839 | controller | - LogJuicer: OK 2026-06-10 07:29:41.102843 | controller | - Logserver: OK 2026-06-10 07:29:41.102846 | controller | - MariaDB: OK 2026-06-10 07:29:41.102850 | controller | - NodePoolBuilder: OK 2026-06-10 07:29:41.102856 | controller | - NodePoolLauncher: OK 2026-06-10 07:29:41.102860 | controller | - Zookeeper: OK 2026-06-10 07:29:41.102864 | controller | - Zuul: Waiting ... 2026-06-10 07:29:41.102867 | controller | - |_ Config Secrets: OK 2026-06-10 07:29:41.102871 | controller | - |_ Executor: OK 2026-06-10 07:29:41.102875 | controller | - |_ Merger: OK 2026-06-10 07:29:41.102878 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:29:41.102882 | controller | - |_ Web: Waiting ... 2026-06-10 07:29:41.102886 | controller | 2026-06-10 07:29:41.102899 | controller | 2026-06-10T07:29:41-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:29:46.108789 | controller | 2026-06-10T07:29:46-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:29:46.112931 | controller | 2026-06-10T07:29:46-04:00 INFO Nothing to clean up. 2026-06-10 07:29:46.725971 | controller | 2026-06-10T07:29:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:29:46.726079 | controller | 2026-06-10T07:29:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:29:46.745471 | controller | 2026-06-10T07:29:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:29:46.745571 | controller | 2026-06-10T07:29:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:29:47.131653 | controller | 2026-06-10T07:29:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:29:47.529398 | controller | 2026-06-10T07:29:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:29:47.529451 | controller | 2026-06-10T07:29:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:29:48.531231 | controller | 2026-06-10T07:29:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:30:18.641675 | controller | 2026-06-10T07:30:18-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:30:18.671434 | controller | 2026-06-10T07:30:18-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:30:18.671514 | controller | 2026-06-10T07:30:18-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:30:19.061664 | controller | 2026-06-10T07:30:19-04:00 INFO 2026-06-10 07:30:19.061713 | controller | - Config: Waiting ... 2026-06-10 07:30:19.061721 | controller | - Gateway: OK 2026-06-10 07:30:19.061727 | controller | - GitServer: OK 2026-06-10 07:30:19.061732 | controller | - HoundSearch: OK 2026-06-10 07:30:19.061738 | controller | - LogJuicer: OK 2026-06-10 07:30:19.061743 | controller | - Logserver: OK 2026-06-10 07:30:19.061748 | controller | - MariaDB: OK 2026-06-10 07:30:19.061754 | controller | - NodePoolBuilder: OK 2026-06-10 07:30:19.061759 | controller | - NodePoolLauncher: OK 2026-06-10 07:30:19.061765 | controller | - Zookeeper: OK 2026-06-10 07:30:19.061769 | controller | - Zuul: Waiting ... 2026-06-10 07:30:19.061774 | controller | - |_ Config Secrets: OK 2026-06-10 07:30:19.061780 | controller | - |_ Executor: OK 2026-06-10 07:30:19.061786 | controller | - |_ Merger: OK 2026-06-10 07:30:19.061791 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:30:19.061796 | controller | - |_ Web: Waiting ... 2026-06-10 07:30:19.061801 | controller | 2026-06-10 07:30:19.061806 | controller | 2026-06-10T07:30:19-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:30:24.072456 | controller | 2026-06-10T07:30:24-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:30:24.074421 | controller | 2026-06-10T07:30:24-04:00 INFO Nothing to clean up. 2026-06-10 07:30:24.687975 | controller | 2026-06-10T07:30:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:30:24.688023 | controller | 2026-06-10T07:30:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:30:24.705821 | controller | 2026-06-10T07:30:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:30:24.705876 | controller | 2026-06-10T07:30:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:30:25.094915 | controller | 2026-06-10T07:30:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:30:25.483949 | controller | 2026-06-10T07:30:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:30:25.484003 | controller | 2026-06-10T07:30:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:30:26.501886 | controller | 2026-06-10T07:30:26-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:30:42.570147 | controller | 2026-06-10T07:30:42-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:30:42.570235 | controller | 2026-06-10T07:30:42-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:30:42.612129 | controller | 2026-06-10T07:30:42-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:30:42.612304 | controller | 2026-06-10T07:30:42-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:30:43.386850 | controller | 2026-06-10T07:30:43-04:00 INFO 2026-06-10 07:30:43.386922 | controller | - Config: OK 2026-06-10 07:30:43.386934 | controller | - Gateway: OK 2026-06-10 07:30:43.386943 | controller | - GitServer: OK 2026-06-10 07:30:43.386951 | controller | - HoundSearch: OK 2026-06-10 07:30:43.386959 | controller | - LogJuicer: OK 2026-06-10 07:30:43.386967 | controller | - Logserver: OK 2026-06-10 07:30:43.386975 | controller | - MariaDB: OK 2026-06-10 07:30:43.386983 | controller | - NodePoolBuilder: OK 2026-06-10 07:30:43.386991 | controller | - NodePoolLauncher: OK 2026-06-10 07:30:43.387001 | controller | - Zookeeper: OK 2026-06-10 07:30:43.387010 | controller | - Zuul: OK 2026-06-10 07:30:43.387018 | controller | 2026-06-10 07:30:43.387035 | controller | 2026-06-10T07:30:43-04:00 INFO Updating controller configmap ... 2026-06-10 07:30:43.775384 | controller | 2026-06-10T07:30:43-04:00 INFO Standalone reconcile done. 2026-06-10 11:30:44.032148 | controller | ok: Runtime: 0:01:08.099948 2026-06-10 11:30:44.043622 | 2026-06-10 11:30:44.043769 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:30:44.576007 | controller | skipping: Conditional result was False 2026-06-10 11:30:44.601820 | 2026-06-10 11:30:44.601976 | TASK [health-check/zuul-authenticators : Wait for the test authenticator to be delisted] 2026-06-10 11:30:45.156484 | controller | ok: OK (388 bytes) 2026-06-10 11:30:45.178876 | 2026-06-10 11:30:45.179038 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 11:30:45.215302 | controller | ok 2026-06-10 11:30:45.252757 | 2026-06-10 11:30:45.252902 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the internal tenant] 2026-06-10 11:30:45.762930 | controller | ok: OK (2 bytes) 2026-06-10 11:30:45.775008 | 2026-06-10 11:30:45.775171 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the internal tenant] 2026-06-10 11:30:45.812695 | controller | skipping: Conditional result was False 2026-06-10 11:30:45.831535 | 2026-06-10 11:30:45.831827 | TASK [health-check/config-update-zuul : Set random value] 2026-06-10 11:30:45.927994 | controller | ok 2026-06-10 11:30:45.936422 | 2026-06-10 11:30:45.936553 | TASK [health-check/config-update-zuul : Create vars to generate and submit resources addition] 2026-06-10 11:30:45.975436 | controller | ok 2026-06-10 11:30:45.983775 | 2026-06-10 11:30:45.983923 | TASK [ansible.builtin.include_role : create-gerrit-repository] 2026-06-10 11:30:46.019288 | controller | ok 2026-06-10 11:30:46.050704 | 2026-06-10 11:30:46.050813 | TASK [create-gerrit-repository : Create gerrit repository named ichiban-config-3qoPMddz] 2026-06-10 07:30:47.942503 | controller | 2026-06-10 11:30:47,941 INFO managesf.model.yamlbkd.engine - Resource engine is using /tmp/direct-apply as workdir 2026-06-10 07:30:47.942998 | controller | 2026-06-10 11:30:47,941 INFO managesf.model.yamlbkd.engine - Resources engine: direct apply resources requested 2026-06-10 07:30:47.970118 | controller | 2026-06-10 11:30:47,969 INFO managesf.GerritClient - ichiban-config-3qoPMddz-core: creating group (Team core for the ichiban-config-3qoPMddz repo) 2026-06-10 07:30:48.104232 | controller | 2026-06-10 11:30:48,103 INFO managesf.GerritClient - ichiban-config-3qoPMddz-core: deleting group member admin 2026-06-10 07:30:48.159893 | controller | 2026-06-10 11:30:48,158 INFO managesf.GerritClient - ichiban-config-3qoPMddz-ptl: creating group (Team lead for the ichiban-config-3qoPMddz repo) 2026-06-10 07:30:48.251929 | controller | 2026-06-10 11:30:48,250 INFO managesf.GerritClient - ichiban-config-3qoPMddz-ptl: deleting group member admin 2026-06-10 07:30:48.350760 | controller | 2026-06-10 11:30:48,349 INFO managesf.GerritClient - ichiban-config-3qoPMddz-ptl: adding group member admin@sfop.me 2026-06-10 07:30:48.450466 | controller | 2026-06-10 11:30:48,449 INFO managesf.GerritClient - ichiban-config-3qoPMddz: creating project (ichiban-config-3qoPMddz repository) 2026-06-10 07:30:48.546142 | controller | 2026-06-10 11:30:48,545 INFO managesf.services.gerrit.utils - [gerrit] Clone repository ichiban-config-3qoPMddz 2026-06-10 07:30:48.842291 | controller | 2026-06-10 11:30:48,841 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'clone', 'ssh://admin@gerrit-sshd:29418/ichiban-config-3qoPMddz', '/tmp/tmpukdiaifu/clone-ichiban-config-3qoPMddz'] output 2026-06-10 07:30:48.842344 | controller | 2026-06-10 11:30:48,841 INFO managesf.services.gerrit.utils - Cloning into '/tmp/tmpukdiaifu/clone-ichiban-config-3qoPMddz'... 2026-06-10 07:30:48.842352 | controller | Warning: Permanently added '[gerrit-sshd]:29418' (ED25519) to the list of known hosts. 2026-06-10 07:30:48.842358 | controller | Total 2 (delta 0), reused 0 (delta 0) 2026-06-10 07:30:48.842364 | controller | 2026-06-10 07:30:49.039965 | controller | 2026-06-10 11:30:49,038 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', '-a'] output 2026-06-10 07:30:49.040027 | controller | 2026-06-10 11:30:49,039 INFO managesf.services.gerrit.utils - 2026-06-10 07:30:49.098102 | controller | 2026-06-10 11:30:49,097 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on config for repository ichiban-config-3qoPMddz 2026-06-10 07:30:49.315502 | controller | 2026-06-10 11:30:49,314 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', 'origin', 'refs/meta/config:refs/remotes/origin/meta/config'] output 2026-06-10 07:30:49.315559 | controller | 2026-06-10 11:30:49,314 INFO managesf.services.gerrit.utils - Total 4 (delta 0), reused 0 (delta 0) 2026-06-10 07:30:49.315565 | controller | From ssh://gerrit-sshd:29418/ichiban-config-3qoPMddz 2026-06-10 07:30:49.315570 | controller | * [new ref] refs/meta/config -> origin/meta/config 2026-06-10 07:30:49.315575 | controller | 2026-06-10 07:30:49.321877 | controller | 2026-06-10 11:30:49,321 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'meta/config'] output 2026-06-10 07:30:49.321892 | controller | 2026-06-10 11:30:49,321 INFO managesf.services.gerrit.utils - Switched to a new branch 'meta/config' 2026-06-10 07:30:49.321896 | controller | branch 'meta/config' set up to track 'origin/meta/config'. 2026-06-10 07:30:49.321900 | controller | 2026-06-10 07:30:49.321903 | controller | 2026-06-10 11:30:49,321 INFO managesf.services.gerrit.utils - [gerrit] Add file project.config to index 2026-06-10 07:30:49.326669 | controller | 2026-06-10 11:30:49,326 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'project.config'] output 2026-06-10 07:30:49.326681 | controller | 2026-06-10 11:30:49,326 INFO managesf.services.gerrit.utils - 2026-06-10 07:30:49.326684 | controller | 2026-06-10 11:30:49,326 INFO managesf.services.gerrit.utils - [gerrit] Add file groups to index 2026-06-10 07:30:49.332344 | controller | 2026-06-10 11:30:49,331 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'groups'] output 2026-06-10 07:30:49.332407 | controller | 2026-06-10 11:30:49,331 INFO managesf.services.gerrit.utils - 2026-06-10 07:30:49.338049 | controller | 2026-06-10 11:30:49,337 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-10 07:30:49.338103 | controller | 2026-06-10 11:30:49,337 INFO managesf.services.gerrit.utils - M groups 2026-06-10 07:30:49.338113 | controller | M project.config 2026-06-10 07:30:49.338122 | controller | 2026-06-10 07:30:49.348685 | controller | 2026-06-10 11:30:49,347 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mProvides ACL and Groups'] output 2026-06-10 07:30:49.348726 | controller | 2026-06-10 11:30:49,348 INFO managesf.services.gerrit.utils - [meta/config d87c955] Provides ACL and Groups 2026-06-10 07:30:49.348738 | controller | 2 files changed, 32 insertions(+), 10 deletions(-) 2026-06-10 07:30:49.348747 | controller | 2026-06-10 07:30:49.794562 | controller | 2026-06-10 11:30:49,793 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'meta/config:meta/config'] output 2026-06-10 07:30:49.794626 | controller | 2026-06-10 11:30:49,793 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 07:30:49.794640 | controller | To ssh://gerrit-sshd:29418/ichiban-config-3qoPMddz 2026-06-10 07:30:49.794646 | controller | 4b33635..d87c955 meta/config -> refs/meta/config 2026-06-10 07:30:49.794652 | controller | 2026-06-10 07:30:49.794660 | controller | 2026-06-10 11:30:49,793 INFO managesf.services.gerrit.utils - [gerrit] Push on config for repository ichiban-config-3qoPMddz 2026-06-10 07:30:49.794666 | controller | 2026-06-10 11:30:49,794 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on master for repository ichiban-config-3qoPMddz 2026-06-10 07:30:49.803351 | controller | 2026-06-10 11:30:49,802 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'master'] output 2026-06-10 07:30:49.803487 | controller | 2026-06-10 11:30:49,802 INFO managesf.services.gerrit.utils - Switched to branch 'master' 2026-06-10 07:30:49.803499 | controller | Your branch is up to date with 'origin/master'. 2026-06-10 07:30:49.803505 | controller | 2026-06-10 07:30:49.811889 | controller | 2026-06-10 11:30:49,810 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'reset', '--hard', 'origin/master'] output 2026-06-10 07:30:49.811917 | controller | 2026-06-10 11:30:49,811 INFO managesf.services.gerrit.utils - HEAD is now at 864f5b2 Initial empty repository 2026-06-10 07:30:49.811924 | controller | 2026-06-10 07:30:49.811932 | controller | 2026-06-10 11:30:49,811 INFO managesf.services.gerrit.utils - [gerrit] Add file .gitreview to index 2026-06-10 07:30:49.819087 | controller | 2026-06-10 11:30:49,818 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', '.gitreview'] output 2026-06-10 07:30:49.819110 | controller | 2026-06-10 11:30:49,818 INFO managesf.services.gerrit.utils - 2026-06-10 07:30:49.825971 | controller | 2026-06-10 11:30:49,825 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-10 07:30:49.825994 | controller | 2026-06-10 11:30:49,825 INFO managesf.services.gerrit.utils - A .gitreview 2026-06-10 07:30:49.826000 | controller | 2026-06-10 07:30:49.841606 | controller | 2026-06-10 11:30:49,840 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mManageSF commit'] output 2026-06-10 07:30:49.841674 | controller | 2026-06-10 11:30:49,840 INFO managesf.services.gerrit.utils - [master 22c5ad3] ManageSF commit 2026-06-10 07:30:49.841686 | controller | 1 file changed, 5 insertions(+) 2026-06-10 07:30:49.841696 | controller | create mode 100644 .gitreview 2026-06-10 07:30:49.841705 | controller | 2026-06-10 07:30:50.227512 | controller | 2026-06-10 11:30:50,226 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'master'] output 2026-06-10 07:30:50.227593 | controller | 2026-06-10 11:30:50,226 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 07:30:50.227612 | controller | To ssh://gerrit-sshd:29418/ichiban-config-3qoPMddz 2026-06-10 07:30:50.227622 | controller | 864f5b2..22c5ad3 master -> master 2026-06-10 07:30:50.227630 | controller | 2026-06-10 07:30:50.227639 | controller | 2026-06-10 11:30:50,226 INFO managesf.services.gerrit.utils - [gerrit] Push on master for repository ichiban-config-3qoPMddz 2026-06-10 07:30:50.227664 | controller | 2026-06-10 11:30:50,227 INFO managesf.services.gerrit.utils - [gerrit] List remote branches 2026-06-10 07:30:50.234663 | controller | 2026-06-10 11:30:50,233 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'branch', '-rv', '--abbrev=40'] output 2026-06-10 07:30:50.234741 | controller | 2026-06-10 11:30:50,233 INFO managesf.services.gerrit.utils - origin/HEAD -> origin/master 2026-06-10 07:30:50.234752 | controller | origin/master 22c5ad3c4b72c759904ef5d0efa5b19f4bea1001 ManageSF commit 2026-06-10 07:30:50.234761 | controller | origin/meta/config 4b3363576434bc67be82a79fc4736f027655e092 Created project 2026-06-10 07:30:50.234769 | controller | 2026-06-10 07:30:50.234785 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-3qoPMddz-core] is going to be created. 2026-06-10 07:30:50.234797 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-3qoPMddz-ptl] is going to be created. 2026-06-10 07:30:50.234900 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-3qoPMddz] is going to be created. 2026-06-10 07:30:50.234997 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-3qoPMddz-acl] is going to be created. 2026-06-10 07:30:50.235016 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-3qoPMddz-core] will be created. 2026-06-10 07:30:50.235028 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-3qoPMddz-core] has been created. 2026-06-10 07:30:50.235070 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-3qoPMddz-ptl] will be created. 2026-06-10 07:30:50.235266 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-3qoPMddz-ptl] has been created. 2026-06-10 07:30:50.235600 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-3qoPMddz-acl] will be created. 2026-06-10 07:30:50.235736 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-3qoPMddz-acl] has been created. 2026-06-10 07:30:50.235752 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-3qoPMddz] will be created. 2026-06-10 07:30:50.235886 | controller | 2026-06-10 11:30:50,234 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-3qoPMddz] has been created. 2026-06-10 07:30:50.235906 | controller | 2026-06-10 07:30:50.235915 | controller | === Resources actions === 2026-06-10 07:30:50.235924 | controller | 2026-06-10 07:30:50.235932 | controller | Resource [type: groups, ID: ichiban-config-3qoPMddz-core] is going to be created. 2026-06-10 07:30:50.235940 | controller | Resource [type: groups, ID: ichiban-config-3qoPMddz-ptl] is going to be created. 2026-06-10 07:30:50.235948 | controller | Resource [type: repos, ID: ichiban-config-3qoPMddz] is going to be created. 2026-06-10 07:30:50.235956 | controller | Resource [type: acls, ID: ichiban-config-3qoPMddz-acl] is going to be created. 2026-06-10 07:30:50.235964 | controller | Resource [type: groups, ID: ichiban-config-3qoPMddz-core] will be created. 2026-06-10 07:30:50.235972 | controller | Resource [type: groups, ID: ichiban-config-3qoPMddz-core] has been created. 2026-06-10 07:30:50.235980 | controller | Resource [type: groups, ID: ichiban-config-3qoPMddz-ptl] will be created. 2026-06-10 07:30:50.235988 | controller | Resource [type: groups, ID: ichiban-config-3qoPMddz-ptl] has been created. 2026-06-10 07:30:50.235996 | controller | Resource [type: acls, ID: ichiban-config-3qoPMddz-acl] will be created. 2026-06-10 07:30:50.236014 | controller | Resource [type: acls, ID: ichiban-config-3qoPMddz-acl] has been created. 2026-06-10 07:30:50.236022 | controller | Resource [type: repos, ID: ichiban-config-3qoPMddz] will be created. 2026-06-10 07:30:50.236030 | controller | Resource [type: repos, ID: ichiban-config-3qoPMddz] has been created. 2026-06-10 11:30:50.591572 | controller | ok: Runtime: 0:00:04.152848 2026-06-10 11:30:50.599568 | 2026-06-10 11:30:50.599670 | TASK [create-gerrit-repository : Ensure repository exists in Gerrit] 2026-06-10 11:30:51.079800 | controller | ok: OK (752 bytes) 2026-06-10 11:30:51.098840 | 2026-06-10 11:30:51.098989 | TASK [health-check/config-update-zuul : Set bad Zuul tenant config] 2026-06-10 11:30:51.532460 | controller | changed: Block inserted 2026-06-10 11:30:51.541656 | 2026-06-10 11:30:51.541792 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 11:30:51.780258 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.008250 2026-06-10 11:30:51.780738 | controller | ok: All items complete 2026-06-10 11:30:51.780795 | 2026-06-10 11:30:52.013183 | controller | [master 7647611] Add zuul/main.yaml 2026-06-10 11:30:52.013294 | controller | 1 file changed, 5 insertions(+) 2026-06-10 11:30:52.013386 | controller | ok: Item: git commit -m 'Add zuul/main.yaml' Runtime: 0:00:00.012788 2026-06-10 11:30:52.032699 | 2026-06-10 11:30:52.032913 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 11:30:52.067675 | controller | ok 2026-06-10 11:30:52.107573 | 2026-06-10 11:30:52.107699 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 11:30:52.137851 | controller | ok 2026-06-10 11:30:52.147531 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 11:30:52.154257 | 2026-06-10 11:30:52.154367 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 07:30:53.775014 | controller | Your change was committed before the commit hook was installed. 2026-06-10 07:30:53.775092 | controller | Amending the commit to add a gerrit change id. 2026-06-10 07:30:53.775101 | controller | remote: 2026-06-10 07:30:53.775108 | controller | remote: Processing changes: new: 1 (\) 2026-06-10 07:30:53.775117 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-10 07:30:53.775142 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-10 07:30:53.775150 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-10 07:30:53.775156 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 07:30:53.775194 | controller | remote: 2026-06-10 07:30:53.775201 | controller | remote: SUCCESS 2026-06-10 07:30:53.775207 | controller | remote: 2026-06-10 07:30:53.775214 | controller | remote: https://gerrit.sfop.me/c/config/+/1 Add zuul/main.yaml [NEW] 2026-06-10 07:30:53.775220 | controller | remote: 2026-06-10 07:30:53.775225 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 07:30:53.775231 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:30:54.274301 | controller | ok: Runtime: 0:00:01.363256 2026-06-10 11:30:54.278581 | 2026-06-10 11:30:54.278647 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 07:30:54.552224 | controller | Iea3775dc696bd145718488a650080f252a7c74b4 2026-06-10 11:30:54.807254 | controller | ok: Runtime: 0:00:00.007630 2026-06-10 11:30:54.817185 | 2026-06-10 11:30:54.817327 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 11:30:55.680873 | controller | ok: OK (47 bytes) 2026-06-10 11:30:55.686660 | 2026-06-10 11:30:55.686736 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 11:30:55.716626 | controller | ok 2026-06-10 11:30:55.722939 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 11:30:55.730089 | 2026-06-10 11:30:55.730159 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 11:30:56.308190 | controller | ok: OK (7142 bytes) 2026-06-10 11:30:56.319015 | 2026-06-10 11:30:56.319172 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 11:30:56.374922 | controller | ok 2026-06-10 11:30:56.389860 | 2026-06-10 11:30:56.390000 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 11:31:58.223965 | controller | ok: OK (1162 bytes) 2026-06-10 11:31:58.235172 | 2026-06-10 11:31:58.235272 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 11:31:58.263738 | controller | skipping: Conditional result was False 2026-06-10 11:31:58.277772 | 2026-06-10 11:31:58.277951 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 11:31:58.305172 | controller | skipping: Conditional result was False 2026-06-10 11:31:58.314646 | 2026-06-10 11:31:58.314776 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 11:31:58.341095 | controller | skipping: Conditional result was False 2026-06-10 11:31:58.353998 | 2026-06-10 11:31:58.354090 | TASK [health-check/config-update-zuul : Set Zuul tenant config] 2026-06-10 11:32:00.778278 | controller | changed: Block inserted 2026-06-10 11:32:00.782716 | 2026-06-10 11:32:00.782783 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 11:32:01.077807 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.005649 2026-06-10 11:32:01.327797 | controller | [master 34d1bce] Add zuul/main.yaml 2026-06-10 11:32:01.327966 | controller | Date: Wed Jun 10 07:30:51 2026 -0400 2026-06-10 11:32:01.328020 | controller | 1 file changed, 8 insertions(+) 2026-06-10 11:32:01.328075 | controller | ok: Item: git commit --amend --no-edit Runtime: 0:00:00.041554 2026-06-10 11:32:01.338748 | 2026-06-10 11:32:01.338991 | TASK [Submit change and verify Zuul console output] 2026-06-10 11:32:01.373501 | controller | ok 2026-06-10 11:32:01.406869 | 2026-06-10 11:32:01.406989 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 11:32:01.436405 | controller | ok 2026-06-10 11:32:01.442889 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 11:32:01.448106 | 2026-06-10 11:32:01.448173 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 07:32:02.372118 | controller | remote: 2026-06-10 07:32:02.372197 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 07:32:02.372205 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 07:32:02.372210 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 07:32:02.372216 | controller | remote: Processing changes: refs: 1, updated: 1, done 2026-06-10 07:32:02.372222 | controller | remote: 2026-06-10 07:32:02.372226 | controller | remote: SUCCESS 2026-06-10 07:32:02.372230 | controller | remote: 2026-06-10 07:32:02.372234 | controller | remote: https://gerrit.sfop.me/c/config/+/1 Add zuul/main.yaml 2026-06-10 07:32:02.372239 | controller | remote: 2026-06-10 07:32:02.372242 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 07:32:02.372248 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:32:02.477261 | controller | ok: Runtime: 0:00:00.608365 2026-06-10 11:32:02.487850 | 2026-06-10 11:32:02.487982 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 07:32:02.799830 | controller | Iea3775dc696bd145718488a650080f252a7c74b4 2026-06-10 11:32:03.035911 | controller | ok: Runtime: 0:00:00.011631 2026-06-10 11:32:03.045337 | 2026-06-10 11:32:03.045542 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 11:32:03.627840 | controller | ok: OK (47 bytes) 2026-06-10 11:32:03.640783 | 2026-06-10 11:32:03.640927 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 11:32:03.668526 | controller | skipping: Conditional result was False 2026-06-10 11:32:03.680621 | 2026-06-10 11:32:03.680758 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 11:32:03.717310 | controller | ok 2026-06-10 11:32:03.730664 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-zuul-console-success.yaml 2026-06-10 11:32:03.744771 | 2026-06-10 11:32:03.744856 | TASK [health-check/repo-submit-change : Wait for config-update job] 2026-06-10 11:33:05.767765 | controller | ok: Runtime: 0:00:00.047589 2026-06-10 11:33:05.777401 | 2026-06-10 11:33:05.777530 | TASK [health-check/repo-submit-change : Connect and validate console stream] 2026-06-10 07:33:06.242491 | controller | 2026-06-10 11:32:57.446454 | Job console starting 2026-06-10 07:33:06.242603 | controller | 2026-06-10 11:32:57.447447 | Updating git repos 2026-06-10 07:33:06.242616 | controller | 2026-06-10 11:32:57.465068 | Cloning repos into workspace 2026-06-10 07:33:06.242650 | controller | 2026-06-10 11:32:58.759336 | Restoring repo states 2026-06-10 07:33:06.242662 | controller | 2026-06-10 11:32:58.765811 | Merging changes 2026-06-10 07:33:06.242673 | controller | 2026-06-10 11:32:58.765834 | Checking out repos 2026-06-10 07:33:06.242683 | controller | 2026-06-10 11:32:58.789547 | Preparing playbooks 2026-06-10 07:33:06.242692 | controller | 2026-06-10 11:32:59.474550 | Running Ansible setup 2026-06-10 07:33:06.242703 | controller | 2026-06-10 11:33:02.475531 | PRE-RUN START: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-10 07:33:06.242712 | controller | 2026-06-10 11:33:03.940904 | 2026-06-10 07:33:06.242721 | controller | 2026-06-10 11:33:03.941078 | PLAY [localhost] 2026-06-10 07:33:06.242729 | controller | 2026-06-10 11:33:03.951573 | 2026-06-10 07:33:06.242737 | controller | 2026-06-10 11:33:03.951698 | TASK [Gathering Facts] 2026-06-10 07:33:06.242745 | controller | 2026-06-10 11:33:05.752269 | localhost | ok 2026-06-10 07:33:06.242753 | controller | 2026-06-10 11:33:05.758748 | 2026-06-10 07:33:06.242761 | controller | 2026-06-10 11:33:05.758817 | TASK [Setup log path fact] 2026-06-10 07:33:06.242769 | controller | 2026-06-10 11:33:05.773795 | localhost | ok 2026-06-10 07:33:06.242777 | controller | 2026-06-10 11:33:05.783092 | 2026-06-10 07:33:06.242785 | controller | 2026-06-10 11:33:05.783159 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 07:33:06.242793 | controller | 2026-06-10 11:33:05.797689 | localhost | ok 2026-06-10 07:33:06.242801 | controller | 2026-06-10 11:33:05. 2026-06-10 07:33:06.242817 | controller | 849970 | 2026-06-10 07:33:06.242832 | controller | 2026-06-10 11:33:05.850243 | TASK [emit-job-header : Print job information] 2026-06-10 07:33:06.242841 | controller | 2026-06-10 11:33:05.953491 | # Job Information 2026-06-10 07:33:06.242849 | controller | 2026-06-10 11:33:05.953693 | Ansible Version: 2.16.18 2026-06-10 07:33:06.242857 | controller | 2026-06-10 11:33:05.953737 | Job: config-update 2026-06-10 07:33:06.242865 | controller | 2026-06-10 11:33:05.953767 | Pipeline: post 2026-06-10 07:33:06.242873 | controller | 2026-06-10 11:33:05.953793 | Executor: zuul-executor-0.zuul-executor-headless.sf.svc.cluster.local 2026-06-10 07:33:06.242882 | controller | 2026-06-10 11:33:05.953818 | Triggered by: https://gerrit.sfop.me/plugins/gitiles/config/+/34d1bce832cc1df8d44b98d95ea747d427f139cf^!/ 2026-06-10 07:33:06.242890 | controller | 2026-06-10 11:33:05.953881 | Log URL (when completed): https://sfop.me/logs/f8a/internal/f8acfc0ca29a480dbf23c0e841cb64d1/ 2026-06-10 07:33:06.242898 | controller | 2026-06-10 11:33:05.953909 | Event ID: 27fd70732dfd4a78aa7111a4d0d22b65 2026-06-10 07:33:06.242907 | controller | 2026-06-10 11:33:05.957483 | 2026-06-10 07:33:06.242915 | controller | 2026-06-10 11:33:05.957572 | LOOP [emit-job-header : Print node information] 2026-06-10 07:33:06.242922 | controller | 2026-06-10 11:33:05.992803 | 2026-06-10 07:33:06.242930 | controller | 2026-06-10 11:33:05.993049 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-10 07:33:06.242938 | controller | 2026-06-10 07:33:07.242664 | controller | 2026-06-10 11:33:07.086773 | localhost -> localhost | changed 2026-06-10 07:33:07.242709 | controller | 2026-06-10 11:33:07.145651 | 2026-06-10 07:33:07.242714 | controller | 2026-06-10 11:33:07.145831 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-06-10 07:33:07.242718 | controller | 2026-06-10 07:33:09.244767 | controller | 2026-06-10 11:33:09.181738 | localhost -> localhost | changed 2026-06-10 07:33:09.244813 | controller | 2026-06-10 11:33:09.242605 | 2026-06-10 07:33:09.244820 | controller | 2026-06-10 11:33:09.242729 | PLAY [all] 2026-06-10 07:33:09.244827 | controller | 2026-06-10 11:33:09.242837 | 2026-06-10 07:33:09.244833 | controller | 2026-06-10 11:33:09.242862 | PLAY RECAP 2026-06-10 07:33:09.244839 | controller | 2026-06-10 11:33:09.242895 | localhost | ok: 5 changed: 2 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-06-10 07:33:09.244845 | controller | 2026-06-10 11:33:09.242918 | 2026-06-10 07:33:09.244851 | controller | 2026-06-10 07:33:09.745296 | controller | 2026-06-10 11:33:09.547240 | PRE-RUN END RESULT_NORMAL: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-10 07:33:09.745366 | controller | 2026-06-10 11:33:09.548660 | RUN START: [trusted : git-server/system-config/playbooks/config/update.yaml@master] 2026-06-10 07:33:09.745398 | controller | 2026-06-10 11:33:11.331422 | controller | ok: Runtime: 0:00:05.016404 2026-06-10 11:33:11.344340 | 2026-06-10 11:33:11.344528 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 11:33:11.380598 | controller | ok 2026-06-10 11:33:11.395332 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 11:33:11.487250 | 2026-06-10 11:33:11.487411 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-10 07:33:11.692340 | controller | 34d1bce832cc1df8d44b98d95ea747d427f139cf 2026-06-10 11:33:12.017537 | controller | ok: Runtime: 0:00:00.008456 2026-06-10 11:33:12.029860 | 2026-06-10 11:33:12.030025 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 11:34:27.629713 | controller | ok: OK (1142 bytes) 2026-06-10 11:34:27.647026 | 2026-06-10 11:34:27.647179 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 11:34:27.683652 | controller | skipping: Conditional result was False 2026-06-10 11:34:27.702827 | 2026-06-10 11:34:27.702971 | TASK [health-check/config-update-zuul : ansible.builtin.set_fact] 2026-06-10 11:34:27.748839 | controller | ok 2026-06-10 11:34:27.761825 | 2026-06-10 11:34:27.761956 | TASK [Check that Zuul has pushed the job console on the logserver] 2026-06-10 11:34:27.796383 | controller | ok 2026-06-10 11:34:27.836573 | 2026-06-10 11:34:27.836675 | TASK [health-check/ensure-job-result-artifacts : Ensure job console (job-output.txt.gz) exists in Logserver] 2026-06-10 11:34:28.347810 | controller | ok: OK (4551 bytes) 2026-06-10 11:34:28.361275 | 2026-06-10 11:34:28.361441 | TASK [health-check/ensure-job-result-artifacts : Ensure internal logserver rewrite from job-output.txt to job-output.txt.gz] 2026-06-10 11:34:28.823445 | controller | ok: OK (4551 bytes) 2026-06-10 11:34:28.841838 | 2026-06-10 11:34:28.841985 | TASK [health-check/config-update-zuul : Get Zuul projects list] 2026-06-10 11:34:29.324535 | controller | ok: OK (144 bytes) 2026-06-10 11:34:29.336340 | 2026-06-10 11:34:29.336649 | TASK [health-check/config-update-zuul : Ensure project exists in Zuul (fail if not)] 2026-06-10 11:34:29.384581 | controller | skipping: Conditional result was False 2026-06-10 11:34:29.406636 | 2026-06-10 11:34:29.406795 | TASK [health-check/check-logjuicer : Get logjuicer status] 2026-06-10 11:34:29.931970 | controller | ok: OK (2 bytes) 2026-06-10 11:34:29.951164 | 2026-06-10 11:34:29.951309 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 11:34:29.989656 | controller | ok 2026-06-10 11:34:30.019464 | 2026-06-10 11:34:30.019605 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the demo-tenant tenant] 2026-06-10 11:34:30.525487 | controller | ok: OK (2 bytes) 2026-06-10 11:34:30.540480 | 2026-06-10 11:34:30.540623 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the demo-tenant tenant] 2026-06-10 11:34:30.579282 | controller | skipping: Conditional result was False 2026-06-10 11:34:30.599788 | 2026-06-10 11:34:30.599936 | TASK [health-check/zuul-demo-tenant-workflow : Fetch and reset demo-project to latest origin for fast-forward submit] 2026-06-10 07:34:30.913113 | controller | HEAD is now at 6f67f48 Automatic secret re-encryption 2026-06-10 11:34:31.139470 | controller | ok: Runtime: 0:00:00.071687 2026-06-10 11:34:31.144133 | 2026-06-10 11:34:31.144198 | TASK [health-check/zuul-demo-tenant-workflow : Update the demo-job to sleep 1 second] 2026-06-10 11:34:31.553537 | controller | changed: 1 replacements made 2026-06-10 11:34:31.562576 | 2026-06-10 11:34:31.562720 | LOOP [health-check/zuul-demo-tenant-workflow : Commit file in the demo-project repo] 2026-06-10 11:34:31.886511 | controller | ok: Item: git add playbooks/run.yaml Runtime: 0:00:00.008005 2026-06-10 11:34:31.887146 | controller | changed: All items complete 2026-06-10 11:34:31.887211 | 2026-06-10 11:34:32.110040 | controller | [master ab870c8] Update demo-job to sleep 1 second 2026-06-10 11:34:32.110201 | controller | 1 file changed, 1 insertion(+), 1 deletion(-) 2026-06-10 11:34:32.110267 | controller | ok: Item: git commit -m 'Update demo-job to sleep 1 second' Runtime: 0:00:00.009853 2026-06-10 11:34:32.129857 | 2026-06-10 11:34:32.130067 | TASK [Submit, merge and verify post-merge job] 2026-06-10 11:34:32.165877 | controller | ok 2026-06-10 11:34:32.207624 | 2026-06-10 11:34:32.207759 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 11:34:32.240118 | controller | ok 2026-06-10 11:34:32.248303 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 11:34:32.255712 | 2026-06-10 11:34:32.255797 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 07:34:33.079150 | controller | Your change was committed before the commit hook was installed. 2026-06-10 07:34:33.079243 | controller | Amending the commit to add a gerrit change id. 2026-06-10 07:34:33.079251 | controller | remote: 2026-06-10 07:34:33.079258 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:34:33.079265 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:34:33.079270 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 07:34:33.079275 | controller | remote: 2026-06-10 07:34:33.079279 | controller | remote: SUCCESS 2026-06-10 07:34:33.079283 | controller | remote: 2026-06-10 07:34:33.079287 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/2 Update demo-job to sleep 1 second [NEW] 2026-06-10 07:34:33.079292 | controller | remote: 2026-06-10 07:34:33.079296 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 07:34:33.079300 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:34:33.291491 | controller | ok: Runtime: 0:00:00.587322 2026-06-10 11:34:33.301768 | 2026-06-10 11:34:33.302010 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 07:34:33.553560 | controller | I3f11e1c83ad5c900f6c3a702f9ae768fc0a9307d 2026-06-10 11:34:33.851802 | controller | ok: Runtime: 0:00:00.006023 2026-06-10 11:34:33.861492 | 2026-06-10 11:34:33.861642 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 11:34:34.394061 | controller | ok: OK (47 bytes) 2026-06-10 11:34:34.405684 | 2026-06-10 11:34:34.405824 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 11:34:34.433115 | controller | skipping: Conditional result was False 2026-06-10 11:34:34.443243 | 2026-06-10 11:34:34.443422 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 11:34:34.470443 | controller | skipping: Conditional result was False 2026-06-10 11:34:34.554679 | 2026-06-10 11:34:34.554825 | TASK [health-check/repo-submit-change : Ensure a success for publish-job post job] 2026-06-10 11:34:34.593194 | controller | ok 2026-06-10 11:34:34.598765 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 11:34:34.604991 | 2026-06-10 11:34:34.605050 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-10 07:34:34.822634 | controller | c8f5c5f9ea4b45f06003754c50dfca1bf26077f0 2026-06-10 11:34:35.133586 | controller | ok: Runtime: 0:00:00.006292 2026-06-10 11:34:35.144816 | 2026-06-10 11:34:35.144949 | TASK [health-check/repo-submit-change : Wait publish-job SUCCESS result using zuul-web API] 2026-06-10 11:35:28.085131 | controller | ok: OK (927 bytes) 2026-06-10 11:35:28.098297 | 2026-06-10 11:35:28.098487 | TASK [health-check/repo-submit-change : Ensure a success for publish-job job] 2026-06-10 11:35:28.136416 | controller | skipping: Conditional result was False 2026-06-10 11:35:28.165926 | 2026-06-10 11:35:28.166076 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale down executor statefulset to ensure 1 replica only] 2026-06-10 07:35:28.513283 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 11:35:28.705998 | controller | ok: Runtime: 0:00:00.092429 2026-06-10 11:35:28.725425 | 2026-06-10 11:35:28.725622 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for only 1 executor running] 2026-06-10 07:35:28.997883 | controller | 1 2026-06-10 11:35:29.268785 | controller | ok: Runtime: 0:00:00.029972 2026-06-10 11:35:29.299224 | 2026-06-10 11:35:29.299443 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure we are on the default branch] 2026-06-10 07:35:29.574792 | controller | Already on 'master' 2026-06-10 07:35:29.575568 | controller | Your branch is ahead of 'origin/master' by 1 commit. 2026-06-10 07:35:29.575601 | controller | (use "git push" to publish your local commits) 2026-06-10 11:35:29.850024 | controller | ok: Runtime: 0:00:00.007347 2026-06-10 11:35:29.860077 | 2026-06-10 11:35:29.860212 | TASK [health-check/zuul-executor-ensure-graceful-stop : Create a test branch] 2026-06-10 07:35:30.105981 | controller | Switched to a new branch 'test_check_url' 2026-06-10 11:35:30.407293 | controller | ok: Runtime: 0:00:00.007178 2026-06-10 11:35:30.419182 | 2026-06-10 11:35:30.419396 | TASK [health-check/zuul-executor-ensure-graceful-stop : Replace demo-job with demo-job-check-url] 2026-06-10 11:35:30.677133 | controller | changed: 2 replacements made 2026-06-10 11:35:30.686538 | 2026-06-10 11:35:30.686665 | TASK [health-check/zuul-executor-ensure-graceful-stop : create the review] 2026-06-10 11:35:30.724023 | controller | ok 2026-06-10 11:35:30.743229 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-executor-ensure-graceful-stop/tasks/create-review.yaml 2026-06-10 11:35:30.752503 | 2026-06-10 11:35:30.752644 | LOOP [health-check/zuul-executor-ensure-graceful-stop : Create review] 2026-06-10 11:35:31.024273 | controller | ok: Item: git add --all Runtime: 0:00:00.007620 2026-06-10 11:35:31.316015 | controller | [test_check_url 4cdf6f9] test check url 2026-06-10 11:35:31.316144 | controller | 1 file changed, 2 insertions(+), 2 deletions(-) 2026-06-10 11:35:31.316182 | controller | ok: Item: git commit -m 'test check url' Runtime: 0:00:00.049926 2026-06-10 11:35:32.138895 | controller | remote: 2026-06-10 11:35:32.139054 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 11:35:32.139119 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 11:35:32.139172 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 11:35:32.139221 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 11:35:32.139269 | controller | remote: 2026-06-10 11:35:32.139316 | controller | remote: SUCCESS 2026-06-10 11:35:32.139442 | controller | remote: 2026-06-10 11:35:32.139492 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/3 test check url [NEW] 2026-06-10 11:35:32.139532 | controller | remote: 2026-06-10 11:35:32.139571 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 11:35:32.139609 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:35:32.139660 | controller | ok: Item: git review Runtime: 0:00:00.594770 2026-06-10 11:35:32.155599 | 2026-06-10 11:35:32.155735 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get Change-id] 2026-06-10 07:35:32.409513 | controller | I94842608fff1182eb6024b0ad47eedc4d0a4dbe9 2026-06-10 11:35:32.696484 | controller | ok: Runtime: 0:00:00.008376 2026-06-10 11:35:32.708565 | 2026-06-10 11:35:32.708723 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 11:35:32.756945 | controller | ok 2026-06-10 11:35:32.767105 | 2026-06-10 11:35:32.767239 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get review number] 2026-06-10 07:35:33.098489 | controller | 3 2026-06-10 11:35:33.303825 | controller | ok: Runtime: 0:00:00.058070 2026-06-10 11:35:33.309988 | 2026-06-10 11:35:33.310089 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 11:35:33.355028 | controller | ok 2026-06-10 11:35:33.364941 | 2026-06-10 11:35:33.365096 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.set_fact] 2026-06-10 11:35:33.413127 | controller | ok 2026-06-10 11:35:33.424718 | 2026-06-10 11:35:33.424913 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for job to start] 2026-06-10 07:35:33.771208 | controller | null 2026-06-10 11:35:44.091634 | controller | ok: Runtime: 0:00:00.074972 2026-06-10 11:35:44.172758 | 2026-06-10 11:35:44.172893 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale down executor statefulset to trigger a graceful stop] 2026-06-10 07:35:44.467320 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 11:35:44.700961 | controller | ok: Runtime: 0:00:00.090707 2026-06-10 11:35:44.710342 | 2026-06-10 11:35:44.710509 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure executor component is marked for shutdown in Zuul API] 2026-06-10 07:35:45.004319 | controller | paused 2026-06-10 11:35:45.257978 | controller | ok: Runtime: 0:00:00.034930 2026-06-10 11:35:45.263998 | 2026-06-10 11:35:45.264081 | TASK [health-check/zuul-executor-ensure-graceful-stop : show executor state] 2026-06-10 11:35:45.307936 | controller | ok: paused 2026-06-10 11:35:45.313566 | 2026-06-10 11:35:45.313644 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get back on default branch] 2026-06-10 07:35:45.565754 | controller | Switched to branch 'master' 2026-06-10 07:35:45.566700 | controller | Your branch is ahead of 'origin/master' by 1 commit. 2026-06-10 07:35:45.566749 | controller | (use "git push" to publish your local commits) 2026-06-10 11:35:45.845217 | controller | ok: Runtime: 0:00:00.010160 2026-06-10 11:35:45.854160 | 2026-06-10 11:35:45.854291 | TASK [health-check/zuul-executor-ensure-graceful-stop : Create a test branch] 2026-06-10 07:35:46.142151 | controller | Switched to a new branch 'test_short_job' 2026-06-10 11:35:46.403405 | controller | ok: Runtime: 0:00:00.006074 2026-06-10 11:35:46.412802 | 2026-06-10 11:35:46.412927 | TASK [health-check/zuul-executor-ensure-graceful-stop : Update the demo-job to sleep 2 seconds] 2026-06-10 11:35:46.704844 | controller | changed: 1 replacements made 2026-06-10 11:35:46.714310 | 2026-06-10 11:35:46.714485 | TASK [health-check/zuul-executor-ensure-graceful-stop : create the review] 2026-06-10 11:35:46.751455 | controller | ok 2026-06-10 11:35:46.765486 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-executor-ensure-graceful-stop/tasks/create-review.yaml 2026-06-10 11:35:46.781710 | 2026-06-10 11:35:46.781790 | LOOP [health-check/zuul-executor-ensure-graceful-stop : Create review] 2026-06-10 11:35:47.029352 | controller | ok: Item: git add --all Runtime: 0:00:00.006357 2026-06-10 11:35:47.029880 | controller | changed: All items complete 2026-06-10 11:35:47.029938 | 2026-06-10 11:35:47.291238 | controller | [test_short_job 67f112f] test short job 2026-06-10 11:35:47.291426 | controller | 1 file changed, 1 insertion(+), 1 deletion(-) 2026-06-10 11:35:47.291495 | controller | ok: Item: git commit -m 'test short job' Runtime: 0:00:00.046017 2026-06-10 11:35:47.954077 | controller | remote: 2026-06-10 11:35:47.954239 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 11:35:47.954290 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 11:35:47.954330 | controller | remote: 2026-06-10 11:35:47.954409 | controller | remote: SUCCESS 2026-06-10 11:35:47.954450 | controller | remote: 2026-06-10 11:35:47.954487 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/4 test short job [NEW] 2026-06-10 11:35:47.954523 | controller | remote: 2026-06-10 11:35:47.954559 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 11:35:47.954594 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:35:47.954648 | controller | ok: Item: git review Runtime: 0:00:00.455492 2026-06-10 11:35:47.969726 | 2026-06-10 11:35:47.969938 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get Change-id] 2026-06-10 07:35:48.235842 | controller | I67b6a928f6899f75ee2a29b423857bb1c0e250aa 2026-06-10 11:35:48.516485 | controller | ok: Runtime: 0:00:00.008970 2026-06-10 11:35:48.522553 | 2026-06-10 11:35:48.522641 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 11:35:48.564163 | controller | ok 2026-06-10 11:35:48.570558 | 2026-06-10 11:35:48.570649 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get review number] 2026-06-10 07:35:48.872664 | controller | 4 2026-06-10 11:35:49.108948 | controller | ok: Runtime: 0:00:00.033885 2026-06-10 11:35:49.119325 | 2026-06-10 11:35:49.119495 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 11:35:49.167894 | controller | ok 2026-06-10 11:35:49.179300 | 2026-06-10 11:35:49.179523 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.set_fact] 2026-06-10 11:35:49.235062 | controller | ok 2026-06-10 11:35:49.244974 | 2026-06-10 11:35:49.245111 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure new review is queued] 2026-06-10 11:35:59.886947 | controller | ok: Runtime: 0:00:00.042742 2026-06-10 11:35:59.896450 | 2026-06-10 11:35:59.896530 | TASK [Create switch file on logserver so that first job can complete] 2026-06-10 11:35:59.926664 | controller | ok 2026-06-10 11:35:59.954913 | 2026-06-10 11:35:59.954996 | TASK [create-file-on-logserver : copy file on log server] 2026-06-10 07:36:00.392111 | controller | Defaulted container "logserver" out of: logserver, logserver-sshd, purgelogs, logserver-nodeexporter 2026-06-10 11:36:00.576209 | controller | ok: Runtime: 0:00:00.145074 2026-06-10 11:36:00.593434 | 2026-06-10 11:36:00.593571 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for executor pod to terminate] 2026-06-10 07:36:00.981130 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 07:36:00.981330 | controller | config-base-secret-1-2837-ns-q9ctm 0/1 Completed 0 28m 2026-06-10 07:36:00.981361 | controller | config-base-secret-1-917-ns-5jhvl 0/1 Completed 0 34m 2026-06-10 07:36:00.981381 | controller | config-base-secret-1-917-ns-rz8lm 0/1 Completed 0 38m 2026-06-10 07:36:00.981396 | controller | gateway-6bbbd758d9-b58rz 1/1 Running 0 27m 2026-06-10 07:36:00.981409 | controller | gerrit-0 2/2 Running 0 45m 2026-06-10 07:36:00.981425 | controller | git-server-0 2/2 Running 0 10m 2026-06-10 07:36:00.981440 | controller | hound-search-0 1/1 Running 0 43m 2026-06-10 07:36:00.981453 | controller | logjuicer-6ff84764b7-29479 1/1 Running 0 43m 2026-06-10 07:36:00.981480 | controller | logserver-0 4/4 Running 0 22m 2026-06-10 07:36:00.981495 | controller | mariadb-0 3/3 Running 0 20m 2026-06-10 07:36:00.981508 | controller | nodepool-builder-0 5/5 Running 0 18m 2026-06-10 07:36:00.981533 | controller | nodepool-launcher-75db676fdd-rdz7v 3/3 Running 0 19m 2026-06-10 07:36:00.981547 | controller | post-init-97qhg 0/1 Completed 0 44m 2026-06-10 07:36:00.981560 | controller | test-fluentbit 1/1 Running 0 37m 2026-06-10 07:36:00.981573 | controller | test-loki-8cd4fd56-2v44p 1/1 Running 0 37m 2026-06-10 07:36:00.981586 | controller | zookeeper-0 3/3 Running 0 19m 2026-06-10 07:36:00.981598 | controller | zookeeper-1 3/3 Running 0 19m 2026-06-10 07:36:00.981611 | controller | zookeeper-2 3/3 Running 1 (19m ago) 19m 2026-06-10 07:36:00.981623 | controller | zuul-db-provision-p79wx 0/1 Completed 0 30m 2026-06-10 07:36:00.981636 | controller | zuul-db-provision-zlqvm 0/1 Completed 0 43m 2026-06-10 07:36:00.981648 | controller | zuul-executor-0 2/2 Terminating 0 10m 2026-06-10 07:36:00.981661 | controller | zuul-merger-0 2/2 Running 0 10m 2026-06-10 07:36:00.981673 | controller | zuul-merger-1 2/2 Running 0 10m 2026-06-10 07:36:00.981685 | controller | zuul-scheduler-0 3/3 Running 0 6m19s 2026-06-10 07:36:00.981697 | controller | zuul-web-5bdd465c4c-hv8z5 1/1 Running 0 6m20s 2026-06-10 07:36:00.981710 | controller | zuul-weeder-5fb87885d-qwtrc 1/1 Running 0 9m21s 2026-06-10 11:36:21.684413 | controller | ok: Runtime: 0:00:00.093026 2026-06-10 11:36:21.692541 | 2026-06-10 11:36:21.692658 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure first job completed] 2026-06-10 11:36:22.226830 | controller | ok: Runtime: 0:00:00.051448 2026-06-10 11:36:22.234127 | 2026-06-10 11:36:22.234219 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure second job hasn't completed] 2026-06-10 11:36:22.771959 | controller | ok: Runtime: 0:00:00.062956 2026-06-10 11:36:22.782094 | 2026-06-10 11:36:22.782229 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale up executor statefulset] 2026-06-10 07:36:23.174448 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 11:36:23.376856 | controller | ok: Runtime: 0:00:00.075944 2026-06-10 11:36:23.387167 | 2026-06-10 11:36:23.387379 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait until executor pod is up] 2026-06-10 07:36:23.784262 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 07:36:23.784369 | controller | zuul-executor-0 0/2 ContainerCreating 0 0s 2026-06-10 11:36:34.184486 | controller | ok: Runtime: 0:00:00.103374 2026-06-10 11:36:34.194121 | 2026-06-10 11:36:34.194255 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure executor component is running] 2026-06-10 07:36:34.520576 | controller | running 2026-06-10 11:36:34.738668 | controller | ok: Runtime: 0:00:00.049333 2026-06-10 11:36:34.747813 | 2026-06-10 11:36:34.747950 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure second review's job completed] 2026-06-10 11:37:06.024909 | controller | ok: Runtime: 0:00:00.061169 2026-06-10 11:37:06.042636 | 2026-06-10 11:37:06.042777 | TASK [health-check/test-nodepool-providers-secrets : Ensure an empty /tmp/kube.config] 2026-06-10 11:37:06.313004 | controller | changed 2026-06-10 11:37:06.322300 | 2026-06-10 11:37:06.322460 | TASK [health-check/test-nodepool-providers-secrets : Get existing kube.config from the secret] 2026-06-10 11:37:06.869859 | controller | ok: Runtime: 0:00:00.075775 2026-06-10 11:37:06.879012 | 2026-06-10 11:37:06.879145 | TASK [health-check/test-nodepool-providers-secrets : Create a fake clouds.yaml] 2026-06-10 11:37:07.656137 | controller | changed 2026-06-10 11:37:07.665285 | 2026-06-10 11:37:07.665449 | TASK [health-check/test-nodepool-providers-secrets : Update the secret clouds.yaml key] 2026-06-10 07:37:08.117444 | controller | Warning: resource secrets/nodepool-providers-secrets is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. 2026-06-10 07:37:08.280226 | controller | secret/nodepool-providers-secrets configured 2026-06-10 11:37:08.710293 | controller | ok: Runtime: 0:00:00.347831 2026-06-10 11:37:08.719453 | 2026-06-10 11:37:08.719587 | TASK [Wait for secrets to be updated] 2026-06-10 11:37:08.754562 | controller | ok 2026-06-10 11:37:08.782116 | 2026-06-10 11:37:08.782242 | TASK [run the operator in standalone mode] 2026-06-10 11:37:08.813884 | controller | ok 2026-06-10 11:37:08.833189 | 2026-06-10 11:37:08.833274 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:37:08.859330 | controller | skipping: Conditional result was False 2026-06-10 11:37:08.867276 | 2026-06-10 11:37:08.867384 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:37:08.893523 | controller | skipping: Conditional result was False 2026-06-10 11:37:08.901662 | 2026-06-10 11:37:08.901752 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:37:09.540563 | controller | 2026-06-10T07:37:09-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:37:09.540640 | controller | 2026-06-10 07:37:09.550972 | controller | 2026-06-10T07:37:09-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:37:09.551000 | controller | 2026-06-10 07:37:09.564363 | controller | 2026-06-10T07:37:09-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:37:09.566151 | controller | 2026-06-10T07:37:09-04:00 INFO Nothing to clean up. 2026-06-10 07:37:10.183898 | controller | 2026-06-10T07:37:10-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:37:10.350535 | controller | 2026-06-10T07:37:10-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:37:10.360867 | controller | 2026-06-10T07:37:10-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:37:10.360911 | controller | 2026-06-10T07:37:10-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:37:10.389100 | controller | 2026-06-10T07:37:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:37:10.389158 | controller | 2026-06-10T07:37:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:37:10.598652 | controller | 2026-06-10T07:37:10-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:37:10.968094 | controller | 2026-06-10T07:37:10-04:00 INFO Updating configmap, name: np-statsd-config-map 2026-06-10 07:37:10.968255 | controller | 2026-06-10T07:37:10-04:00 INFO Updating object, name: np-statsd-config-map 2026-06-10 07:37:10.988996 | controller | 2026-06-10T07:37:10-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-10 07:37:10.989069 | controller | 2026-06-10T07:37:10-04:00 INFO Updating object, name: nodepool-launcher 2026-06-10 07:37:11.568600 | controller | 2026-06-10T07:37:11-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-10 07:37:11.568672 | controller | 2026-06-10T07:37:11-04:00 INFO Updating object, name: nodepool-builder 2026-06-10 07:37:12.379239 | controller | 2026-06-10T07:37:12-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:37:12.379330 | controller | 2026-06-10T07:37:12-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:37:13.964010 | controller | 2026-06-10T07:37:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:37:13.964107 | controller | 2026-06-10T07:37:13-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:37:15.176498 | controller | 2026-06-10T07:37:15-04:00 INFO 2026-06-10 07:37:15.176611 | controller | - Config: OK 2026-06-10 07:37:15.176623 | controller | - Gateway: OK 2026-06-10 07:37:15.176632 | controller | - GitServer: OK 2026-06-10 07:37:15.176640 | controller | - HoundSearch: OK 2026-06-10 07:37:15.176648 | controller | - LogJuicer: OK 2026-06-10 07:37:15.176658 | controller | - Logserver: OK 2026-06-10 07:37:15.176666 | controller | - MariaDB: OK 2026-06-10 07:37:15.176674 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:37:15.176683 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:37:15.176691 | controller | - Zookeeper: OK 2026-06-10 07:37:15.176699 | controller | - Zuul: OK 2026-06-10 07:37:15.176707 | controller | 2026-06-10 07:37:15.176715 | controller | 2026-06-10T07:37:15-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:37:20.187592 | controller | 2026-06-10T07:37:20-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:37:20.190953 | controller | 2026-06-10T07:37:20-04:00 INFO Nothing to clean up. 2026-06-10 07:37:20.808884 | controller | 2026-06-10T07:37:20-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:37:20.809014 | controller | 2026-06-10T07:37:20-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:37:20.827933 | controller | 2026-06-10T07:37:20-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:37:20.827990 | controller | 2026-06-10T07:37:20-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:37:21.204560 | controller | 2026-06-10T07:37:21-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:37:21.605220 | controller | 2026-06-10T07:37:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:37:21.605345 | controller | 2026-06-10T07:37:21-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 07:37:22.624335 | controller | 2026-06-10T07:37:22-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:37:22.624409 | controller | 2026-06-10T07:37:22-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:37:24.190219 | controller | 2026-06-10T07:37:24-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:37:24.190282 | controller | 2026-06-10T07:37:24-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:37:25.396251 | controller | 2026-06-10T07:37:25-04:00 INFO 2026-06-10 07:37:25.396334 | controller | - Config: OK 2026-06-10 07:37:25.396341 | controller | - Gateway: OK 2026-06-10 07:37:25.396346 | controller | - GitServer: OK 2026-06-10 07:37:25.396350 | controller | - HoundSearch: OK 2026-06-10 07:37:25.396354 | controller | - LogJuicer: OK 2026-06-10 07:37:25.396357 | controller | - Logserver: OK 2026-06-10 07:37:25.396360 | controller | - MariaDB: OK 2026-06-10 07:37:25.396364 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:37:25.396368 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:37:25.396371 | controller | - Zookeeper: OK 2026-06-10 07:37:25.396375 | controller | - Zuul: OK 2026-06-10 07:37:25.396379 | controller | 2026-06-10 07:37:25.396391 | controller | 2026-06-10T07:37:25-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:37:30.404587 | controller | 2026-06-10T07:37:30-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:37:30.407149 | controller | 2026-06-10T07:37:30-04:00 INFO Nothing to clean up. 2026-06-10 07:37:31.026320 | controller | 2026-06-10T07:37:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:37:31.026446 | controller | 2026-06-10T07:37:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:37:31.052824 | controller | 2026-06-10T07:37:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:37:31.052934 | controller | 2026-06-10T07:37:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:37:31.436603 | controller | 2026-06-10T07:37:31-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:37:31.819089 | controller | 2026-06-10T07:37:31-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:37:31.819280 | controller | 2026-06-10T07:37:31-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 07:37:32.826730 | controller | 2026-06-10T07:37:32-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:37:32.826833 | controller | 2026-06-10T07:37:32-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:37:34.408153 | controller | 2026-06-10T07:37:34-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:37:34.408230 | controller | 2026-06-10T07:37:34-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:37:35.616868 | controller | 2026-06-10T07:37:35-04:00 INFO 2026-06-10 07:37:35.616917 | controller | - Config: OK 2026-06-10 07:37:35.616924 | controller | - Gateway: OK 2026-06-10 07:37:35.616930 | controller | - GitServer: OK 2026-06-10 07:37:35.616937 | controller | - HoundSearch: OK 2026-06-10 07:37:35.616943 | controller | - LogJuicer: OK 2026-06-10 07:37:35.616948 | controller | - Logserver: OK 2026-06-10 07:37:35.616953 | controller | - MariaDB: OK 2026-06-10 07:37:35.616958 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:37:35.616964 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:37:35.616969 | controller | - Zookeeper: OK 2026-06-10 07:37:35.616974 | controller | - Zuul: OK 2026-06-10 07:37:35.616978 | controller | 2026-06-10 07:37:35.616983 | controller | 2026-06-10T07:37:35-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:37:40.627994 | controller | 2026-06-10T07:37:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:37:40.630542 | controller | 2026-06-10T07:37:40-04:00 INFO Nothing to clean up. 2026-06-10 07:37:41.245630 | controller | 2026-06-10T07:37:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:37:41.245673 | controller | 2026-06-10T07:37:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:37:41.266395 | controller | 2026-06-10T07:37:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:37:41.266435 | controller | 2026-06-10T07:37:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:37:41.650873 | controller | 2026-06-10T07:37:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:37:42.043967 | controller | 2026-06-10T07:37:42-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:37:42.044048 | controller | 2026-06-10T07:37:42-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 07:37:43.057466 | controller | 2026-06-10T07:37:43-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:37:43.057572 | controller | 2026-06-10T07:37:43-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:37:44.631096 | controller | 2026-06-10T07:37:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:37:44.631221 | controller | 2026-06-10T07:37:44-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:37:45.840791 | controller | 2026-06-10T07:37:45-04:00 INFO 2026-06-10 07:37:45.840856 | controller | - Config: OK 2026-06-10 07:37:45.840872 | controller | - Gateway: OK 2026-06-10 07:37:45.840883 | controller | - GitServer: OK 2026-06-10 07:37:45.840895 | controller | - HoundSearch: OK 2026-06-10 07:37:45.840906 | controller | - LogJuicer: OK 2026-06-10 07:37:45.840918 | controller | - Logserver: OK 2026-06-10 07:37:45.840929 | controller | - MariaDB: OK 2026-06-10 07:37:45.840940 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:37:45.840952 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:37:45.840963 | controller | - Zookeeper: OK 2026-06-10 07:37:45.840975 | controller | - Zuul: OK 2026-06-10 07:37:45.840987 | controller | 2026-06-10 07:37:45.840999 | controller | 2026-06-10T07:37:45-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 07:37:50.850042 | controller | 2026-06-10T07:37:50-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:37:50.853064 | controller | 2026-06-10T07:37:50-04:00 INFO Nothing to clean up. 2026-06-10 07:37:51.464005 | controller | 2026-06-10T07:37:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:37:51.464048 | controller | 2026-06-10T07:37:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:37:51.482314 | controller | 2026-06-10T07:37:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:37:51.482356 | controller | 2026-06-10T07:37:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:37:51.880753 | controller | 2026-06-10T07:37:51-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:37:52.263051 | controller | 2026-06-10T07:37:52-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:37:52.263116 | controller | 2026-06-10T07:37:52-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:37:53.290676 | controller | 2026-06-10T07:37:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:37:53.290801 | controller | 2026-06-10T07:37:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:37:54.856668 | controller | 2026-06-10T07:37:54-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:37:54.856773 | controller | 2026-06-10T07:37:54-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:37:56.067119 | controller | 2026-06-10T07:37:56-04:00 INFO 2026-06-10 07:37:56.067210 | controller | - Config: OK 2026-06-10 07:37:56.067224 | controller | - Gateway: OK 2026-06-10 07:37:56.067233 | controller | - GitServer: OK 2026-06-10 07:37:56.067242 | controller | - HoundSearch: OK 2026-06-10 07:37:56.067250 | controller | - LogJuicer: OK 2026-06-10 07:37:56.067258 | controller | - Logserver: OK 2026-06-10 07:37:56.067266 | controller | - MariaDB: OK 2026-06-10 07:37:56.067274 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:37:56.067283 | controller | - NodePoolLauncher: OK 2026-06-10 07:37:56.067291 | controller | - Zookeeper: OK 2026-06-10 07:37:56.067299 | controller | - Zuul: OK 2026-06-10 07:37:56.067308 | controller | 2026-06-10 07:37:56.067318 | controller | 2026-06-10T07:37:56-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 07:38:01.075328 | controller | 2026-06-10T07:38:01-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:38:01.077067 | controller | 2026-06-10T07:38:01-04:00 INFO Nothing to clean up. 2026-06-10 07:38:01.692722 | controller | 2026-06-10T07:38:01-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:38:01.692761 | controller | 2026-06-10T07:38:01-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:38:01.710285 | controller | 2026-06-10T07:38:01-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:38:01.710360 | controller | 2026-06-10T07:38:01-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:38:02.100840 | controller | 2026-06-10T07:38:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:38:02.498686 | controller | 2026-06-10T07:38:02-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:38:02.498785 | controller | 2026-06-10T07:38:02-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:38:03.508585 | controller | 2026-06-10T07:38:03-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:38:03.508914 | controller | 2026-06-10T07:38:03-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:38:05.078987 | controller | 2026-06-10T07:38:05-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:38:05.079027 | controller | 2026-06-10T07:38:05-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:38:06.287812 | controller | 2026-06-10T07:38:06-04:00 INFO 2026-06-10 07:38:06.287881 | controller | - Config: OK 2026-06-10 07:38:06.287898 | controller | - Gateway: OK 2026-06-10 07:38:06.287911 | controller | - GitServer: OK 2026-06-10 07:38:06.287924 | controller | - HoundSearch: OK 2026-06-10 07:38:06.287936 | controller | - LogJuicer: OK 2026-06-10 07:38:06.287948 | controller | - Logserver: OK 2026-06-10 07:38:06.287960 | controller | - MariaDB: OK 2026-06-10 07:38:06.287971 | controller | - NodePoolBuilder: OK 2026-06-10 07:38:06.287983 | controller | - NodePoolLauncher: OK 2026-06-10 07:38:06.287995 | controller | - Zookeeper: OK 2026-06-10 07:38:06.288006 | controller | - Zuul: OK 2026-06-10 07:38:06.288017 | controller | 2026-06-10 07:38:06.288029 | controller | 2026-06-10T07:38:06-04:00 INFO Updating controller configmap ... 2026-06-10 07:38:06.678465 | controller | 2026-06-10T07:38:06-04:00 INFO Standalone reconcile done. 2026-06-10 11:38:07.009913 | controller | ok: Runtime: 0:00:57.513634 2026-06-10 11:38:07.021605 | 2026-06-10 11:38:07.021739 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:38:07.647921 | controller | skipping: Conditional result was False 2026-06-10 11:38:07.677097 | 2026-06-10 11:38:07.677381 | TASK [health-check/test-nodepool-providers-secrets : Check that clouds.yaml available in nodepool-builder] 2026-06-10 07:38:08.045526 | controller | Defaulted container "nodepool-builder" out of: nodepool-builder, fluentbit, np-statsd, nodepool-builder-nodeexporter, build-logs-httpd, nodepool-builder-init (init) 2026-06-10 07:38:08.165543 | controller | /var/lib/nodepool/.config/openstack/clouds.yaml 2026-06-10 11:38:08.224995 | controller | ok: Runtime: 0:00:00.215775 2026-06-10 11:38:08.234233 | 2026-06-10 11:38:08.234417 | TASK [health-check/test-nodepool-providers-secrets : Check that clouds.yaml available in nodepool-launcher containers] 2026-06-10 07:38:08.744310 | controller | Defaulted container "launcher" out of: launcher, np-statsd, zuul-capacity, nodepool-launcher-init (init) 2026-06-10 07:38:08.770343 | controller | error: Internal error occurred: unable to upgrade connection: container not found ("launcher") 2026-06-10 11:38:25.013468 | controller | ok: Runtime: 0:00:00.345825 2026-06-10 11:38:25.031685 | 2026-06-10 11:38:25.031824 | TASK [health-check/config-update-nodepool-launcher : Set random value] 2026-06-10 11:38:25.126402 | controller | ok 2026-06-10 11:38:25.133758 | 2026-06-10 11:38:25.133851 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool directory exists] 2026-06-10 11:38:25.422645 | controller | changed 2026-06-10 11:38:25.437152 | 2026-06-10 11:38:25.437301 | TASK [health-check/config-update-nodepool-launcher : Set bad Nodepool config] 2026-06-10 11:38:25.697690 | controller | changed: File created 2026-06-10 11:38:25.707957 | 2026-06-10 11:38:25.708120 | LOOP [health-check/config-update-nodepool-launcher : Commit file in the config repo] 2026-06-10 11:38:26.015132 | controller | ok: Item: git add nodepool/nodepool.yaml Runtime: 0:00:00.006825 2026-06-10 11:38:26.304500 | controller | [master 12fa3ad] Add nodepool/nodepool.yaml 2026-06-10 11:38:26.304615 | controller | 1 file changed, 3 insertions(+) 2026-06-10 11:38:26.304638 | controller | create mode 100644 nodepool/nodepool.yaml 2026-06-10 11:38:26.304665 | controller | ok: Item: git commit -m 'Add nodepool/nodepool.yaml' Runtime: 0:00:00.059910 2026-06-10 11:38:26.314955 | 2026-06-10 11:38:26.315162 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 11:38:26.337463 | controller | ok 2026-06-10 11:38:26.365065 | 2026-06-10 11:38:26.365174 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 11:38:26.387932 | controller | ok 2026-06-10 11:38:26.393863 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 11:38:26.399187 | 2026-06-10 11:38:26.399248 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 07:38:27.273992 | controller | remote: 2026-06-10 07:38:27.274088 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:38:27.274104 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:38:27.274117 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:38:27.274133 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 07:38:27.274145 | controller | remote: 2026-06-10 07:38:27.274156 | controller | remote: SUCCESS 2026-06-10 07:38:27.274196 | controller | remote: 2026-06-10 07:38:27.274216 | controller | remote: https://gerrit.sfop.me/c/config/+/5 Add nodepool/nodepool.yaml [NEW] 2026-06-10 07:38:27.274228 | controller | remote: 2026-06-10 07:38:27.274239 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 07:38:27.274253 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:38:27.428797 | controller | ok: Runtime: 0:00:00.625846 2026-06-10 11:38:27.435129 | 2026-06-10 11:38:27.435218 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 07:38:27.733688 | controller | I862f59171646985a9d7629b785ab8831fb3e2257 2026-06-10 11:38:27.970758 | controller | ok: Runtime: 0:00:00.009555 2026-06-10 11:38:27.980648 | 2026-06-10 11:38:27.980784 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 11:38:28.560500 | controller | ok: OK (47 bytes) 2026-06-10 11:38:28.566761 | 2026-06-10 11:38:28.566826 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 11:38:28.596072 | controller | ok 2026-06-10 11:38:28.603493 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 11:38:28.609979 | 2026-06-10 11:38:28.610042 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 11:38:29.106535 | controller | ok: OK (7150 bytes) 2026-06-10 11:38:29.111818 | 2026-06-10 11:38:29.111882 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 11:38:29.162575 | controller | ok 2026-06-10 11:38:29.170757 | 2026-06-10 11:38:29.170832 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 11:39:22.179432 | controller | ok: OK (1946 bytes) 2026-06-10 11:39:22.192005 | 2026-06-10 11:39:22.192155 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 11:39:22.222185 | controller | skipping: Conditional result was False 2026-06-10 11:39:22.236847 | 2026-06-10 11:39:22.236995 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 11:39:22.264719 | controller | skipping: Conditional result was False 2026-06-10 11:39:22.279384 | 2026-06-10 11:39:22.279534 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 11:39:22.307719 | controller | skipping: Conditional result was False 2026-06-10 11:39:22.328196 | 2026-06-10 11:39:22.328343 | TASK [health-check/config-update-nodepool-launcher : Reset the config repo] 2026-06-10 07:39:22.583883 | controller | HEAD is now at 34d1bce Add zuul/main.yaml 2026-06-10 11:39:22.872414 | controller | ok: Runtime: 0:00:00.005723 2026-06-10 11:39:22.882475 | 2026-06-10 11:39:22.882607 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool directory exists] 2026-06-10 11:39:23.183728 | controller | changed 2026-06-10 11:39:23.195602 | 2026-06-10 11:39:23.195735 | TASK [health-check/config-update-nodepool-launcher : Check nodepool.yaml exists] 2026-06-10 11:39:23.479620 | controller | ok 2026-06-10 11:39:23.490052 | 2026-06-10 11:39:23.490194 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool.yaml default content] 2026-06-10 11:39:24.196567 | controller | changed 2026-06-10 11:39:24.207964 | 2026-06-10 11:39:24.208120 | TASK [health-check/config-update-nodepool-launcher : Load current nodepool launcher config] 2026-06-10 07:39:24.598345 | controller | labels: [] 2026-06-10 07:39:24.598408 | controller | providers: [] 2026-06-10 11:39:24.859489 | controller | ok: Runtime: 0:00:00.003693 2026-06-10 11:39:24.870067 | 2026-06-10 11:39:24.870224 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 11:39:24.919251 | controller | ok 2026-06-10 11:39:24.929123 | 2026-06-10 11:39:24.929285 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 11:39:24.980072 | controller | ok 2026-06-10 11:39:24.989426 | 2026-06-10 11:39:24.989567 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 11:39:25.017910 | controller | skipping: Conditional result was False 2026-06-10 11:39:25.030924 | 2026-06-10 11:39:25.031077 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 11:39:25.080149 | controller | ok 2026-06-10 11:39:25.089873 | 2026-06-10 11:39:25.090013 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 11:39:25.118331 | controller | skipping: Conditional result was False 2026-06-10 11:39:25.131714 | 2026-06-10 11:39:25.131996 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 11:39:25.180710 | controller | ok 2026-06-10 11:39:25.190557 | 2026-06-10 11:39:25.190689 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 11:39:25.257305 | controller | ok 2026-06-10 11:39:25.266798 | 2026-06-10 11:39:25.266941 | TASK [health-check/config-update-nodepool-launcher : Set dummy Nodepool config] 2026-06-10 11:39:25.977004 | controller | changed 2026-06-10 11:39:25.986537 | 2026-06-10 11:39:25.986678 | LOOP [health-check/config-update-nodepool-launcher : Commit file in the config repo] 2026-06-10 11:39:26.288597 | controller | ok: Item: git add nodepool/nodepool.yaml Runtime: 0:00:00.007828 2026-06-10 11:39:26.289096 | controller | changed: All items complete 2026-06-10 11:39:26.289155 | 2026-06-10 11:39:26.541131 | controller | [master 2d85b4d] Add nodepool/nodepool.yaml 2026-06-10 11:39:26.541311 | controller | 1 file changed, 12 insertions(+) 2026-06-10 11:39:26.541396 | controller | create mode 100644 nodepool/nodepool.yaml 2026-06-10 11:39:26.541455 | controller | ok: Item: git commit -m 'Add nodepool/nodepool.yaml' Runtime: 0:00:00.056541 2026-06-10 11:39:26.561340 | 2026-06-10 11:39:26.561629 | TASK [Update config repo change] 2026-06-10 11:39:26.599758 | controller | ok 2026-06-10 11:39:26.639010 | 2026-06-10 11:39:26.639107 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 11:39:26.674887 | controller | ok 2026-06-10 11:39:26.691062 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 11:39:26.700204 | 2026-06-10 11:39:26.700292 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 07:39:27.398266 | controller | remote: 2026-06-10 07:39:27.398346 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:39:27.398355 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:39:27.398362 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:39:27.398370 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 07:39:27.398376 | controller | remote: 2026-06-10 07:39:27.398382 | controller | remote: SUCCESS 2026-06-10 07:39:27.398387 | controller | remote: 2026-06-10 07:39:27.398393 | controller | remote: https://gerrit.sfop.me/c/config/+/6 Add nodepool/nodepool.yaml [NEW] 2026-06-10 07:39:27.398398 | controller | remote: 2026-06-10 07:39:27.398403 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 07:39:27.398411 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:39:27.735987 | controller | ok: Runtime: 0:00:00.472817 2026-06-10 11:39:27.745641 | 2026-06-10 11:39:27.745773 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 07:39:28.006717 | controller | Iacf7cb763862ef642c2be504aa386986ba99e832 2026-06-10 11:39:28.292737 | controller | ok: Runtime: 0:00:00.009511 2026-06-10 11:39:28.302916 | 2026-06-10 11:39:28.303057 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 11:39:28.846099 | controller | ok: OK (47 bytes) 2026-06-10 11:39:28.859036 | 2026-06-10 11:39:28.859199 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 11:39:28.888871 | controller | skipping: Conditional result was False 2026-06-10 11:39:28.901731 | 2026-06-10 11:39:28.901872 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 11:39:28.929852 | controller | skipping: Conditional result was False 2026-06-10 11:39:28.944829 | 2026-06-10 11:39:28.944973 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 11:39:28.995938 | controller | ok 2026-06-10 11:39:29.007803 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 11:39:29.020602 | 2026-06-10 11:39:29.020685 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-10 07:39:29.304773 | controller | 2d85b4df99ffe2a683f4c691e7c6e224b7a2eb28 2026-06-10 11:39:29.556902 | controller | ok: Runtime: 0:00:00.007929 2026-06-10 11:39:29.568874 | 2026-06-10 11:39:29.569020 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 11:42:07.545739 | controller | ok: OK (1143 bytes) 2026-06-10 11:42:07.558793 | 2026-06-10 11:42:07.558944 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 11:42:07.595935 | controller | skipping: Conditional result was False 2026-06-10 11:42:07.617270 | 2026-06-10 11:42:07.617442 | TASK [health-check/config-update-nodepool-launcher : Get nodepool-launcher pod name] 2026-06-10 07:42:08.018285 | controller | nodepool-launcher-5cd9886968-w7mwx 2026-06-10 11:42:08.165755 | controller | ok: Runtime: 0:00:00.106662 2026-06-10 11:42:08.178956 | 2026-06-10 11:42:08.179116 | LOOP [health-check/config-update-nodepool-launcher : Check that new zuul-worker-sf-operator-ci-5v7p8a7d label exists in the destination file] 2026-06-10 11:42:08.698525 | controller | - name: zuul-worker-sf-operator-ci-5v7p8a7d 2026-06-10 11:42:08.699287 | controller | name: zuul-worker-sf-operator-ci-5v7p8a7d 2026-06-10 11:42:08.699435 | controller | ok: Item: zuul-worker-sf-operator-ci-5v7p8a7d Runtime: 0:00:00.211626 2026-06-10 11:42:09.158179 | controller | name: microshiftLocal 2026-06-10 11:42:09.158423 | controller | ok: Item: microshiftLocal Runtime: 0:00:00.218473 2026-06-10 11:42:09.177996 | 2026-06-10 11:42:09.178171 | TASK [health-check/config-update-nodepool-launcher : Ensure the new label listed in the Nodepool API] 2026-06-10 11:42:09.758771 | controller | ok: OK (199 bytes) 2026-06-10 11:42:09.773294 | 2026-06-10 11:42:09.773468 | TASK [health-check/config-update-nodepool-launcher : Ensure the new label listed in the Zuul API] 2026-06-10 11:42:10.336944 | controller | ok: OK (49 bytes) 2026-06-10 11:42:10.393308 | 2026-06-10 11:42:10.393602 | TASK [health-check/test-deploy-zero-replica : Set executor replica set to 0] 2026-06-10 07:42:10.771494 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 11:42:10.944964 | controller | ok: Runtime: 0:00:00.087591 2026-06-10 11:42:10.954513 | 2026-06-10 11:42:10.954645 | TASK [health-check/test-deploy-zero-replica : Ensure executor pod is down] 2026-06-10 07:42:11.466796 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 07:42:11.466911 | controller | config-base-secret-1-2837-ns-q9ctm 0/1 Completed 0 34m 2026-06-10 07:42:11.466930 | controller | config-base-secret-1-917-ns-5jhvl 0/1 Completed 0 40m 2026-06-10 07:42:11.466939 | controller | config-base-secret-1-917-ns-rz8lm 0/1 Completed 0 45m 2026-06-10 07:42:11.466944 | controller | gateway-6bbbd758d9-b58rz 1/1 Running 0 33m 2026-06-10 07:42:11.466954 | controller | gerrit-0 2/2 Running 0 51m 2026-06-10 07:42:11.466961 | controller | git-server-0 2/2 Running 0 17m 2026-06-10 07:42:11.466967 | controller | hound-search-0 1/1 Running 0 50m 2026-06-10 07:42:11.466971 | controller | logjuicer-6ff84764b7-29479 1/1 Running 0 50m 2026-06-10 07:42:11.466976 | controller | logserver-0 4/4 Running 0 28m 2026-06-10 07:42:11.466981 | controller | mariadb-0 3/3 Running 0 26m 2026-06-10 07:42:11.466986 | controller | nodepool-builder-0 5/5 Running 0 4m59s 2026-06-10 07:42:11.466995 | controller | nodepool-launcher-5cd9886968-w7mwx 3/3 Running 0 5m 2026-06-10 07:42:11.467000 | controller | post-init-97qhg 0/1 Completed 0 50m 2026-06-10 07:42:11.467006 | controller | test-fluentbit 1/1 Running 0 43m 2026-06-10 07:42:11.467010 | controller | test-loki-8cd4fd56-2v44p 1/1 Running 0 43m 2026-06-10 07:42:11.467015 | controller | zookeeper-0 3/3 Running 0 25m 2026-06-10 07:42:11.467019 | controller | zookeeper-1 3/3 Running 0 25m 2026-06-10 07:42:11.467023 | controller | zookeeper-2 3/3 Running 1 (26m ago) 26m 2026-06-10 07:42:11.467027 | controller | zuul-db-provision-p79wx 0/1 Completed 0 36m 2026-06-10 07:42:11.467031 | controller | zuul-db-provision-zlqvm 0/1 Completed 0 49m 2026-06-10 07:42:11.467036 | controller | zuul-executor-0 2/2 Terminating 0 5m48s 2026-06-10 07:42:11.467040 | controller | zuul-merger-0 2/2 Running 0 16m 2026-06-10 07:42:11.467044 | controller | zuul-merger-1 2/2 Running 0 16m 2026-06-10 07:42:11.467048 | controller | zuul-scheduler-0 3/3 Running 0 12m 2026-06-10 07:42:11.467052 | controller | zuul-web-5bdd465c4c-hv8z5 1/1 Running 0 12m 2026-06-10 07:42:11.467057 | controller | zuul-weeder-5fb87885d-qwtrc 1/1 Running 0 15m 2026-06-10 11:42:21.888202 | controller | ok: Runtime: 0:00:00.130586 2026-06-10 11:42:21.898768 | 2026-06-10 11:42:21.898898 | TASK [health-check/test-deploy-zero-replica : Set zuul-web replica set to 0] 2026-06-10 07:42:22.243131 | controller | deployment.apps/zuul-web scaled 2026-06-10 11:42:22.443685 | controller | ok: Runtime: 0:00:00.064187 2026-06-10 11:42:22.453295 | 2026-06-10 11:42:22.453457 | TASK [health-check/test-deploy-zero-replica : Ensure zuul-web pod is down] 2026-06-10 07:42:22.832245 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 07:42:22.832329 | controller | config-base-secret-1-2837-ns-q9ctm 0/1 Completed 0 35m 2026-06-10 07:42:22.832341 | controller | config-base-secret-1-917-ns-5jhvl 0/1 Completed 0 40m 2026-06-10 07:42:22.832355 | controller | config-base-secret-1-917-ns-rz8lm 0/1 Completed 0 45m 2026-06-10 07:42:22.832362 | controller | gateway-6bbbd758d9-b58rz 1/1 Running 0 33m 2026-06-10 07:42:22.832368 | controller | gerrit-0 2/2 Running 0 52m 2026-06-10 07:42:22.832375 | controller | git-server-0 2/2 Running 0 17m 2026-06-10 07:42:22.832381 | controller | hound-search-0 1/1 Running 0 50m 2026-06-10 07:42:22.832386 | controller | logjuicer-6ff84764b7-29479 1/1 Running 0 50m 2026-06-10 07:42:22.832392 | controller | logserver-0 4/4 Running 0 28m 2026-06-10 07:42:22.832406 | controller | mariadb-0 3/3 Running 0 26m 2026-06-10 07:42:22.832412 | controller | nodepool-builder-0 5/5 Running 0 5m10s 2026-06-10 07:42:22.832422 | controller | nodepool-launcher-5cd9886968-w7mwx 3/3 Running 0 5m11s 2026-06-10 07:42:22.832427 | controller | post-init-97qhg 0/1 Completed 0 50m 2026-06-10 07:42:22.832433 | controller | test-fluentbit 1/1 Running 0 43m 2026-06-10 07:42:22.832438 | controller | test-loki-8cd4fd56-2v44p 1/1 Running 0 43m 2026-06-10 07:42:22.832443 | controller | zookeeper-0 3/3 Running 0 25m 2026-06-10 07:42:22.832448 | controller | zookeeper-1 3/3 Running 0 25m 2026-06-10 07:42:22.832453 | controller | zookeeper-2 3/3 Running 1 (26m ago) 26m 2026-06-10 07:42:22.832459 | controller | zuul-db-provision-p79wx 0/1 Completed 0 37m 2026-06-10 07:42:22.832464 | controller | zuul-db-provision-zlqvm 0/1 Completed 0 49m 2026-06-10 07:42:22.832469 | controller | zuul-merger-0 2/2 Running 0 16m 2026-06-10 07:42:22.832474 | controller | zuul-merger-1 2/2 Running 0 16m 2026-06-10 07:42:22.832479 | controller | zuul-scheduler-0 3/3 Running 0 12m 2026-06-10 07:42:22.832487 | controller | zuul-web-5bdd465c4c-hv8z5 1/1 Terminating 0 12m 2026-06-10 07:42:22.832493 | controller | zuul-weeder-5fb87885d-qwtrc 1/1 Running 0 15m 2026-06-10 11:42:33.222182 | controller | ok: Runtime: 0:00:00.114542 2026-06-10 11:42:33.232116 | 2026-06-10 11:42:33.232253 | TASK [health-check/test-deploy-zero-replica : Run sf-operator and expect reconciliation] 2026-06-10 07:42:33.753307 | controller | 2026-06-10T07:42:33-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:42:33.753399 | controller | 2026-06-10 07:42:33.763557 | controller | 2026-06-10T07:42:33-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:42:33.763616 | controller | 2026-06-10 07:42:33.774699 | controller | 2026-06-10T07:42:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:42:33.776349 | controller | 2026-06-10T07:42:33-04:00 INFO Nothing to clean up. 2026-06-10 07:42:33.809638 | controller | 2026-06-10T07:42:33-04:00 INFO Statefulset "mariadb" configuration changed, applying... 2026-06-10 07:42:33.809712 | controller | 2026-06-10T07:42:33-04:00 INFO Updating object, name: mariadb 2026-06-10 07:42:34.191508 | controller | 2026-06-10T07:42:34-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:42:34.345346 | controller | 2026-06-10T07:42:34-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:42:34.353948 | controller | 2026-06-10T07:42:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:42:34.354014 | controller | 2026-06-10T07:42:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:42:34.375298 | controller | 2026-06-10T07:42:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:42:34.375365 | controller | 2026-06-10T07:42:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:42:34.596971 | controller | 2026-06-10T07:42:34-04:00 INFO Statefulset "zookeeper" configuration changed, applying... 2026-06-10 07:42:34.597550 | controller | 2026-06-10T07:42:34-04:00 INFO Updating object, name: zookeeper 2026-06-10 07:42:34.607140 | controller | 2026-06-10T07:42:34-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-10 07:42:34.607251 | controller | 2026-06-10T07:42:34-04:00 INFO 2026-06-10 07:42:34.607263 | controller | - Config: Waiting ... 2026-06-10 07:42:34.607274 | controller | - Gateway: OK 2026-06-10 07:42:34.607282 | controller | - GitServer: OK 2026-06-10 07:42:34.607290 | controller | - HoundSearch: OK 2026-06-10 07:42:34.607298 | controller | - LogJuicer: OK 2026-06-10 07:42:34.607306 | controller | - Logserver: OK 2026-06-10 07:42:34.607314 | controller | - MariaDB: Waiting ... 2026-06-10 07:42:34.607322 | controller | - Zookeeper: Waiting ... 2026-06-10 07:42:34.607330 | controller | 2026-06-10 07:42:34.607338 | controller | 2026-06-10T07:42:34-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:42:39.612840 | controller | 2026-06-10T07:42:39-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:42:39.614774 | controller | 2026-06-10T07:42:39-04:00 INFO Nothing to clean up. 2026-06-10 07:42:39.742745 | controller | 2026-06-10T07:42:39-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-10 07:42:49.834689 | controller | 2026-06-10T07:42:49-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:42:49.834761 | controller | 2026-06-10T07:42:49-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:42:49.855957 | controller | 2026-06-10T07:42:49-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:42:49.856037 | controller | 2026-06-10T07:42:49-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:42:49.885961 | controller | 2026-06-10T07:42:49-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-10 07:43:26.041881 | controller | 2026-06-10T07:43:26-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:43:26.053599 | controller | 2026-06-10T07:43:26-04:00 INFO Updating configmap, name: nodepool-launcher-extra-config-config-map 2026-06-10 07:43:26.053637 | controller | 2026-06-10T07:43:26-04:00 INFO Updating object, name: nodepool-launcher-extra-config-config-map 2026-06-10 07:43:26.068662 | controller | 2026-06-10T07:43:26-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-10 07:43:26.068727 | controller | 2026-06-10T07:43:26-04:00 INFO Updating object, name: nodepool-launcher 2026-06-10 07:43:26.088945 | controller | 2026-06-10T07:43:26-04:00 INFO Updating configmap, name: nodepool-builder-extra-config-config-map 2026-06-10 07:43:26.089005 | controller | 2026-06-10T07:43:26-04:00 INFO Updating object, name: nodepool-builder-extra-config-config-map 2026-06-10 07:43:26.106510 | controller | 2026-06-10T07:43:26-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-10 07:43:26.106584 | controller | 2026-06-10T07:43:26-04:00 INFO Updating object, name: nodepool-builder 2026-06-10 07:43:26.141416 | controller | 2026-06-10T07:43:26-04:00 INFO Updating secret, name: zuul-config 2026-06-10 07:43:26.141499 | controller | 2026-06-10T07:43:26-04:00 INFO Updating object, name: zuul-config 2026-06-10 07:43:26.650281 | controller | 2026-06-10T07:43:26-04:00 INFO Updating configmap, name: zuul-logging-config-map 2026-06-10 07:43:26.650357 | controller | 2026-06-10T07:43:26-04:00 INFO Updating object, name: zuul-logging-config-map 2026-06-10 07:43:26.856332 | controller | 2026-06-10T07:43:26-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 07:43:26.856420 | controller | 2026-06-10T07:43:26-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 07:43:27.653225 | controller | 2026-06-10T07:43:27-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 07:43:27.653291 | controller | 2026-06-10T07:43:27-04:00 INFO Updating object, name: zuul-executor 2026-06-10 07:43:28.453556 | controller | 2026-06-10T07:43:28-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:43:28.453666 | controller | 2026-06-10T07:43:28-04:00 INFO Updating object, name: zuul-web 2026-06-10 07:43:29.253224 | controller | 2026-06-10T07:43:29-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-10 07:43:29.253289 | controller | 2026-06-10T07:43:29-04:00 INFO Updating object, name: zuul-merger 2026-06-10 07:43:29.261109 | controller | 2026-06-10T07:43:29-04:00 INFO 2026-06-10 07:43:29.261162 | controller | - Config: Waiting ... 2026-06-10 07:43:29.261193 | controller | - Gateway: OK 2026-06-10 07:43:29.261198 | controller | - GitServer: OK 2026-06-10 07:43:29.261201 | controller | - HoundSearch: OK 2026-06-10 07:43:29.261206 | controller | - LogJuicer: OK 2026-06-10 07:43:29.261210 | controller | - Logserver: OK 2026-06-10 07:43:29.261214 | controller | - MariaDB: OK 2026-06-10 07:43:29.261217 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:43:29.261221 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:43:29.261225 | controller | - Zookeeper: OK 2026-06-10 07:43:29.261228 | controller | - Zuul: Waiting ... 2026-06-10 07:43:29.261232 | controller | - |_ Config Secrets: OK 2026-06-10 07:43:29.261235 | controller | - |_ Executor: Waiting ... 2026-06-10 07:43:29.261239 | controller | - |_ Merger: Waiting ... 2026-06-10 07:43:29.261243 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:43:29.261246 | controller | - |_ Web: Waiting ... 2026-06-10 07:43:29.261250 | controller | 2026-06-10 07:43:29.261278 | controller | 2026-06-10T07:43:29-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:43:34.269869 | controller | 2026-06-10T07:43:34-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:43:34.272348 | controller | 2026-06-10T07:43:34-04:00 INFO Nothing to clean up. 2026-06-10 07:43:34.887399 | controller | 2026-06-10T07:43:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:43:34.887511 | controller | 2026-06-10T07:43:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:43:34.904481 | controller | 2026-06-10T07:43:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:43:34.904564 | controller | 2026-06-10T07:43:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:43:35.307549 | controller | 2026-06-10T07:43:35-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:43:35.682872 | controller | 2026-06-10T07:43:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:43:35.682933 | controller | 2026-06-10T07:43:35-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 07:43:36.696698 | controller | 2026-06-10T07:43:36-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:44:06.804482 | controller | 2026-06-10T07:44:06-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:44:06.838940 | controller | 2026-06-10T07:44:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:44:07.220505 | controller | 2026-06-10T07:44:07-04:00 INFO Waiting for statefulset, name: zuul-merger 2026-06-10 07:44:29.306394 | controller | 2026-06-10T07:44:29-04:00 INFO 2026-06-10 07:44:29.306466 | controller | - Config: Waiting ... 2026-06-10 07:44:29.306471 | controller | - Gateway: OK 2026-06-10 07:44:29.306491 | controller | - GitServer: OK 2026-06-10 07:44:29.306495 | controller | - HoundSearch: OK 2026-06-10 07:44:29.306499 | controller | - LogJuicer: OK 2026-06-10 07:44:29.306502 | controller | - Logserver: OK 2026-06-10 07:44:29.306506 | controller | - MariaDB: OK 2026-06-10 07:44:29.306509 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:44:29.306523 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:44:29.306527 | controller | - Zookeeper: OK 2026-06-10 07:44:29.306530 | controller | - Zuul: Waiting ... 2026-06-10 07:44:29.306534 | controller | - |_ Config Secrets: OK 2026-06-10 07:44:29.306537 | controller | - |_ Executor: OK 2026-06-10 07:44:29.306541 | controller | - |_ Merger: OK 2026-06-10 07:44:29.306544 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:44:29.306548 | controller | - |_ Web: OK 2026-06-10 07:44:29.306552 | controller | 2026-06-10 07:44:29.306556 | controller | 2026-06-10T07:44:29-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:44:34.316527 | controller | 2026-06-10T07:44:34-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:44:34.319215 | controller | 2026-06-10T07:44:34-04:00 INFO Nothing to clean up. 2026-06-10 07:44:34.933050 | controller | 2026-06-10T07:44:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:44:34.933130 | controller | 2026-06-10T07:44:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:44:34.949802 | controller | 2026-06-10T07:44:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:44:34.949871 | controller | 2026-06-10T07:44:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:44:35.340159 | controller | 2026-06-10T07:44:35-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:44:35.737745 | controller | 2026-06-10T07:44:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:44:35.737839 | controller | 2026-06-10T07:44:35-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:44:36.743637 | controller | 2026-06-10T07:44:36-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-10 07:44:36.743705 | controller | 2026-06-10T07:44:36-04:00 INFO Updating object, name: zuul-weeder 2026-06-10 07:44:37.719712 | controller | 2026-06-10T07:44:37-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:44:38.529509 | controller | 2026-06-10T07:44:38-04:00 INFO 2026-06-10 07:44:38.529596 | controller | - Config: Waiting ... 2026-06-10 07:44:38.529611 | controller | - Gateway: OK 2026-06-10 07:44:38.529624 | controller | - GitServer: OK 2026-06-10 07:44:38.529636 | controller | - HoundSearch: OK 2026-06-10 07:44:38.529648 | controller | - LogJuicer: OK 2026-06-10 07:44:38.529659 | controller | - Logserver: OK 2026-06-10 07:44:38.529671 | controller | - MariaDB: OK 2026-06-10 07:44:38.529681 | controller | - NodePoolBuilder: OK 2026-06-10 07:44:38.529689 | controller | - NodePoolLauncher: OK 2026-06-10 07:44:38.529698 | controller | - Zookeeper: OK 2026-06-10 07:44:38.529706 | controller | - Zuul: Waiting ... 2026-06-10 07:44:38.529751 | controller | - |_ Config Secrets: OK 2026-06-10 07:44:38.529760 | controller | - |_ Executor: OK 2026-06-10 07:44:38.529768 | controller | - |_ Merger: OK 2026-06-10 07:44:38.529777 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:44:38.529786 | controller | - |_ Web: OK 2026-06-10 07:44:38.529795 | controller | 2026-06-10 07:44:38.529803 | controller | 2026-06-10T07:44:38-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 07:44:43.538834 | controller | 2026-06-10T07:44:43-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:44:43.540992 | controller | 2026-06-10T07:44:43-04:00 INFO Nothing to clean up. 2026-06-10 07:44:44.155205 | controller | 2026-06-10T07:44:44-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:44:44.155265 | controller | 2026-06-10T07:44:44-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:44:44.169112 | controller | 2026-06-10T07:44:44-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:44:44.169198 | controller | 2026-06-10T07:44:44-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:44:44.559921 | controller | 2026-06-10T07:44:44-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:44:44.955211 | controller | 2026-06-10T07:44:44-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:44:44.955287 | controller | 2026-06-10T07:44:44-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:44:45.961997 | controller | 2026-06-10T07:44:45-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:44:45.962079 | controller | 2026-06-10T07:44:45-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:44:46.942639 | controller | 2026-06-10T07:44:46-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:44:48.152280 | controller | 2026-06-10T07:44:48-04:00 INFO 2026-06-10 07:44:48.152358 | controller | - Config: OK 2026-06-10 07:44:48.152386 | controller | - Gateway: OK 2026-06-10 07:44:48.152395 | controller | - GitServer: OK 2026-06-10 07:44:48.152436 | controller | - HoundSearch: OK 2026-06-10 07:44:48.152445 | controller | - LogJuicer: OK 2026-06-10 07:44:48.152452 | controller | - Logserver: OK 2026-06-10 07:44:48.152460 | controller | - MariaDB: OK 2026-06-10 07:44:48.152468 | controller | - NodePoolBuilder: OK 2026-06-10 07:44:48.152476 | controller | - NodePoolLauncher: OK 2026-06-10 07:44:48.152484 | controller | - Zookeeper: OK 2026-06-10 07:44:48.152491 | controller | - Zuul: OK 2026-06-10 07:44:48.152500 | controller | 2026-06-10 07:44:48.152508 | controller | 2026-06-10T07:44:48-04:00 INFO Updating controller configmap ... 2026-06-10 07:44:48.541542 | controller | 2026-06-10T07:44:48-04:00 INFO Standalone reconcile done. 2026-06-10 11:44:48.942192 | controller | ok: Runtime: 0:02:15.064806 2026-06-10 11:44:48.953297 | 2026-06-10 11:44:48.953461 | TASK [health-check/test-deploy-zero-replica : Set executor replica set to 1] 2026-06-10 07:44:49.321236 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 11:44:49.500817 | controller | ok: Runtime: 0:00:00.087696 2026-06-10 11:44:49.510112 | 2026-06-10 11:44:49.510245 | TASK [health-check/test-deploy-zero-replica : Set web replica set to 1] 2026-06-10 07:44:49.839345 | controller | deployment.apps/zuul-web scaled 2026-06-10 11:44:50.057858 | controller | ok: Runtime: 0:00:00.098027 2026-06-10 11:44:50.066988 | 2026-06-10 11:44:50.067124 | TASK [health-check/test-deploy-zero-replica : Ensure executor pod is up] 2026-06-10 07:44:50.394064 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 07:44:50.394188 | controller | zuul-executor-0 0/2 ContainerCreating 0 1s 2026-06-10 11:45:12.522504 | controller | ok: Runtime: 0:00:00.071620 2026-06-10 11:45:12.532012 | 2026-06-10 11:45:12.532138 | TASK [health-check/test-deploy-zero-replica : Ensure web pod is up] 2026-06-10 07:45:13.482994 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 07:45:13.483087 | controller | gateway-6bbbd758d9-b58rz 1/1 Running 0 36m 2026-06-10 07:45:13.483122 | controller | gerrit-0 2/2 Running 0 54m 2026-06-10 07:45:13.483134 | controller | git-server-0 2/2 Running 0 20m 2026-06-10 07:45:13.483146 | controller | hound-search-0 1/1 Running 0 53m 2026-06-10 07:45:13.483229 | controller | logjuicer-6ff84764b7-29479 1/1 Running 0 53m 2026-06-10 07:45:13.483248 | controller | logserver-0 4/4 Running 0 31m 2026-06-10 07:45:13.483257 | controller | mariadb-0 2/2 Running 0 2m38s 2026-06-10 07:45:13.483266 | controller | nodepool-builder-0 4/4 Running 0 106s 2026-06-10 07:45:13.483274 | controller | nodepool-launcher-58bc794b9d-h6pmm 3/3 Running 0 107s 2026-06-10 07:45:13.483285 | controller | test-fluentbit 1/1 Running 0 46m 2026-06-10 07:45:13.483296 | controller | test-loki-8cd4fd56-2v44p 1/1 Running 0 46m 2026-06-10 07:45:13.483308 | controller | zookeeper-0 2/2 Running 0 2m3s 2026-06-10 07:45:13.483319 | controller | zookeeper-1 2/2 Running 0 2m20s 2026-06-10 07:45:13.483336 | controller | zookeeper-2 2/2 Running 0 2m37s 2026-06-10 07:45:13.483349 | controller | zuul-executor-0 2/2 Running 0 24s 2026-06-10 07:45:13.483360 | controller | zuul-merger-0 2/2 Running 0 58s 2026-06-10 07:45:13.483372 | controller | zuul-merger-1 2/2 Running 0 102s 2026-06-10 07:45:13.483383 | controller | zuul-scheduler-0 3/3 Running 0 104s 2026-06-10 07:45:13.483395 | controller | zuul-web-5779988f86-mkb2t 0/1 Running 0 24s 2026-06-10 07:45:13.483407 | controller | zuul-weeder-57c6567f5f-9qkgc 1/1 Running 0 37s 2026-06-10 11:45:13.730587 | controller | ok: Runtime: 0:00:00.097753 2026-06-10 11:45:13.741277 | 2026-06-10 11:45:13.741438 | TASK [Run dry-run update detection test] 2026-06-10 11:45:13.778911 | controller | ok 2026-06-10 11:45:13.816647 | 2026-06-10 11:45:13.816848 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-executor] 2026-06-10 11:45:14.355407 | controller | ok: Runtime: 0:00:00.079564 2026-06-10 11:45:14.366942 | 2026-06-10 11:45:14.367076 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-web] 2026-06-10 11:45:14.914325 | controller | ok: Runtime: 0:00:00.065779 2026-06-10 11:45:14.928961 | 2026-06-10 11:45:14.929238 | TASK [test-dry-run-update-detection : Patch zuul-executor statefulset template to trigger reconciliation] 2026-06-10 07:45:16.053269 | controller | statefulset.apps/zuul-executor patched 2026-06-10 11:45:16.819250 | controller | ok: Runtime: 0:00:00.082123 2026-06-10 11:45:16.825111 | 2026-06-10 11:45:16.825203 | TASK [test-dry-run-update-detection : Patch zuul-web deployment template to trigger reconciliation] 2026-06-10 07:45:17.138615 | controller | deployment.apps/zuul-web patched 2026-06-10 11:45:18.321456 | controller | ok: Runtime: 0:00:00.105116 2026-06-10 11:45:18.325856 | 2026-06-10 11:45:18.325918 | TASK [Run deploy with --dry-run to check for update detection] 2026-06-10 11:45:18.355181 | controller | ok 2026-06-10 11:45:18.371733 | 2026-06-10 11:45:18.371826 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:45:20.577938 | controller | ok 2026-06-10 11:45:20.584529 | 2026-06-10 11:45:20.584625 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:45:20.844892 | controller | ok 2026-06-10 11:45:20.854578 | 2026-06-10 11:45:20.854730 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:45:21.535777 | controller | 2026-06-10T07:45:21-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:45:21.535862 | controller | 2026-06-10 07:45:21.552126 | controller | 2026-06-10T07:45:21-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:45:21.552248 | controller | 2026-06-10 07:45:21.570563 | controller | 2026-06-10T07:45:21-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-10 07:45:21.572148 | controller | 2026-06-10T07:45:21-04:00 INFO Nothing to clean up. 2026-06-10 07:45:22.183841 | controller | 2026-06-10T07:45:22-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:45:22.296972 | controller | 2026-06-10T07:45:22-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:45:22.595525 | controller | 2026-06-10T07:45:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:45:24.366353 | controller | 2026-06-10T07:45:24-04:00 INFO [Dry Run] Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 07:45:24.366425 | controller | 2026-06-10T07:45:24-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '11' to '0' 2026-06-10 07:45:24.366435 | controller | 2026-06-10T07:45:24-04:00 INFO Updating object, name: zuul-executor (server dry run) 2026-06-10 07:45:25.165133 | controller | 2026-06-10T07:45:25-04:00 INFO [Dry Run] Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:45:25.165245 | controller | 2026-06-10T07:45:25-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 07:45:25.165344 | controller | 2026-06-10T07:45:25-04:00 INFO Updating object, name: zuul-web (server dry run) 2026-06-10 07:45:25.969206 | controller | 2026-06-10T07:45:25-04:00 INFO 2026-06-10 07:45:25.969265 | controller | - Config: Waiting ... 2026-06-10 07:45:25.969271 | controller | - Gateway: OK 2026-06-10 07:45:25.969276 | controller | - GitServer: OK 2026-06-10 07:45:25.969281 | controller | - HoundSearch: OK 2026-06-10 07:45:25.969286 | controller | - LogJuicer: OK 2026-06-10 07:45:25.969290 | controller | - Logserver: OK 2026-06-10 07:45:25.969296 | controller | - MariaDB: OK 2026-06-10 07:45:25.969302 | controller | - NodePoolBuilder: OK 2026-06-10 07:45:25.969306 | controller | - NodePoolLauncher: OK 2026-06-10 07:45:25.969311 | controller | - Zookeeper: OK 2026-06-10 07:45:25.969315 | controller | - Zuul: Waiting ... 2026-06-10 07:45:25.969320 | controller | - |_ Config Secrets: OK 2026-06-10 07:45:25.969325 | controller | - |_ Executor: Waiting ... 2026-06-10 07:45:25.969330 | controller | - |_ Merger: OK 2026-06-10 07:45:25.969334 | controller | - |_ Scheduler: OK 2026-06-10 07:45:25.969339 | controller | - |_ Web: Waiting ... 2026-06-10 07:45:25.969344 | controller | 2026-06-10 07:45:25.969349 | controller | 2026-06-10T07:45:25-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 11:45:26.019980 | controller | ok: Runtime: 0:00:04.752513 2026-06-10 11:45:26.030817 | 2026-06-10 11:45:26.031050 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:45:26.573249 | controller | ok: Runtime: 0:00:00.005970 2026-06-10 11:45:26.597184 | 2026-06-10 11:45:26.597421 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-executor] 2026-06-10 07:45:26.857434 | controller | 2026-06-10T07:45:24-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '11' to '0' 2026-06-10 11:45:27.139718 | controller | ok: Runtime: 0:00:00.004602 2026-06-10 11:45:27.146853 | 2026-06-10 11:45:27.146943 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-web] 2026-06-10 07:45:27.443734 | controller | 2026-06-10T07:45:25-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 11:45:27.677957 | controller | ok: Runtime: 0:00:00.007843 2026-06-10 11:45:27.688246 | 2026-06-10 11:45:27.688417 | TASK [test-dry-run-update-detection : Restore original serial for zuul-executor] 2026-06-10 07:45:28.064495 | controller | statefulset.apps/zuul-executor patched 2026-06-10 11:45:28.234212 | controller | ok: Runtime: 0:00:00.084064 2026-06-10 11:45:28.244749 | 2026-06-10 11:45:28.244884 | TASK [test-dry-run-update-detection : Restore original serial for zuul-web] 2026-06-10 07:45:28.669417 | controller | deployment.apps/zuul-web patched 2026-06-10 11:45:28.787544 | controller | ok: Runtime: 0:00:00.124827 2026-06-10 11:45:28.817010 | 2026-06-10 11:45:28.817171 | TASK [health-check/test-dry-run-lifecycle : Create a temporary directory for generated CRs] 2026-06-10 11:45:29.045421 | controller | changed 2026-06-10 11:45:29.054618 | 2026-06-10 11:45:29.054747 | TASK [health-check/test-dry-run-lifecycle : Generate CR with zuul-executor disabled] 2026-06-10 11:45:29.842843 | controller | changed 2026-06-10 11:45:29.868233 | 2026-06-10 11:45:29.868509 | TASK [Deploy SF without zuul-executor to ensure its removal] 2026-06-10 11:45:29.905413 | controller | ok 2026-06-10 11:45:29.937669 | 2026-06-10 11:45:29.937783 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:45:30.216199 | controller | ok 2026-06-10 11:45:30.225986 | 2026-06-10 11:45:30.226118 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:45:30.458037 | controller | changed 2026-06-10 11:45:30.471154 | 2026-06-10 11:45:30.471308 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:45:31.104285 | controller | 2026-06-10T07:45:31-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:45:31.104422 | controller | 2026-06-10 07:45:31.118802 | controller | 2026-06-10T07:45:31-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:45:31.118864 | controller | 2026-06-10 07:45:31.137982 | controller | 2026-06-10T07:45:31-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:45:31.140815 | controller | 2026-06-10T07:45:31-04:00 INFO Nothing to clean up. 2026-06-10 07:45:31.757818 | controller | 2026-06-10T07:45:31-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:45:31.880943 | controller | 2026-06-10T07:45:31-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:45:31.892472 | controller | 2026-06-10T07:45:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:45:31.892545 | controller | 2026-06-10T07:45:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:45:31.912378 | controller | 2026-06-10T07:45:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:45:31.912420 | controller | 2026-06-10T07:45:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:45:32.167231 | controller | 2026-06-10T07:45:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:45:32.553478 | controller | 2026-06-10T07:45:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:45:32.553532 | controller | 2026-06-10T07:45:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:45:32.761860 | controller | 2026-06-10T07:45:32-04:00 INFO zuul-executor is disabled but running. Deleting the executor ... 2026-06-10 07:45:32.761918 | controller | 2026-06-10T07:45:32-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 07:45:33.575118 | controller | 2026-06-10T07:45:33-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:45:33.575213 | controller | 2026-06-10T07:45:33-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:45:33.937095 | controller | 2026-06-10T07:45:33-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:45:33.937151 | controller | 2026-06-10T07:45:33-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:45:34.745017 | controller | 2026-06-10T07:45:34-04:00 INFO 2026-06-10 07:45:34.745070 | controller | - Config: Waiting ... 2026-06-10 07:45:34.745080 | controller | - Gateway: OK 2026-06-10 07:45:34.745089 | controller | - GitServer: OK 2026-06-10 07:45:34.745097 | controller | - HoundSearch: OK 2026-06-10 07:45:34.745106 | controller | - LogJuicer: OK 2026-06-10 07:45:34.745114 | controller | - Logserver: OK 2026-06-10 07:45:34.745122 | controller | - MariaDB: OK 2026-06-10 07:45:34.745130 | controller | - NodePoolBuilder: OK 2026-06-10 07:45:34.745138 | controller | - NodePoolLauncher: OK 2026-06-10 07:45:34.745146 | controller | - Zookeeper: OK 2026-06-10 07:45:34.745154 | controller | - Zuul: Waiting ... 2026-06-10 07:45:34.745162 | controller | - |_ Config Secrets: OK 2026-06-10 07:45:34.745210 | controller | - |_ Merger: OK 2026-06-10 07:45:34.745219 | controller | - |_ Scheduler: OK 2026-06-10 07:45:34.745227 | controller | - |_ Web: Waiting ... 2026-06-10 07:45:34.745236 | controller | 2026-06-10 07:45:34.745244 | controller | 2026-06-10T07:45:34-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:45:39.754738 | controller | 2026-06-10T07:45:39-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:45:39.756686 | controller | 2026-06-10T07:45:39-04:00 INFO Nothing to clean up. 2026-06-10 07:45:40.378517 | controller | 2026-06-10T07:45:40-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:45:40.378611 | controller | 2026-06-10T07:45:40-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:45:40.405727 | controller | 2026-06-10T07:45:40-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:45:40.405845 | controller | 2026-06-10T07:45:40-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:45:40.776107 | controller | 2026-06-10T07:45:40-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:45:41.172877 | controller | 2026-06-10T07:45:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:45:41.172997 | controller | 2026-06-10T07:45:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:45:42.188231 | controller | 2026-06-10T07:45:42-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:45:42.188341 | controller | 2026-06-10T07:45:42-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:45:42.362564 | controller | 2026-06-10T07:45:42-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:45:42.362635 | controller | 2026-06-10T07:45:42-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:45:43.168402 | controller | 2026-06-10T07:45:43-04:00 INFO 2026-06-10 07:45:43.168459 | controller | - Config: Waiting ... 2026-06-10 07:45:43.168469 | controller | - Gateway: OK 2026-06-10 07:45:43.168477 | controller | - GitServer: OK 2026-06-10 07:45:43.168484 | controller | - HoundSearch: OK 2026-06-10 07:45:43.168491 | controller | - LogJuicer: OK 2026-06-10 07:45:43.168498 | controller | - Logserver: OK 2026-06-10 07:45:43.168505 | controller | - MariaDB: OK 2026-06-10 07:45:43.168511 | controller | - NodePoolBuilder: OK 2026-06-10 07:45:43.168518 | controller | - NodePoolLauncher: OK 2026-06-10 07:45:43.168525 | controller | - Zookeeper: OK 2026-06-10 07:45:43.168532 | controller | - Zuul: Waiting ... 2026-06-10 07:45:43.168539 | controller | - |_ Config Secrets: OK 2026-06-10 07:45:43.168546 | controller | - |_ Merger: OK 2026-06-10 07:45:43.168553 | controller | - |_ Scheduler: OK 2026-06-10 07:45:43.168559 | controller | - |_ Web: Waiting ... 2026-06-10 07:45:43.168564 | controller | 2026-06-10 07:45:43.168575 | controller | 2026-06-10T07:45:43-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:45:48.177696 | controller | 2026-06-10T07:45:48-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:45:48.179985 | controller | 2026-06-10T07:45:48-04:00 INFO Nothing to clean up. 2026-06-10 07:45:48.795786 | controller | 2026-06-10T07:45:48-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:45:48.795869 | controller | 2026-06-10T07:45:48-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:45:48.818024 | controller | 2026-06-10T07:45:48-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:45:48.818105 | controller | 2026-06-10T07:45:48-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:45:49.189993 | controller | 2026-06-10T07:45:49-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:45:49.602493 | controller | 2026-06-10T07:45:49-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:45:49.602560 | controller | 2026-06-10T07:45:49-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:45:50.605716 | controller | 2026-06-10T07:45:50-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:45:50.605796 | controller | 2026-06-10T07:45:50-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:45:50.782697 | controller | 2026-06-10T07:45:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:45:50.782770 | controller | 2026-06-10T07:45:50-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:45:51.593458 | controller | 2026-06-10T07:45:51-04:00 INFO 2026-06-10 07:45:51.593568 | controller | - Config: Waiting ... 2026-06-10 07:45:51.593578 | controller | - Gateway: OK 2026-06-10 07:45:51.593586 | controller | - GitServer: OK 2026-06-10 07:45:51.593593 | controller | - HoundSearch: OK 2026-06-10 07:45:51.593598 | controller | - LogJuicer: OK 2026-06-10 07:45:51.593603 | controller | - Logserver: OK 2026-06-10 07:45:51.593608 | controller | - MariaDB: OK 2026-06-10 07:45:51.593613 | controller | - NodePoolBuilder: OK 2026-06-10 07:45:51.593618 | controller | - NodePoolLauncher: OK 2026-06-10 07:45:51.593623 | controller | - Zookeeper: OK 2026-06-10 07:45:51.593628 | controller | - Zuul: Waiting ... 2026-06-10 07:45:51.593633 | controller | - |_ Config Secrets: OK 2026-06-10 07:45:51.593638 | controller | - |_ Merger: OK 2026-06-10 07:45:51.593642 | controller | - |_ Scheduler: OK 2026-06-10 07:45:51.593647 | controller | - |_ Web: Waiting ... 2026-06-10 07:45:51.593653 | controller | 2026-06-10 07:45:51.593666 | controller | 2026-06-10T07:45:51-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:45:56.601747 | controller | 2026-06-10T07:45:56-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:45:56.603775 | controller | 2026-06-10T07:45:56-04:00 INFO Nothing to clean up. 2026-06-10 07:45:57.218779 | controller | 2026-06-10T07:45:57-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:45:57.218855 | controller | 2026-06-10T07:45:57-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:45:57.240019 | controller | 2026-06-10T07:45:57-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:45:57.240089 | controller | 2026-06-10T07:45:57-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:45:57.620313 | controller | 2026-06-10T07:45:57-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:45:58.022320 | controller | 2026-06-10T07:45:58-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:45:58.022360 | controller | 2026-06-10T07:45:58-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:45:59.033720 | controller | 2026-06-10T07:45:59-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:45:59.033821 | controller | 2026-06-10T07:45:59-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:45:59.205226 | controller | 2026-06-10T07:45:59-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:45:59.205317 | controller | 2026-06-10T07:45:59-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:46:00.415834 | controller | 2026-06-10T07:46:00-04:00 INFO 2026-06-10 07:46:00.415882 | controller | - Config: OK 2026-06-10 07:46:00.415889 | controller | - Gateway: OK 2026-06-10 07:46:00.415895 | controller | - GitServer: OK 2026-06-10 07:46:00.415900 | controller | - HoundSearch: OK 2026-06-10 07:46:00.415905 | controller | - LogJuicer: OK 2026-06-10 07:46:00.415911 | controller | - Logserver: OK 2026-06-10 07:46:00.415916 | controller | - MariaDB: OK 2026-06-10 07:46:00.415921 | controller | - NodePoolBuilder: OK 2026-06-10 07:46:00.415926 | controller | - NodePoolLauncher: OK 2026-06-10 07:46:00.415931 | controller | - Zookeeper: OK 2026-06-10 07:46:00.415936 | controller | - Zuul: OK 2026-06-10 07:46:00.415941 | controller | 2026-06-10 07:46:00.415947 | controller | 2026-06-10T07:46:00-04:00 INFO Updating controller configmap ... 2026-06-10 07:46:00.802184 | controller | 2026-06-10T07:46:00-04:00 INFO Standalone reconcile done. 2026-06-10 11:46:01.915377 | controller | ok: Runtime: 0:00:30.090446 2026-06-10 11:46:01.921287 | 2026-06-10 11:46:01.921399 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:46:05.737867 | controller | ok: Runtime: 0:00:00.005190 2026-06-10 11:46:05.751081 | 2026-06-10 11:46:05.751192 | TASK [health-check/test-dry-run-lifecycle : Ensure zuul-executor StatefulSet is not present] 2026-06-10 07:46:08.052911 | controller | Error from server (NotFound): statefulsets.apps "zuul-executor" not found 2026-06-10 11:46:08.285791 | controller | ok: Runtime: 0:00:00.099637 2026-06-10 11:46:08.295233 | 2026-06-10 11:46:08.295402 | TASK [Run update detection test on zuul-web only] 2026-06-10 11:46:08.331538 | controller | ok 2026-06-10 11:46:08.373212 | 2026-06-10 11:46:08.373455 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-executor] 2026-06-10 11:46:08.902972 | controller | skipping: Conditional result was False 2026-06-10 11:46:08.916215 | 2026-06-10 11:46:08.916460 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-web] 2026-06-10 11:46:09.465067 | controller | ok: Runtime: 0:00:00.116880 2026-06-10 11:46:09.475340 | 2026-06-10 11:46:09.475525 | TASK [test-dry-run-update-detection : Patch zuul-executor statefulset template to trigger reconciliation] 2026-06-10 11:46:10.094069 | controller | skipping: Conditional result was False 2026-06-10 11:46:10.107449 | 2026-06-10 11:46:10.107756 | TASK [test-dry-run-update-detection : Patch zuul-web deployment template to trigger reconciliation] 2026-06-10 07:46:10.524373 | controller | deployment.apps/zuul-web patched 2026-06-10 11:46:10.656866 | controller | ok: Runtime: 0:00:00.099642 2026-06-10 11:46:10.661685 | 2026-06-10 11:46:10.661751 | TASK [Run deploy with --dry-run to check for update detection] 2026-06-10 11:46:10.694149 | controller | ok 2026-06-10 11:46:10.711717 | 2026-06-10 11:46:10.711788 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:46:10.988305 | controller | ok 2026-06-10 11:46:10.994904 | 2026-06-10 11:46:10.994986 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:46:11.276440 | controller | changed 2026-06-10 11:46:11.287394 | 2026-06-10 11:46:11.287605 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:46:11.956271 | controller | 2026-06-10T07:46:11-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:46:11.956372 | controller | 2026-06-10 07:46:11.970565 | controller | 2026-06-10T07:46:11-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:46:11.970631 | controller | 2026-06-10 07:46:11.984664 | controller | 2026-06-10T07:46:11-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-10 07:46:11.986748 | controller | 2026-06-10T07:46:11-04:00 INFO Nothing to clean up. 2026-06-10 07:46:12.598498 | controller | 2026-06-10T07:46:12-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:46:12.746920 | controller | 2026-06-10T07:46:12-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:46:13.014880 | controller | 2026-06-10T07:46:13-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:46:14.785063 | controller | 2026-06-10T07:46:14-04:00 INFO [Dry Run] Statefulset "zuul-web" configuration changed, applying... 2026-06-10 07:46:14.785126 | controller | 2026-06-10T07:46:14-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 07:46:14.785141 | controller | 2026-06-10T07:46:14-04:00 INFO Updating object, name: zuul-web (server dry run) 2026-06-10 07:46:15.594561 | controller | 2026-06-10T07:46:15-04:00 INFO 2026-06-10 07:46:15.594640 | controller | - Config: Waiting ... 2026-06-10 07:46:15.594654 | controller | - Gateway: OK 2026-06-10 07:46:15.594663 | controller | - GitServer: OK 2026-06-10 07:46:15.594672 | controller | - HoundSearch: OK 2026-06-10 07:46:15.594680 | controller | - LogJuicer: OK 2026-06-10 07:46:15.594688 | controller | - Logserver: OK 2026-06-10 07:46:15.594707 | controller | - MariaDB: OK 2026-06-10 07:46:15.594716 | controller | - NodePoolBuilder: OK 2026-06-10 07:46:15.594723 | controller | - NodePoolLauncher: OK 2026-06-10 07:46:15.594731 | controller | - Zookeeper: OK 2026-06-10 07:46:15.594739 | controller | - Zuul: Waiting ... 2026-06-10 07:46:15.594747 | controller | - |_ Config Secrets: OK 2026-06-10 07:46:15.594755 | controller | - |_ Merger: OK 2026-06-10 07:46:15.594763 | controller | - |_ Scheduler: OK 2026-06-10 07:46:15.594772 | controller | - |_ Web: Waiting ... 2026-06-10 07:46:15.594780 | controller | 2026-06-10 07:46:15.594788 | controller | 2026-06-10T07:46:15-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 11:46:15.839918 | controller | ok: Runtime: 0:00:04.006782 2026-06-10 11:46:15.845704 | 2026-06-10 11:46:15.845773 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:46:16.921381 | controller | ok: Runtime: 0:00:00.006071 2026-06-10 11:46:16.938287 | 2026-06-10 11:46:16.938444 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-executor] 2026-06-10 11:46:17.005245 | controller | skipping: Conditional result was False 2026-06-10 11:46:17.016862 | 2026-06-10 11:46:17.016993 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-web] 2026-06-10 07:46:18.295732 | controller | 2026-06-10T07:46:14-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 11:46:19.021231 | controller | ok: Runtime: 0:00:00.008410 2026-06-10 11:46:19.047382 | 2026-06-10 11:46:19.047565 | TASK [test-dry-run-update-detection : Restore original serial for zuul-executor] 2026-06-10 11:46:19.113088 | controller | skipping: Conditional result was False 2026-06-10 11:46:19.132788 | 2026-06-10 11:46:19.133016 | TASK [test-dry-run-update-detection : Restore original serial for zuul-web] 2026-06-10 07:46:20.520264 | controller | deployment.apps/zuul-web patched 2026-06-10 11:46:21.533158 | controller | ok: Runtime: 0:00:00.083203 2026-06-10 11:46:21.544531 | 2026-06-10 11:46:21.544626 | TASK [health-check/test-dry-run-lifecycle : Generate CR with zuul-executor enabled] 2026-06-10 11:46:28.833800 | controller | changed 2026-06-10 11:46:28.844066 | 2026-06-10 11:46:28.844204 | TASK [Run deploy with --dry-run to test from-scratch creation] 2026-06-10 11:46:28.884588 | controller | ok 2026-06-10 11:46:28.925754 | 2026-06-10 11:46:28.925874 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:46:31.919970 | controller | ok 2026-06-10 11:46:31.924997 | 2026-06-10 11:46:31.925066 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:46:32.150677 | controller | ok 2026-06-10 11:46:32.161911 | 2026-06-10 11:46:32.162043 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:46:32.807587 | controller | 2026-06-10T07:46:32-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:46:32.807647 | controller | 2026-06-10 07:46:32.816635 | controller | 2026-06-10T07:46:32-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:46:32.816658 | controller | 2026-06-10 07:46:32.830625 | controller | 2026-06-10T07:46:32-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-10 07:46:32.832109 | controller | 2026-06-10T07:46:32-04:00 INFO Nothing to clean up. 2026-06-10 07:46:33.442739 | controller | 2026-06-10T07:46:33-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:46:33.559359 | controller | 2026-06-10T07:46:33-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:46:33.851938 | controller | 2026-06-10T07:46:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:46:35.632258 | controller | 2026-06-10T07:46:35-04:00 INFO Creating object, name: zuul-executor (server dry run) 2026-06-10 07:46:37.236244 | controller | 2026-06-10T07:46:37-04:00 INFO 2026-06-10 07:46:37.236337 | controller | - Config: Waiting ... 2026-06-10 07:46:37.236349 | controller | - Gateway: OK 2026-06-10 07:46:37.236359 | controller | - GitServer: OK 2026-06-10 07:46:37.236368 | controller | - HoundSearch: OK 2026-06-10 07:46:37.236376 | controller | - LogJuicer: OK 2026-06-10 07:46:37.236383 | controller | - Logserver: OK 2026-06-10 07:46:37.236391 | controller | - MariaDB: OK 2026-06-10 07:46:37.236399 | controller | - NodePoolBuilder: OK 2026-06-10 07:46:37.236407 | controller | - NodePoolLauncher: OK 2026-06-10 07:46:37.236415 | controller | - Zookeeper: OK 2026-06-10 07:46:37.236423 | controller | - Zuul: Waiting ... 2026-06-10 07:46:37.236430 | controller | - |_ Config Secrets: OK 2026-06-10 07:46:37.236438 | controller | - |_ Executor: Waiting ... 2026-06-10 07:46:37.236446 | controller | - |_ Merger: OK 2026-06-10 07:46:37.236466 | controller | - |_ Scheduler: OK 2026-06-10 07:46:37.236474 | controller | - |_ Web: OK 2026-06-10 07:46:37.236483 | controller | 2026-06-10 07:46:37.236491 | controller | 2026-06-10T07:46:37-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 11:46:37.714558 | controller | ok: Runtime: 0:00:04.752787 2026-06-10 11:46:37.725292 | 2026-06-10 11:46:37.725467 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:46:38.275422 | controller | ok: Runtime: 0:00:00.009654 2026-06-10 11:46:38.293808 | 2026-06-10 11:46:38.294000 | TASK [health-check/test-dry-run-lifecycle : Check for 'create' dry-run messages for zuul-executor resources] 2026-06-10 07:46:38.569258 | controller | 2026-06-10T07:46:35-04:00 INFO Creating object, name: zuul-executor (server dry run) 2026-06-10 11:46:38.839931 | controller | ok: Runtime: 0:00:00.004883 2026-06-10 11:46:38.849909 | 2026-06-10 11:46:38.850037 | TASK [Restore original state by deploying with zuul-executor enabled] 2026-06-10 11:46:38.907254 | controller | ok 2026-06-10 11:46:38.961778 | 2026-06-10 11:46:38.961888 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 11:46:39.272517 | controller | ok 2026-06-10 11:46:39.282062 | 2026-06-10 11:46:39.282193 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 11:46:39.561939 | controller | changed 2026-06-10 11:46:39.571947 | 2026-06-10 11:46:39.572075 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 07:46:40.312895 | controller | 2026-06-10T07:46:40-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:46:40.312961 | controller | 2026-06-10 07:46:40.322827 | controller | 2026-06-10T07:46:40-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:46:40.322855 | controller | 2026-06-10 07:46:40.336978 | controller | 2026-06-10T07:46:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:46:40.338130 | controller | 2026-06-10T07:46:40-04:00 INFO Nothing to clean up. 2026-06-10 07:46:40.951452 | controller | 2026-06-10T07:46:40-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:46:41.069431 | controller | 2026-06-10T07:46:41-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:46:41.078719 | controller | 2026-06-10T07:46:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:46:41.078765 | controller | 2026-06-10T07:46:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:46:41.101159 | controller | 2026-06-10T07:46:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:46:41.101234 | controller | 2026-06-10T07:46:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:46:41.362186 | controller | 2026-06-10T07:46:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:46:41.749237 | controller | 2026-06-10T07:46:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:46:41.749295 | controller | 2026-06-10T07:46:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:46:42.764231 | controller | 2026-06-10T07:46:42-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:46:42.764287 | controller | 2026-06-10T07:46:42-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:46:43.136377 | controller | 2026-06-10T07:46:43-04:00 INFO Creating object, name: zuul-executor 2026-06-10 07:46:43.939271 | controller | 2026-06-10T07:46:43-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:46:43.939328 | controller | 2026-06-10T07:46:43-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:46:44.745012 | controller | 2026-06-10T07:46:44-04:00 INFO 2026-06-10 07:46:44.745054 | controller | - Config: Waiting ... 2026-06-10 07:46:44.745060 | controller | - Gateway: OK 2026-06-10 07:46:44.745065 | controller | - GitServer: OK 2026-06-10 07:46:44.745069 | controller | - HoundSearch: OK 2026-06-10 07:46:44.745074 | controller | - LogJuicer: OK 2026-06-10 07:46:44.745078 | controller | - Logserver: OK 2026-06-10 07:46:44.745082 | controller | - MariaDB: OK 2026-06-10 07:46:44.745087 | controller | - NodePoolBuilder: OK 2026-06-10 07:46:44.745092 | controller | - NodePoolLauncher: OK 2026-06-10 07:46:44.745096 | controller | - Zookeeper: OK 2026-06-10 07:46:44.745100 | controller | - Zuul: Waiting ... 2026-06-10 07:46:44.745105 | controller | - |_ Config Secrets: OK 2026-06-10 07:46:44.745109 | controller | - |_ Executor: Waiting ... 2026-06-10 07:46:44.745114 | controller | - |_ Merger: OK 2026-06-10 07:46:44.745118 | controller | - |_ Scheduler: OK 2026-06-10 07:46:44.745122 | controller | - |_ Web: OK 2026-06-10 07:46:44.745127 | controller | 2026-06-10 07:46:44.745178 | controller | 2026-06-10T07:46:44-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:46:49.755745 | controller | 2026-06-10T07:46:49-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:46:49.758107 | controller | 2026-06-10T07:46:49-04:00 INFO Nothing to clean up. 2026-06-10 07:46:50.368600 | controller | 2026-06-10T07:46:50-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:46:50.368660 | controller | 2026-06-10T07:46:50-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:46:50.385572 | controller | 2026-06-10T07:46:50-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:46:50.385615 | controller | 2026-06-10T07:46:50-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:46:50.781371 | controller | 2026-06-10T07:46:50-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:46:51.173514 | controller | 2026-06-10T07:46:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:46:51.173567 | controller | 2026-06-10T07:46:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:46:52.188482 | controller | 2026-06-10T07:46:52-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:46:52.188544 | controller | 2026-06-10T07:46:52-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:46:52.368309 | controller | 2026-06-10T07:46:52-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-10 07:46:56.400610 | controller | 2026-06-10T07:46:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:46:56.400657 | controller | 2026-06-10T07:46:56-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:46:56.449038 | controller | 2026-06-10T07:46:56-04:00 INFO 2026-06-10 07:46:56.449084 | controller | - Config: OK 2026-06-10 07:46:56.449091 | controller | - Gateway: OK 2026-06-10 07:46:56.449097 | controller | - GitServer: OK 2026-06-10 07:46:56.449102 | controller | - HoundSearch: OK 2026-06-10 07:46:56.449108 | controller | - LogJuicer: OK 2026-06-10 07:46:56.449113 | controller | - Logserver: OK 2026-06-10 07:46:56.449118 | controller | - MariaDB: OK 2026-06-10 07:46:56.449124 | controller | - NodePoolBuilder: OK 2026-06-10 07:46:56.449130 | controller | - NodePoolLauncher: OK 2026-06-10 07:46:56.449136 | controller | - Zookeeper: OK 2026-06-10 07:46:56.449141 | controller | - Zuul: OK 2026-06-10 07:46:56.449147 | controller | 2026-06-10 07:46:56.449153 | controller | 2026-06-10T07:46:56-04:00 INFO Updating controller configmap ... 2026-06-10 07:46:56.788411 | controller | 2026-06-10T07:46:56-04:00 INFO Standalone reconcile done. 2026-06-10 11:46:57.255512 | controller | ok: Runtime: 0:00:16.771248 2026-06-10 11:46:57.267048 | 2026-06-10 11:46:57.267186 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 11:46:57.816461 | controller | ok: Runtime: 0:00:00.004795 2026-06-10 11:46:57.833655 | 2026-06-10 11:46:57.833819 | TASK [health-check/test-dry-run-lifecycle : Remove temporary directory] 2026-06-10 11:46:58.134781 | controller | changed 2026-06-10 11:46:58.153311 | 2026-06-10 11:46:58.153484 | TASK [health-check/test-backup-restore : Backup the Software Factory deployment] 2026-06-10 07:46:58.733405 | controller | 2026-06-10T07:46:58-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:46:58.733517 | controller | 2026-06-10 07:46:58.745156 | controller | 2026-06-10T07:46:58-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:46:58.745244 | controller | 2026-06-10 07:46:58.750413 | controller | 2026-06-10T07:46:58-04:00 INFO Starting backup process for services in namespace: sf 2026-06-10 07:46:58.750486 | controller | 2026-06-10T07:46:58-04:00 INFO Creating secrets backup... 2026-06-10 07:46:58.781562 | controller | 2026-06-10T07:46:58-04:00 INFO Finished doing secret backup! 2026-06-10 07:46:58.781620 | controller | 2026-06-10T07:46:58-04:00 INFO Doing Zuul keys backup... 2026-06-10 07:46:58.783897 | controller | 2026-06-10T07:46:58-04:00 INFO Creating object, name: zuul-kazoo 2026-06-10 07:46:59.791298 | controller | 2026-06-10T07:46:59-04:00 INFO Running pod execution pod: zuul-kazoo, command: [zuul-admin export-keys /tmp/zuul-backup] 2026-06-10 07:47:01.448564 | controller | INFO:kazoo.client:Connecting to zookeeper-1.zookeeper-headless.sf.svc.cluster.local(10.244.0.105):2281, use_ssl: True 2026-06-10 07:47:01.508413 | controller | INFO:kazoo.client:Zookeeper connection established, state: CONNECTED 2026-06-10 07:47:01.586790 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh 2026-06-10 07:47:01.586892 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 2026-06-10 07:47:01.586984 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/git-server/system-config/system-config/ssh 2026-06-10 07:47:01.587482 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/git-server/system-config/system-config/secrets 2026-06-10 07:47:01.587505 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh 2026-06-10 07:47:01.587925 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 2026-06-10 07:47:01.588053 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-project/demo-project/ssh 2026-06-10 07:47:01.588316 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-10 07:47:01.588682 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/config/config/ssh 2026-06-10 07:47:01.588701 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/config/config/secrets 2026-06-10 07:47:01.589114 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/ichiban-config-3qoPMddz/ichiban-config-3qoPMddz/ssh 2026-06-10 07:47:01.589137 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/ichiban-config-3qoPMddz/ichiban-config-3qoPMddz/secrets 2026-06-10 07:47:01.589148 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/internal/ssh 2026-06-10 07:47:01.589771 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/ichiban-tenant-3qoPMddz/ssh 2026-06-10 07:47:01.589800 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/demo-tenant/ssh 2026-06-10 07:47:02.190524 | controller | 2026-06-10T07:47:02-04:00 INFO Running pod execution pod: zuul-kazoo, command: [cat /tmp/zuul-backup] 2026-06-10 07:47:02.315074 | controller | 2026-06-10T07:47:02-04:00 INFO Running pod execution pod: zuul-kazoo, command: [rm /tmp/zuul-backup] 2026-06-10 07:47:02.414546 | controller | 2026-06-10T07:47:02-04:00 INFO Finished doing Zuul private keys backup! 2026-06-10 07:47:02.418915 | controller | 2026-06-10T07:47:02-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 07:47:33.546558 | controller | 2026-06-10T07:47:33-04:00 INFO Doing DB backup... 2026-06-10 07:47:33.550112 | controller | 2026-06-10T07:47:33-04:00 INFO Running pod execution pod: mariadb-0, command: [mysqldump --databases zuul --single-transaction] 2026-06-10 07:47:33.686507 | controller | mysqldump: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb-dump' instead 2026-06-10 07:47:33.686605 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-10 07:47:33.885467 | controller | 2026-06-10T07:47:33-04:00 INFO Finished doing DBs backup! 2026-06-10 11:47:34.242947 | controller | ok: Runtime: 0:00:35.448739 2026-06-10 11:47:34.247465 | 2026-06-10 11:47:34.247533 | TASK [health-check/test-backup-restore : Fetch the last build ID reported by Zuul web] 2026-06-10 11:47:34.809676 | controller | ok: OK (698 bytes) 2026-06-10 11:47:34.822286 | 2026-06-10 11:47:34.822451 | TASK [health-check/test-backup-restore : Wipe Software Factory deployment] 2026-06-10 07:47:35.456140 | controller | 2026-06-10T07:47:35-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:47:35.456236 | controller | 2026-06-10 07:47:35.466891 | controller | 2026-06-10T07:47:35-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:47:35.466936 | controller | 2026-06-10 07:47:35.490320 | controller | 2026-06-10T07:47:35-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-10 07:47:35.490364 | controller | 2026-06-10T07:47:35-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-10 07:47:35.581630 | controller | 2026-06-10T07:47:35-04:00 INFO Deleting object, name: nodepool-builder 2026-06-10 07:47:35.586820 | controller | 2026-06-10T07:47:35-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-58bc794b9d-h6pmm launcher 2026-06-10 07:47:35.586862 | controller | 2026-06-10T07:47:35-04:00 INFO Running pod execution pod: nodepool-launcher-58bc794b9d-h6pmm, command: [kill -9 1] 2026-06-10 07:47:35.702495 | controller | 2026-06-10T07:47:35-04:00 INFO Deleting object, name: launcher 2026-06-10 07:47:35.705302 | controller | 2026-06-10T07:47:35-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-p79wx mariadb-client 2026-06-10 07:47:35.705321 | controller | 2026-06-10T07:47:35-04:00 INFO Running pod execution pod: zuul-db-provision-p79wx, command: [kill -9 1] 2026-06-10 07:47:35.721680 | controller | 2026-06-10T07:47:35-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 07:47:35.724089 | controller | 2026-06-10T07:47:35-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-zlqvm mariadb-client 2026-06-10 07:47:35.724109 | controller | 2026-06-10T07:47:35-04:00 INFO Running pod execution pod: zuul-db-provision-zlqvm, command: [kill -9 1] 2026-06-10 07:47:35.740719 | controller | 2026-06-10T07:47:35-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 07:47:35.743221 | controller | 2026-06-10T07:47:35-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-10 07:47:35.743275 | controller | 2026-06-10T07:47:35-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-10 07:47:35.799347 | controller | 2026-06-10T07:47:35-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 07:47:35.803136 | controller | 2026-06-10T07:47:35-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-10 07:47:35.803158 | controller | 2026-06-10T07:47:35-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-10 07:47:35.900530 | controller | 2026-06-10T07:47:35-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 07:47:35.903409 | controller | 2026-06-10T07:47:35-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-1 zuul-merger 2026-06-10 07:47:35.903427 | controller | 2026-06-10T07:47:35-04:00 INFO Running pod execution pod: zuul-merger-1, command: [kill -9 1] 2026-06-10 07:47:37.180149 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 07:47:37.184621 | controller | 2026-06-10T07:47:37-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-10 07:47:37.184645 | controller | 2026-06-10T07:47:37-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-10 07:47:37.305882 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-10 07:47:37.310525 | controller | 2026-06-10T07:47:37-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-5779988f86-mkb2t zuul-web 2026-06-10 07:47:37.310568 | controller | 2026-06-10T07:47:37-04:00 INFO Running pod execution pod: zuul-web-5779988f86-mkb2t, command: [kill -9 1] 2026-06-10 07:47:37.407947 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:47:37.412873 | controller | 2026-06-10T07:47:37-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-9qkgc zuul-weeder 2026-06-10 07:47:37.412917 | controller | 2026-06-10T07:47:37-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-9qkgc, command: [kill -9 1] 2026-06-10 07:47:37.503781 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 07:47:37.507125 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zookeeper 2026-06-10 07:47:37.513484 | controller | 2026-06-10T07:47:37-04:00 INFO Standalone mode detected. Deleting owner ConfigMap with foreground propagation. 2026-06-10 07:47:37.513582 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: sf-standalone-owner 2026-06-10 07:47:37.516351 | controller | 2026-06-10T07:47:37-04:00 INFO Cleaning resources... 2026-06-10 07:47:37.520026 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: gateway 2026-06-10 07:47:37.528565 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: hound-search 2026-06-10 07:47:37.535694 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: logjuicer 2026-06-10 07:47:37.544715 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: nodepool-launcher 2026-06-10 07:47:37.550874 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zookeeper 2026-06-10 07:47:37.557835 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zookeeper-headless 2026-06-10 07:47:37.562550 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-capacity 2026-06-10 07:47:37.571726 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-executor-headless 2026-06-10 07:47:37.576002 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:47:37.582288 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 07:47:37.592024 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: gateway 2026-06-10 07:47:37.594779 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: logjuicer 2026-06-10 07:47:37.597808 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: nodepool-launcher 2026-06-10 07:47:37.603601 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: git-server 2026-06-10 07:47:37.606578 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: hound-search 2026-06-10 07:47:37.610310 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: logserver 2026-06-10 07:47:37.614022 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: mariadb 2026-06-10 07:47:37.625929 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-b58rz", "status": "Running"} 2026-06-10 07:47:37.625989 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: gateway-6bbbd758d9-b58rz 2026-06-10 07:47:37.629050 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "git-server-0", "status": "Running"} 2026-06-10 07:47:37.629098 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: git-server-0 2026-06-10 07:47:37.633466 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "hound-search-0", "status": "Running"} 2026-06-10 07:47:37.633517 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: hound-search-0 2026-06-10 07:47:37.637231 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:37.637281 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:37.643064 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:37.643119 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:37.646995 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "mariadb-0", "status": "Running"} 2026-06-10 07:47:37.647059 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: mariadb-0 2026-06-10 07:47:37.653114 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:37.653207 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:37.661150 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "zookeeper-0", "status": "Running"} 2026-06-10 07:47:37.661260 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zookeeper-0 2026-06-10 07:47:37.664273 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "zookeeper-1", "status": "Running"} 2026-06-10 07:47:37.664332 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zookeeper-1 2026-06-10 07:47:37.669976 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "zookeeper-2", "status": "Running"} 2026-06-10 07:47:37.670024 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zookeeper-2 2026-06-10 07:47:37.680670 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "zuul-executor-0", "status": "Running"} 2026-06-10 07:47:37.680739 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-executor-0 2026-06-10 07:47:37.829580 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting pod {"name": "zuul-merger-0", "status": "Failed"} 2026-06-10 07:47:37.829822 | controller | 2026-06-10T07:47:37-04:00 INFO Deleting object, name: zuul-merger-0 2026-06-10 07:47:38.029847 | controller | 2026-06-10T07:47:38-04:00 INFO Deleting pod {"name": "zuul-merger-1", "status": "Failed"} 2026-06-10 07:47:38.029901 | controller | 2026-06-10T07:47:38-04:00 INFO Deleting object, name: zuul-merger-1 2026-06-10 07:47:38.230087 | controller | 2026-06-10T07:47:38-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:38.230161 | controller | 2026-06-10T07:47:38-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:38.429692 | controller | 2026-06-10T07:47:38-04:00 INFO Deleting pod {"name": "zuul-web-5779988f86-mkb2t", "status": "Running"} 2026-06-10 07:47:38.429764 | controller | 2026-06-10T07:47:38-04:00 INFO Deleting object, name: zuul-web-5779988f86-mkb2t 2026-06-10 07:47:38.629607 | controller | 2026-06-10T07:47:38-04:00 INFO Deleting pod {"name": "zuul-weeder-57c6567f5f-9qkgc", "status": "Running"} 2026-06-10 07:47:38.629662 | controller | 2026-06-10T07:47:38-04:00 INFO Deleting object, name: zuul-weeder-57c6567f5f-9qkgc 2026-06-10 07:47:40.839157 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-mrs4f", "status": "Running"} 2026-06-10 07:47:40.839254 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting object, name: gateway-6bbbd758d9-mrs4f 2026-06-10 07:47:40.844364 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:40.844415 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:40.848330 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:40.848375 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:40.856912 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:40.856972 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:40.868747 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:40.868859 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:40.876686 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-htdsv", "status": "Pending"} 2026-06-10 07:47:40.876778 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-htdsv 2026-06-10 07:47:40.888398 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:40.888486 | controller | 2026-06-10T07:47:40-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:42.900378 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-xnn7t", "status": "Pending"} 2026-06-10 07:47:42.900445 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting object, name: gateway-6bbbd758d9-xnn7t 2026-06-10 07:47:42.903584 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:42.903626 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:42.906487 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:42.906520 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:42.909144 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:42.909207 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:42.911911 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-9hmpn", "status": "Pending"} 2026-06-10 07:47:42.911947 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-9hmpn 2026-06-10 07:47:42.914494 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:42.914532 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:42.916946 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-htdsv", "status": "Pending"} 2026-06-10 07:47:42.916982 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-htdsv 2026-06-10 07:47:42.919602 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:42.919640 | controller | 2026-06-10T07:47:42-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:44.930321 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:44.930446 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:44.937276 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:44.937377 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:44.939884 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:44.939920 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:44.942122 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:44.942160 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:44.944755 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-htdsv", "status": "Pending"} 2026-06-10 07:47:44.944796 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-htdsv 2026-06-10 07:47:44.947289 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:44.947328 | controller | 2026-06-10T07:47:44-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:46.960749 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:46.960826 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:46.963519 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:46.963588 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:46.966134 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:46.966159 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:46.968254 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:46.968277 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:46.969930 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-htdsv", "status": "Pending"} 2026-06-10 07:47:46.969956 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-htdsv 2026-06-10 07:47:46.971876 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:46.971900 | controller | 2026-06-10T07:47:46-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:48.979558 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:48.979630 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:48.982289 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:48.982340 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:48.984558 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:48.984594 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:48.987138 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:48.987195 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:48.989990 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-htdsv", "status": "Pending"} 2026-06-10 07:47:48.990039 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-htdsv 2026-06-10 07:47:48.992972 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:48.993015 | controller | 2026-06-10T07:47:48-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:51.001883 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:51.001981 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:51.005049 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:51.005107 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:51.007525 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:51.007571 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:51.010117 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:51.010207 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:51.013273 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-htdsv", "status": "Pending"} 2026-06-10 07:47:51.013319 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-htdsv 2026-06-10 07:47:51.016347 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:51.016392 | controller | 2026-06-10T07:47:51-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:53.029090 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:53.029195 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:53.032368 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:53.032399 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:53.035580 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:53.035612 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:53.038367 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:53.038395 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:53.040731 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-htdsv", "status": "Pending"} 2026-06-10 07:47:53.040761 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-htdsv 2026-06-10 07:47:53.042231 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:53.042258 | controller | 2026-06-10T07:47:53-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:55.050803 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:55.050865 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:55.054526 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:55.054575 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:55.056980 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:55.057025 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:55.059857 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:55.059886 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:55.062371 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-htdsv", "status": "Pending"} 2026-06-10 07:47:55.062433 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-htdsv 2026-06-10 07:47:55.064550 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:55.064595 | controller | 2026-06-10T07:47:55-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:57.074646 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:57.074740 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:57.077215 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:57.077262 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:57.079315 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:57.079396 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:57.082016 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:57.082065 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:57.084630 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:57.084712 | controller | 2026-06-10T07:47:57-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:47:59.093099 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:47:59.093234 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:47:59.095482 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:47:59.095509 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:47:59.097936 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:47:59.097973 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:47:59.099742 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:47:59.099777 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:47:59.102154 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:47:59.102211 | controller | 2026-06-10T07:47:59-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:48:01.113137 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:48:01.113235 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:48:01.116304 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:48:01.116354 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:48:01.118527 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:48:01.118564 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:48:01.121366 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:48:01.121403 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:48:01.124714 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:48:01.124754 | controller | 2026-06-10T07:48:01-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:48:03.136143 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:48:03.136235 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:48:03.138394 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:48:03.138470 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:48:03.142273 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:48:03.142310 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:48:03.144360 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:48:03.144385 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:48:03.147144 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:48:03.147205 | controller | 2026-06-10T07:48:03-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:48:05.155591 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:48:05.155689 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:48:05.158340 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:48:05.158399 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:48:05.161970 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:48:05.162012 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:48:05.165025 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:48:05.165074 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:48:05.168648 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:48:05.168713 | controller | 2026-06-10T07:48:05-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:48:07.179709 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-29479", "status": "Running"} 2026-06-10 07:48:07.179782 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-29479 2026-06-10 07:48:07.182793 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:48:07.182824 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:48:07.185312 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 07:48:07.185338 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting object, name: logserver-0 2026-06-10 07:48:07.188091 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting pod {"name": "nodepool-launcher-58bc794b9d-h6pmm", "status": "Running"} 2026-06-10 07:48:07.188115 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting object, name: nodepool-launcher-58bc794b9d-h6pmm 2026-06-10 07:48:07.190745 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 07:48:07.190790 | controller | 2026-06-10T07:48:07-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 07:48:09.198544 | controller | 2026-06-10T07:48:09-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:48:09.198632 | controller | 2026-06-10T07:48:09-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:48:11.207490 | controller | 2026-06-10T07:48:11-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-9jt4p", "status": "Running"} 2026-06-10 07:48:11.207553 | controller | 2026-06-10T07:48:11-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-9jt4p 2026-06-10 07:48:13.215135 | controller | 2026-06-10T07:48:13-04:00 INFO Removing dangling persistent volume claims if any... 2026-06-10 07:48:13.218432 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "git-server-git-server-0", "status": "Bound"} 2026-06-10 07:48:13.218454 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: git-server-git-server-0 2026-06-10 07:48:13.221866 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "hound-search-data-hound-search-0", "status": "Bound"} 2026-06-10 07:48:13.221890 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: hound-search-data-hound-search-0 2026-06-10 07:48:13.225962 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "logserver-logserver-0", "status": "Bound"} 2026-06-10 07:48:13.225986 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: logserver-logserver-0 2026-06-10 07:48:13.228670 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "mariadb-logs-mariadb-0", "status": "Bound"} 2026-06-10 07:48:13.228700 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: mariadb-logs-mariadb-0 2026-06-10 07:48:13.231552 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "mariadb-mariadb-0", "status": "Bound"} 2026-06-10 07:48:13.231574 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: mariadb-mariadb-0 2026-06-10 07:48:13.234355 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "nodepool-builder-nodepool-builder-0", "status": "Bound"} 2026-06-10 07:48:13.234451 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: nodepool-builder-nodepool-builder-0 2026-06-10 07:48:13.237616 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "zookeeper-data-zookeeper-0", "status": "Bound"} 2026-06-10 07:48:13.237657 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: zookeeper-data-zookeeper-0 2026-06-10 07:48:13.239994 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "zookeeper-data-zookeeper-1", "status": "Bound"} 2026-06-10 07:48:13.240037 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: zookeeper-data-zookeeper-1 2026-06-10 07:48:13.243007 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "zookeeper-data-zookeeper-2", "status": "Bound"} 2026-06-10 07:48:13.243030 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: zookeeper-data-zookeeper-2 2026-06-10 07:48:13.245210 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "zookeeper-logs-zookeeper-0", "status": "Bound"} 2026-06-10 07:48:13.245289 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: zookeeper-logs-zookeeper-0 2026-06-10 07:48:13.249833 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "zookeeper-logs-zookeeper-1", "status": "Bound"} 2026-06-10 07:48:13.249891 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: zookeeper-logs-zookeeper-1 2026-06-10 07:48:13.424088 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "zookeeper-logs-zookeeper-2", "status": "Bound"} 2026-06-10 07:48:13.424150 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: zookeeper-logs-zookeeper-2 2026-06-10 07:48:13.624094 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "zuul-executor-zuul-executor-0", "status": "Bound"} 2026-06-10 07:48:13.624212 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: zuul-executor-zuul-executor-0 2026-06-10 07:48:13.822188 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting pvc {"name": "zuul-executor-zuul-executor-1", "status": "Bound"} 2026-06-10 07:48:13.822253 | controller | 2026-06-10T07:48:13-04:00 INFO Deleting object, name: zuul-executor-zuul-executor-1 2026-06-10 07:48:14.022482 | controller | 2026-06-10T07:48:14-04:00 INFO Deleting pvc {"name": "zuul-merger-zuul-merger-0", "status": "Bound"} 2026-06-10 07:48:14.022558 | controller | 2026-06-10T07:48:14-04:00 INFO Deleting object, name: zuul-merger-zuul-merger-0 2026-06-10 07:48:14.223630 | controller | 2026-06-10T07:48:14-04:00 INFO Deleting pvc {"name": "zuul-merger-zuul-merger-1", "status": "Bound"} 2026-06-10 07:48:14.223719 | controller | 2026-06-10T07:48:14-04:00 INFO Deleting object, name: zuul-merger-zuul-merger-1 2026-06-10 07:48:14.424035 | controller | 2026-06-10T07:48:14-04:00 INFO Deleting pvc {"name": "zuul-scheduler-zuul-scheduler-0", "status": "Bound"} 2026-06-10 07:48:14.424148 | controller | 2026-06-10T07:48:14-04:00 INFO Deleting object, name: zuul-scheduler-zuul-scheduler-0 2026-06-10 11:48:16.913730 | controller | ok: Runtime: 0:00:41.530192 2026-06-10 11:48:16.929208 | 2026-06-10 11:48:16.929384 | TASK [health-check/test-backup-restore : Restore backup of the Software Factory previous deployment] 2026-06-10 07:48:17.545033 | controller | 2026-06-10T07:48:17-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 07:48:17.545107 | controller | 2026-06-10 07:48:17.555070 | controller | 2026-06-10T07:48:17-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 07:48:17.555120 | controller | 2026-06-10 07:48:17.564745 | controller | 2026-06-10T07:48:17-04:00 INFO Restoring secrets... 2026-06-10 07:48:17.565278 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-10 07:48:17.569684 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-10 07:48:17.573256 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: nodepool-builder-ssh-key 2026-06-10 07:48:17.575864 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: zuul-ssh-key 2026-06-10 07:48:17.579576 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: zuul-spare-ssh-key 2026-06-10 07:48:17.581771 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: zuul-keystore-password 2026-06-10 07:48:17.584247 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-10 07:48:17.586782 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: logserver-keys 2026-06-10 07:48:17.589458 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: logserver-uploader-keys 2026-06-10 07:48:17.591922 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: logserver-uploader-spare-keys 2026-06-10 07:48:17.595263 | controller | 2026-06-10T07:48:17-04:00 INFO Spawning backend services... 2026-06-10 07:48:17.768617 | controller | 2026-06-10T07:48:17-04:00 INFO Creating secret, name: mariadb-root-password 2026-06-10 07:48:17.768704 | controller | 2026-06-10T07:48:17-04:00 INFO Creating object, name: mariadb-root-password 2026-06-10 07:48:18.168832 | controller | 2026-06-10T07:48:18-04:00 INFO Creating secret, name: mariadb-config-secrets 2026-06-10 07:48:18.168906 | controller | 2026-06-10T07:48:18-04:00 INFO Creating object, name: mariadb-config-secrets 2026-06-10 07:48:18.568904 | controller | 2026-06-10T07:48:18-04:00 INFO Creating secret, name: mariadb-initdb-secrets 2026-06-10 07:48:18.569046 | controller | 2026-06-10T07:48:18-04:00 INFO Creating object, name: mariadb-initdb-secrets 2026-06-10 07:48:18.774734 | controller | 2026-06-10T07:48:18-04:00 INFO Creating object, name: mariadb 2026-06-10 07:48:19.172156 | controller | 2026-06-10T07:48:19-04:00 INFO Creating config map name: zookeeper-pi-config-map 2026-06-10 07:48:19.172261 | controller | 2026-06-10T07:48:19-04:00 INFO Creating object, name: zookeeper-pi-config-map 2026-06-10 07:48:19.178201 | controller | 2026-06-10T07:48:19-04:00 INFO Creating service, name: zookeeper-headless 2026-06-10 07:48:19.178234 | controller | 2026-06-10T07:48:19-04:00 INFO Creating object, name: zookeeper-headless 2026-06-10 07:48:19.184874 | controller | 2026-06-10T07:48:19-04:00 INFO Creating service, name: zookeeper 2026-06-10 07:48:19.184942 | controller | 2026-06-10T07:48:19-04:00 INFO Creating object, name: zookeeper 2026-06-10 07:48:19.198813 | controller | 2026-06-10T07:48:19-04:00 INFO Creating PodDisruptionBudget, name: zookeeper 2026-06-10 07:48:19.198909 | controller | 2026-06-10T07:48:19-04:00 INFO Creating object, name: zookeeper 2026-06-10 07:48:19.207338 | controller | 2026-06-10T07:48:19-04:00 INFO Creating object, name: zookeeper 2026-06-10 07:48:19.212149 | controller | 2026-06-10T07:48:19-04:00 INFO Waiting for backend services... 2026-06-10 07:48:19.774804 | controller | 2026-06-10T07:48:19-04:00 INFO Creating service, name: mariadb 2026-06-10 07:48:19.774886 | controller | 2026-06-10T07:48:19-04:00 INFO Creating object, name: mariadb 2026-06-10 07:48:19.794037 | controller | 2026-06-10T07:48:19-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-10 07:48:47.903511 | controller | 2026-06-10T07:48:47-04:00 INFO Creating object, name: zuul-db-connection 2026-06-10 07:48:47.908517 | controller | 2026-06-10T07:48:47-04:00 INFO Creating object, name: zuul-db-provision 2026-06-10 07:48:54.072288 | controller | 2026-06-10T07:48:54-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-10 07:49:08.140793 | controller | 2026-06-10T07:49:08-04:00 INFO Creating secret, name: zuul-config 2026-06-10 07:49:08.140877 | controller | 2026-06-10T07:49:08-04:00 INFO Creating object, name: zuul-config 2026-06-10 07:49:08.145148 | controller | 2026-06-10T07:49:08-04:00 INFO Creating config map name: zuul-scheduler-tooling-config-map 2026-06-10 07:49:08.145229 | controller | 2026-06-10T07:49:08-04:00 INFO Creating object, name: zuul-scheduler-tooling-config-map 2026-06-10 07:49:08.151411 | controller | 2026-06-10T07:49:08-04:00 INFO Creating object, name: zuul-kazoo 2026-06-10 07:49:10.162651 | controller | 2026-06-10T07:49:10-04:00 INFO Restoring Zuul... 2026-06-10 07:49:10.162754 | controller | 2026-06-10T07:49:10-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c rm -rf /tmp/zuul-import && mkdir -p /tmp/zuul-import] 2026-06-10 07:49:10.277621 | controller | 2026-06-10T07:49:10-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /tmp/zuul.keys && zuul-admin import-keys --force /tmp/zuul.keys] 2026-06-10 07:49:11.926776 | controller | INFO:kazoo.client:Connecting to zookeeper-1.zookeeper-headless.sf.svc.cluster.local(10.244.0.125):2281, use_ssl: True 2026-06-10 07:49:12.176531 | controller | INFO:kazoo.client:Zookeeper connection established, state: CONNECTED 2026-06-10 07:49:12.547363 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh 2026-06-10 07:49:12.576259 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 2026-06-10 07:49:12.663694 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/git-server/system-config/system-config/ssh 2026-06-10 07:49:12.673673 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/git-server/system-config/system-config/secrets 2026-06-10 07:49:12.769618 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh 2026-06-10 07:49:12.781258 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 2026-06-10 07:49:12.884120 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-project/demo-project/ssh 2026-06-10 07:49:12.897863 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-10 07:49:12.965121 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/config/config/ssh 2026-06-10 07:49:12.974845 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/config/config/secrets 2026-06-10 07:49:13.075720 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/ichiban-config-3qoPMddz/ichiban-config-3qoPMddz/ssh 2026-06-10 07:49:13.076890 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/ichiban-config-3qoPMddz/ichiban-config-3qoPMddz/secrets 2026-06-10 07:49:13.167259 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/internal/ssh 2026-06-10 07:49:13.203240 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/ichiban-tenant-3qoPMddz/ssh 2026-06-10 07:49:13.252774 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/demo-tenant/ssh 2026-06-10 07:49:13.861972 | controller | 2026-06-10T07:49:13-04:00 INFO Finished doing Zuul private keys restore! 2026-06-10 07:49:13.862055 | controller | 2026-06-10T07:49:13-04:00 INFO Restoring DB... 2026-06-10 07:49:13.865632 | controller | 2026-06-10T07:49:13-04:00 INFO Running pod execution pod: mariadb-0, command: [mysql -e DROP DATABASE zuul;] 2026-06-10 07:49:13.949839 | controller | mysql: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb' instead 2026-06-10 07:49:13.949903 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-10 07:49:14.006653 | controller | 2026-06-10T07:49:14-04:00 INFO Running pod execution pod: mariadb-0, command: [mysql -h0] 2026-06-10 07:49:14.084237 | controller | mysql: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb' instead 2026-06-10 07:49:14.084302 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-10 07:49:14.699682 | controller | 2026-06-10T07:49:14-04:00 INFO Finished restoring DB from backup! 2026-06-10 07:49:14.702072 | controller | 2026-06-10T07:49:14-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 07:49:45.836566 | controller | 2026-06-10T07:49:45-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:49:45.839135 | controller | 2026-06-10T07:49:45-04:00 INFO Nothing to clean up. 2026-06-10 07:49:45.853828 | controller | 2026-06-10T07:49:45-04:00 INFO Creating config map name: git-server-pi-config-map 2026-06-10 07:49:45.853902 | controller | 2026-06-10T07:49:45-04:00 INFO Creating object, name: git-server-pi-config-map 2026-06-10 07:49:45.859898 | controller | 2026-06-10T07:49:45-04:00 INFO Creating object, name: git-server 2026-06-10 07:49:46.235933 | controller | 2026-06-10T07:49:46-04:00 INFO Creating config map name: logserver-config-map 2026-06-10 07:49:46.235996 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: logserver-config-map 2026-06-10 07:49:46.240207 | controller | 2026-06-10T07:49:46-04:00 INFO Creating service, name: logserver 2026-06-10 07:49:46.240237 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: logserver 2026-06-10 07:49:46.438484 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: logserver 2026-06-10 07:49:46.447322 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: gateway 2026-06-10 07:49:46.454513 | controller | 2026-06-10T07:49:46-04:00 INFO Creating config map name: gateway-config-map 2026-06-10 07:49:46.454564 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: gateway-config-map 2026-06-10 07:49:46.458771 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: gateway 2026-06-10 07:49:46.470448 | controller | 2026-06-10T07:49:46-04:00 INFO Creating service, name: hound-search 2026-06-10 07:49:46.470510 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: hound-search 2026-06-10 07:49:46.482104 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: hound-search 2026-06-10 07:49:46.490909 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: logjuicer-pvc 2026-06-10 07:49:46.496716 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: logjuicer 2026-06-10 07:49:46.507512 | controller | 2026-06-10T07:49:46-04:00 INFO Creating object, name: logjuicer 2026-06-10 07:49:46.857379 | controller | 2026-06-10T07:49:46-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 07:49:46.857435 | controller | 2026-06-10T07:49:46-04:00 INFO 2026-06-10 07:49:46.857442 | controller | - Config: Waiting ... 2026-06-10 07:49:46.857447 | controller | - Gateway: Waiting ... 2026-06-10 07:49:46.857451 | controller | - GitServer: Waiting ... 2026-06-10 07:49:46.857455 | controller | - HoundSearch: Waiting ... 2026-06-10 07:49:46.857459 | controller | - LogJuicer: Waiting ... 2026-06-10 07:49:46.857463 | controller | - Logserver: Waiting ... 2026-06-10 07:49:46.857467 | controller | - MariaDB: OK 2026-06-10 07:49:46.857472 | controller | - Zookeeper: OK 2026-06-10 07:49:46.857476 | controller | 2026-06-10 07:49:46.857480 | controller | 2026-06-10T07:49:46-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 07:49:51.862078 | controller | 2026-06-10T07:49:51-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:49:51.863954 | controller | 2026-06-10T07:49:51-04:00 INFO Nothing to clean up. 2026-06-10 07:49:51.876862 | controller | 2026-06-10T07:49:51-04:00 INFO Creating service, name: git-server 2026-06-10 07:49:51.876931 | controller | 2026-06-10T07:49:51-04:00 INFO Creating object, name: git-server 2026-06-10 07:49:51.884195 | controller | 2026-06-10T07:49:51-04:00 INFO Creating service, name: git-server-rw 2026-06-10 07:49:51.884263 | controller | 2026-06-10T07:49:51-04:00 INFO Creating object, name: git-server-rw 2026-06-10 07:49:51.918191 | controller | 2026-06-10T07:49:51-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 07:50:10.052725 | controller | 2026-06-10T07:50:10-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 07:50:10.167430 | controller | 2026-06-10T07:50:10-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 07:50:10.177587 | controller | 2026-06-10T07:50:10-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:50:10.177633 | controller | 2026-06-10T07:50:10-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:50:10.199949 | controller | 2026-06-10T07:50:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:50:10.200092 | controller | 2026-06-10T07:50:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:50:10.238814 | controller | 2026-06-10T07:50:10-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:50:10.390080 | controller | 2026-06-10T07:50:10-04:00 INFO Creating config map name: np-statsd-config-map 2026-06-10 07:50:10.390152 | controller | 2026-06-10T07:50:10-04:00 INFO Creating object, name: np-statsd-config-map 2026-06-10 07:50:10.396801 | controller | 2026-06-10T07:50:10-04:00 INFO Creating config map name: nodepool-tooling-config-map 2026-06-10 07:50:10.396887 | controller | 2026-06-10T07:50:10-04:00 INFO Creating object, name: nodepool-tooling-config-map 2026-06-10 07:50:10.402935 | controller | 2026-06-10T07:50:10-04:00 INFO Creating config map name: nodepool-launcher-extra-config-config-map 2026-06-10 07:50:10.403044 | controller | 2026-06-10T07:50:10-04:00 INFO Creating object, name: nodepool-launcher-extra-config-config-map 2026-06-10 07:50:10.428064 | controller | 2026-06-10T07:50:10-04:00 INFO Creating object, name: zuul-capacity 2026-06-10 07:50:10.437403 | controller | 2026-06-10T07:50:10-04:00 INFO Creating object, name: nodepool-launcher 2026-06-10 07:50:10.824064 | controller | 2026-06-10T07:50:10-04:00 INFO Creating config map name: nodepool-builder-extra-config-config-map 2026-06-10 07:50:10.824153 | controller | 2026-06-10T07:50:10-04:00 INFO Creating object, name: nodepool-builder-extra-config-config-map 2026-06-10 07:50:11.229625 | controller | 2026-06-10T07:50:11-04:00 INFO Creating service, name: nodepool-builder 2026-06-10 07:50:11.229694 | controller | 2026-06-10T07:50:11-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 07:50:11.239878 | controller | 2026-06-10T07:50:11-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 07:50:11.389033 | controller | 2026-06-10T07:50:11-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:50:11.398247 | controller | 2026-06-10T07:50:11-04:00 INFO Creating object, name: zuul-executor-headless 2026-06-10 07:50:11.623124 | controller | 2026-06-10T07:50:11-04:00 INFO Creating config map name: zuul-extra-config-map 2026-06-10 07:50:11.623247 | controller | 2026-06-10T07:50:11-04:00 INFO Creating object, name: zuul-extra-config-map 2026-06-10 07:50:12.023542 | controller | 2026-06-10T07:50:12-04:00 INFO Creating config map name: zuul-statsd-config-map 2026-06-10 07:50:12.023606 | controller | 2026-06-10T07:50:12-04:00 INFO Creating object, name: zuul-statsd-config-map 2026-06-10 07:50:12.424227 | controller | 2026-06-10T07:50:12-04:00 INFO Creating config map name: zuul-logging-config-map 2026-06-10 07:50:12.424356 | controller | 2026-06-10T07:50:12-04:00 INFO Creating object, name: zuul-logging-config-map 2026-06-10 07:50:12.628438 | controller | 2026-06-10T07:50:12-04:00 INFO Creating object, name: zuul-scheduler 2026-06-10 07:50:13.426933 | controller | 2026-06-10T07:50:13-04:00 INFO Creating object, name: zuul-executor 2026-06-10 07:50:14.226811 | controller | 2026-06-10T07:50:14-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:50:15.028974 | controller | 2026-06-10T07:50:15-04:00 INFO Creating object, name: zuul-merger 2026-06-10 07:50:15.035782 | controller | 2026-06-10T07:50:15-04:00 INFO 2026-06-10 07:50:15.035821 | controller | - Config: Waiting ... 2026-06-10 07:50:15.035832 | controller | - Gateway: OK 2026-06-10 07:50:15.035840 | controller | - GitServer: OK 2026-06-10 07:50:15.035849 | controller | - HoundSearch: OK 2026-06-10 07:50:15.035857 | controller | - LogJuicer: OK 2026-06-10 07:50:15.035865 | controller | - Logserver: OK 2026-06-10 07:50:15.035872 | controller | - MariaDB: OK 2026-06-10 07:50:15.035881 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:50:15.035889 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:50:15.035897 | controller | - Zookeeper: OK 2026-06-10 07:50:15.035905 | controller | - Zuul: Waiting ... 2026-06-10 07:50:15.035913 | controller | - |_ Config Secrets: OK 2026-06-10 07:50:15.035921 | controller | - |_ Executor: Waiting ... 2026-06-10 07:50:15.035929 | controller | - |_ Merger: Waiting ... 2026-06-10 07:50:15.035937 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:50:15.035945 | controller | - |_ Web: Waiting ... 2026-06-10 07:50:15.035953 | controller | 2026-06-10 07:50:15.035961 | controller | 2026-06-10T07:50:15-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 07:50:20.051406 | controller | 2026-06-10T07:50:20-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:50:20.063507 | controller | 2026-06-10T07:50:20-04:00 INFO Nothing to clean up. 2026-06-10 07:50:20.679353 | controller | 2026-06-10T07:50:20-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:50:20.679413 | controller | 2026-06-10T07:50:20-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:50:20.701286 | controller | 2026-06-10T07:50:20-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:50:20.701347 | controller | 2026-06-10T07:50:20-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:50:21.087786 | controller | 2026-06-10T07:50:21-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:50:21.466206 | controller | 2026-06-10T07:50:21-04:00 INFO Creating object, name: nodepool-launcher 2026-06-10 07:50:21.472690 | controller | 2026-06-10T07:50:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:50:21.472830 | controller | 2026-06-10T07:50:21-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 07:50:22.473799 | controller | 2026-06-10T07:50:22-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:50:52.588297 | controller | 2026-06-10T07:50:52-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 07:50:52.624958 | controller | 2026-06-10T07:50:52-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:50:52.625024 | controller | 2026-06-10T07:50:52-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:50:53.002261 | controller | 2026-06-10T07:50:53-04:00 INFO 2026-06-10 07:50:53.002320 | controller | - Config: Waiting ... 2026-06-10 07:50:53.002326 | controller | - Gateway: OK 2026-06-10 07:50:53.002330 | controller | - GitServer: OK 2026-06-10 07:50:53.002334 | controller | - HoundSearch: OK 2026-06-10 07:50:53.002337 | controller | - LogJuicer: OK 2026-06-10 07:50:53.002341 | controller | - Logserver: OK 2026-06-10 07:50:53.002344 | controller | - MariaDB: OK 2026-06-10 07:50:53.002348 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 07:50:53.002352 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 07:50:53.002355 | controller | - Zookeeper: OK 2026-06-10 07:50:53.002359 | controller | - Zuul: Waiting ... 2026-06-10 07:50:53.002362 | controller | - |_ Config Secrets: OK 2026-06-10 07:50:53.002366 | controller | - |_ Executor: OK 2026-06-10 07:50:53.002370 | controller | - |_ Merger: OK 2026-06-10 07:50:53.002373 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:50:53.002377 | controller | - |_ Web: Waiting ... 2026-06-10 07:50:53.002381 | controller | 2026-06-10 07:50:53.002385 | controller | 2026-06-10T07:50:53-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 07:50:58.008909 | controller | 2026-06-10T07:50:58-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:50:58.012092 | controller | 2026-06-10T07:50:58-04:00 INFO Nothing to clean up. 2026-06-10 07:50:58.627711 | controller | 2026-06-10T07:50:58-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:50:58.627776 | controller | 2026-06-10T07:50:58-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:50:58.645420 | controller | 2026-06-10T07:50:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:50:58.645489 | controller | 2026-06-10T07:50:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:50:59.029782 | controller | 2026-06-10T07:50:59-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:50:59.423765 | controller | 2026-06-10T07:50:59-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:50:59.423835 | controller | 2026-06-10T07:50:59-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:51:00.431270 | controller | 2026-06-10T07:51:00-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 07:51:16.501799 | controller | 2026-06-10T07:51:16-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 07:51:16.518631 | controller | 2026-06-10T07:51:16-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 07:51:16.573823 | controller | 2026-06-10T07:51:16-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:51:16.573936 | controller | 2026-06-10T07:51:16-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:51:16.939571 | controller | 2026-06-10T07:51:16-04:00 INFO 2026-06-10 07:51:16.939636 | controller | - Config: Waiting ... 2026-06-10 07:51:16.939647 | controller | - Gateway: OK 2026-06-10 07:51:16.939657 | controller | - GitServer: OK 2026-06-10 07:51:16.939666 | controller | - HoundSearch: OK 2026-06-10 07:51:16.939675 | controller | - LogJuicer: OK 2026-06-10 07:51:16.939684 | controller | - Logserver: OK 2026-06-10 07:51:16.939693 | controller | - MariaDB: OK 2026-06-10 07:51:16.939701 | controller | - NodePoolBuilder: OK 2026-06-10 07:51:16.939710 | controller | - NodePoolLauncher: OK 2026-06-10 07:51:16.939718 | controller | - Zookeeper: OK 2026-06-10 07:51:16.939727 | controller | - Zuul: Waiting ... 2026-06-10 07:51:16.939736 | controller | - |_ Config Secrets: OK 2026-06-10 07:51:16.939744 | controller | - |_ Executor: OK 2026-06-10 07:51:16.939753 | controller | - |_ Merger: OK 2026-06-10 07:51:16.939762 | controller | - |_ Scheduler: Waiting ... 2026-06-10 07:51:16.939771 | controller | - |_ Web: OK 2026-06-10 07:51:16.939781 | controller | 2026-06-10 07:51:16.939791 | controller | 2026-06-10T07:51:16-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 07:51:21.950289 | controller | 2026-06-10T07:51:21-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:51:21.952408 | controller | 2026-06-10T07:51:21-04:00 INFO Nothing to clean up. 2026-06-10 07:51:22.565585 | controller | 2026-06-10T07:51:22-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:51:22.565662 | controller | 2026-06-10T07:51:22-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:51:22.585459 | controller | 2026-06-10T07:51:22-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:51:22.585519 | controller | 2026-06-10T07:51:22-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:51:22.962570 | controller | 2026-06-10T07:51:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:51:23.366357 | controller | 2026-06-10T07:51:23-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:51:23.366423 | controller | 2026-06-10T07:51:23-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:51:24.372824 | controller | 2026-06-10T07:51:24-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:51:24.372939 | controller | 2026-06-10T07:51:24-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:51:25.353101 | controller | 2026-06-10T07:51:25-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:51:25.353221 | controller | 2026-06-10T07:51:25-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:51:26.351451 | controller | 2026-06-10T07:51:26-04:00 INFO Creating config map name: pymod-sf-operator-config-map 2026-06-10 07:51:26.351542 | controller | 2026-06-10T07:51:26-04:00 INFO Creating object, name: pymod-sf-operator-config-map 2026-06-10 07:51:26.752432 | controller | 2026-06-10T07:51:26-04:00 INFO Creating object, name: config-updater 2026-06-10 07:51:26.759879 | controller | 2026-06-10T07:51:26-04:00 INFO Creating object, name: config-updater-role 2026-06-10 07:51:26.765559 | controller | 2026-06-10T07:51:26-04:00 INFO Creating object, name: config-updater 2026-06-10 07:51:26.770723 | controller | 2026-06-10T07:51:26-04:00 INFO Creating the config-update service account secret 2026-06-10 07:51:26.770761 | controller | 2026-06-10T07:51:26-04:00 INFO Creating object, name: config-update-secrets 2026-06-10 07:51:26.777125 | controller | 2026-06-10T07:51:26-04:00 INFO Creating base secret job 2026-06-10 07:51:26.777825 | controller | 2026-06-10T07:51:26-04:00 INFO Creating object, name: config-base-secret-1-9189-ns 2026-06-10 07:51:26.782413 | controller | 2026-06-10T07:51:26-04:00 INFO 2026-06-10 07:51:26.782470 | controller | - Config: Waiting ... 2026-06-10 07:51:26.782486 | controller | - Gateway: OK 2026-06-10 07:51:26.782499 | controller | - GitServer: OK 2026-06-10 07:51:26.782530 | controller | - HoundSearch: OK 2026-06-10 07:51:26.782543 | controller | - LogJuicer: OK 2026-06-10 07:51:26.782556 | controller | - Logserver: OK 2026-06-10 07:51:26.782568 | controller | - MariaDB: OK 2026-06-10 07:51:26.782580 | controller | - NodePoolBuilder: OK 2026-06-10 07:51:26.782592 | controller | - NodePoolLauncher: OK 2026-06-10 07:51:26.782604 | controller | - Zookeeper: OK 2026-06-10 07:51:26.782616 | controller | - Zuul: OK 2026-06-10 07:51:26.782629 | controller | 2026-06-10 07:51:26.782642 | controller | 2026-06-10T07:51:26-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 07:51:31.791010 | controller | 2026-06-10T07:51:31-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:51:31.793344 | controller | 2026-06-10T07:51:31-04:00 INFO Nothing to clean up. 2026-06-10 07:51:32.406069 | controller | 2026-06-10T07:51:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:51:32.406129 | controller | 2026-06-10T07:51:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:51:32.426244 | controller | 2026-06-10T07:51:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:51:32.426320 | controller | 2026-06-10T07:51:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:51:32.810673 | controller | 2026-06-10T07:51:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:51:33.206213 | controller | 2026-06-10T07:51:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:51:33.206281 | controller | 2026-06-10T07:51:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:51:34.221127 | controller | 2026-06-10T07:51:34-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:51:34.221239 | controller | 2026-06-10T07:51:34-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:51:35.194276 | controller | 2026-06-10T07:51:35-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:51:35.194335 | controller | 2026-06-10T07:51:35-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:51:36.401998 | controller | 2026-06-10T07:51:36-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 07:51:36.592502 | controller | 2026-06-10T07:51:36-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 07:51:41.971189 | controller | 2026-06-10T07:51:41-04:00 INFO Deleting object, name: zuul-web 2026-06-10 07:51:41.978052 | controller | 2026-06-10T07:51:41-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-10 07:51:41.982320 | controller | 2026-06-10T07:51:41-04:00 INFO 2026-06-10 07:51:41.982392 | controller | - Config: Waiting ... 2026-06-10 07:51:41.982401 | controller | - Gateway: OK 2026-06-10 07:51:41.982406 | controller | - GitServer: OK 2026-06-10 07:51:41.982411 | controller | - HoundSearch: OK 2026-06-10 07:51:41.982416 | controller | - LogJuicer: OK 2026-06-10 07:51:41.982421 | controller | - Logserver: OK 2026-06-10 07:51:41.982426 | controller | - MariaDB: OK 2026-06-10 07:51:41.982431 | controller | - NodePoolBuilder: OK 2026-06-10 07:51:41.982436 | controller | - NodePoolLauncher: OK 2026-06-10 07:51:41.982441 | controller | - Zookeeper: OK 2026-06-10 07:51:41.982447 | controller | - Zuul: OK 2026-06-10 07:51:41.982453 | controller | 2026-06-10 07:51:41.982489 | controller | 2026-06-10T07:51:41-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 07:51:46.990939 | controller | 2026-06-10T07:51:46-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:51:46.993869 | controller | 2026-06-10T07:51:46-04:00 INFO Nothing to clean up. 2026-06-10 07:51:47.606198 | controller | 2026-06-10T07:51:47-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:51:47.606271 | controller | 2026-06-10T07:51:47-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:51:47.623581 | controller | 2026-06-10T07:51:47-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:51:47.623643 | controller | 2026-06-10T07:51:47-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:51:48.005906 | controller | 2026-06-10T07:51:48-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:51:48.410237 | controller | 2026-06-10T07:51:48-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:51:48.410304 | controller | 2026-06-10T07:51:48-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:51:49.422000 | controller | 2026-06-10T07:51:49-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:51:49.422123 | controller | 2026-06-10T07:51:49-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:51:50.394877 | controller | 2026-06-10T07:51:50-04:00 INFO Creating object, name: zuul-web 2026-06-10 07:51:51.205465 | controller | 2026-06-10T07:51:51-04:00 INFO 2026-06-10 07:51:51.205546 | controller | - Config: Waiting ... 2026-06-10 07:51:51.205560 | controller | - Gateway: OK 2026-06-10 07:51:51.205570 | controller | - GitServer: OK 2026-06-10 07:51:51.205582 | controller | - HoundSearch: OK 2026-06-10 07:51:51.205594 | controller | - LogJuicer: OK 2026-06-10 07:51:51.205604 | controller | - Logserver: OK 2026-06-10 07:51:51.205614 | controller | - MariaDB: OK 2026-06-10 07:51:51.205624 | controller | - NodePoolBuilder: OK 2026-06-10 07:51:51.205635 | controller | - NodePoolLauncher: OK 2026-06-10 07:51:51.205645 | controller | - Zookeeper: OK 2026-06-10 07:51:51.205655 | controller | - Zuul: Waiting ... 2026-06-10 07:51:51.205665 | controller | - |_ Config Secrets: OK 2026-06-10 07:51:51.205677 | controller | - |_ Executor: OK 2026-06-10 07:51:51.205687 | controller | - |_ Merger: OK 2026-06-10 07:51:51.205698 | controller | - |_ Scheduler: OK 2026-06-10 07:51:51.205710 | controller | - |_ Web: Waiting ... 2026-06-10 07:51:51.205722 | controller | 2026-06-10 07:51:51.205734 | controller | 2026-06-10T07:51:51-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-10 07:51:56.215341 | controller | 2026-06-10T07:51:56-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:51:56.217687 | controller | 2026-06-10T07:51:56-04:00 INFO Nothing to clean up. 2026-06-10 07:51:56.832696 | controller | 2026-06-10T07:51:56-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:51:56.832769 | controller | 2026-06-10T07:51:56-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:51:56.853893 | controller | 2026-06-10T07:51:56-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:51:56.854008 | controller | 2026-06-10T07:51:56-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:51:57.234003 | controller | 2026-06-10T07:51:57-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:51:57.625373 | controller | 2026-06-10T07:51:57-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:51:57.625452 | controller | 2026-06-10T07:51:57-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:51:58.636521 | controller | 2026-06-10T07:51:58-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:51:58.636590 | controller | 2026-06-10T07:51:58-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:51:59.618503 | controller | 2026-06-10T07:51:59-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:51:59.618572 | controller | 2026-06-10T07:51:59-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:52:00.427329 | controller | 2026-06-10T07:52:00-04:00 INFO 2026-06-10 07:52:00.427409 | controller | - Config: Waiting ... 2026-06-10 07:52:00.427415 | controller | - Gateway: OK 2026-06-10 07:52:00.427420 | controller | - GitServer: OK 2026-06-10 07:52:00.427424 | controller | - HoundSearch: OK 2026-06-10 07:52:00.427428 | controller | - LogJuicer: OK 2026-06-10 07:52:00.427433 | controller | - Logserver: OK 2026-06-10 07:52:00.427437 | controller | - MariaDB: OK 2026-06-10 07:52:00.427441 | controller | - NodePoolBuilder: OK 2026-06-10 07:52:00.427445 | controller | - NodePoolLauncher: OK 2026-06-10 07:52:00.427496 | controller | - Zookeeper: OK 2026-06-10 07:52:00.427503 | controller | - Zuul: Waiting ... 2026-06-10 07:52:00.427508 | controller | - |_ Config Secrets: OK 2026-06-10 07:52:00.427512 | controller | - |_ Executor: OK 2026-06-10 07:52:00.427516 | controller | - |_ Merger: OK 2026-06-10 07:52:00.427520 | controller | - |_ Scheduler: OK 2026-06-10 07:52:00.427524 | controller | - |_ Web: Waiting ... 2026-06-10 07:52:00.427529 | controller | 2026-06-10 07:52:00.427540 | controller | 2026-06-10T07:52:00-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-10 07:52:05.431832 | controller | 2026-06-10T07:52:05-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:52:05.433734 | controller | 2026-06-10T07:52:05-04:00 INFO Nothing to clean up. 2026-06-10 07:52:06.048661 | controller | 2026-06-10T07:52:06-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:52:06.048766 | controller | 2026-06-10T07:52:06-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:52:06.064304 | controller | 2026-06-10T07:52:06-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:52:06.064374 | controller | 2026-06-10T07:52:06-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:52:06.448027 | controller | 2026-06-10T07:52:06-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:52:06.842113 | controller | 2026-06-10T07:52:06-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:52:06.842202 | controller | 2026-06-10T07:52:06-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:52:07.859224 | controller | 2026-06-10T07:52:07-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:52:07.859331 | controller | 2026-06-10T07:52:07-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:52:08.835613 | controller | 2026-06-10T07:52:08-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:52:08.835710 | controller | 2026-06-10T07:52:08-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:52:09.647599 | controller | 2026-06-10T07:52:09-04:00 INFO 2026-06-10 07:52:09.647664 | controller | - Config: Waiting ... 2026-06-10 07:52:09.647671 | controller | - Gateway: OK 2026-06-10 07:52:09.647678 | controller | - GitServer: OK 2026-06-10 07:52:09.647683 | controller | - HoundSearch: OK 2026-06-10 07:52:09.647688 | controller | - LogJuicer: OK 2026-06-10 07:52:09.647693 | controller | - Logserver: OK 2026-06-10 07:52:09.647699 | controller | - MariaDB: OK 2026-06-10 07:52:09.647704 | controller | - NodePoolBuilder: OK 2026-06-10 07:52:09.647709 | controller | - NodePoolLauncher: OK 2026-06-10 07:52:09.647714 | controller | - Zookeeper: OK 2026-06-10 07:52:09.647720 | controller | - Zuul: Waiting ... 2026-06-10 07:52:09.647725 | controller | - |_ Config Secrets: OK 2026-06-10 07:52:09.647731 | controller | - |_ Executor: OK 2026-06-10 07:52:09.647736 | controller | - |_ Merger: OK 2026-06-10 07:52:09.647741 | controller | - |_ Scheduler: OK 2026-06-10 07:52:09.647760 | controller | - |_ Web: Waiting ... 2026-06-10 07:52:09.647765 | controller | 2026-06-10 07:52:09.647771 | controller | 2026-06-10T07:52:09-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-10 07:52:14.653919 | controller | 2026-06-10T07:52:14-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:52:14.655437 | controller | 2026-06-10T07:52:14-04:00 INFO Nothing to clean up. 2026-06-10 07:52:15.275959 | controller | 2026-06-10T07:52:15-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:52:15.276033 | controller | 2026-06-10T07:52:15-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:52:15.303890 | controller | 2026-06-10T07:52:15-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:52:15.303959 | controller | 2026-06-10T07:52:15-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:52:15.678801 | controller | 2026-06-10T07:52:15-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:52:16.071254 | controller | 2026-06-10T07:52:16-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:52:16.071336 | controller | 2026-06-10T07:52:16-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:52:17.079499 | controller | 2026-06-10T07:52:17-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:52:17.079564 | controller | 2026-06-10T07:52:17-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:52:18.058257 | controller | 2026-06-10T07:52:18-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:52:18.058321 | controller | 2026-06-10T07:52:18-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:52:18.866088 | controller | 2026-06-10T07:52:18-04:00 INFO 2026-06-10 07:52:18.866203 | controller | - Config: Waiting ... 2026-06-10 07:52:18.866217 | controller | - Gateway: OK 2026-06-10 07:52:18.866226 | controller | - GitServer: OK 2026-06-10 07:52:18.866234 | controller | - HoundSearch: OK 2026-06-10 07:52:18.866242 | controller | - LogJuicer: OK 2026-06-10 07:52:18.866250 | controller | - Logserver: OK 2026-06-10 07:52:18.866258 | controller | - MariaDB: OK 2026-06-10 07:52:18.866266 | controller | - NodePoolBuilder: OK 2026-06-10 07:52:18.866274 | controller | - NodePoolLauncher: OK 2026-06-10 07:52:18.866282 | controller | - Zookeeper: OK 2026-06-10 07:52:18.866290 | controller | - Zuul: Waiting ... 2026-06-10 07:52:18.866298 | controller | - |_ Config Secrets: OK 2026-06-10 07:52:18.866306 | controller | - |_ Executor: OK 2026-06-10 07:52:18.866314 | controller | - |_ Merger: OK 2026-06-10 07:52:18.866321 | controller | - |_ Scheduler: OK 2026-06-10 07:52:18.866329 | controller | - |_ Web: Waiting ... 2026-06-10 07:52:18.866337 | controller | 2026-06-10 07:52:18.866346 | controller | 2026-06-10T07:52:18-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-10 07:52:23.872569 | controller | 2026-06-10T07:52:23-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:52:23.875769 | controller | 2026-06-10T07:52:23-04:00 INFO Nothing to clean up. 2026-06-10 07:52:24.487960 | controller | 2026-06-10T07:52:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:52:24.488022 | controller | 2026-06-10T07:52:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:52:24.503567 | controller | 2026-06-10T07:52:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:52:24.503637 | controller | 2026-06-10T07:52:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:52:24.892452 | controller | 2026-06-10T07:52:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:52:25.287618 | controller | 2026-06-10T07:52:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:52:25.287672 | controller | 2026-06-10T07:52:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:52:26.292549 | controller | 2026-06-10T07:52:26-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:52:26.292622 | controller | 2026-06-10T07:52:26-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:52:27.274465 | controller | 2026-06-10T07:52:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:52:27.274538 | controller | 2026-06-10T07:52:27-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:52:28.081090 | controller | 2026-06-10T07:52:28-04:00 INFO 2026-06-10 07:52:28.081220 | controller | - Config: Waiting ... 2026-06-10 07:52:28.081237 | controller | - Gateway: OK 2026-06-10 07:52:28.081245 | controller | - GitServer: OK 2026-06-10 07:52:28.081253 | controller | - HoundSearch: OK 2026-06-10 07:52:28.081275 | controller | - LogJuicer: OK 2026-06-10 07:52:28.081281 | controller | - Logserver: OK 2026-06-10 07:52:28.081286 | controller | - MariaDB: OK 2026-06-10 07:52:28.081291 | controller | - NodePoolBuilder: OK 2026-06-10 07:52:28.081296 | controller | - NodePoolLauncher: OK 2026-06-10 07:52:28.081302 | controller | - Zookeeper: OK 2026-06-10 07:52:28.081307 | controller | - Zuul: Waiting ... 2026-06-10 07:52:28.081312 | controller | - |_ Config Secrets: OK 2026-06-10 07:52:28.081317 | controller | - |_ Executor: OK 2026-06-10 07:52:28.081322 | controller | - |_ Merger: OK 2026-06-10 07:52:28.081328 | controller | - |_ Scheduler: OK 2026-06-10 07:52:28.081333 | controller | - |_ Web: Waiting ... 2026-06-10 07:52:28.081338 | controller | 2026-06-10 07:52:28.081354 | controller | 2026-06-10T07:52:28-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-10 07:52:33.088583 | controller | 2026-06-10T07:52:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:52:33.089750 | controller | 2026-06-10T07:52:33-04:00 INFO Nothing to clean up. 2026-06-10 07:52:33.713297 | controller | 2026-06-10T07:52:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:52:33.713375 | controller | 2026-06-10T07:52:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:52:33.731040 | controller | 2026-06-10T07:52:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:52:33.731113 | controller | 2026-06-10T07:52:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:52:34.111354 | controller | 2026-06-10T07:52:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:52:34.500140 | controller | 2026-06-10T07:52:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:52:34.500225 | controller | 2026-06-10T07:52:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:52:35.511915 | controller | 2026-06-10T07:52:35-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:52:35.511986 | controller | 2026-06-10T07:52:35-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:52:36.495902 | controller | 2026-06-10T07:52:36-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:52:36.495985 | controller | 2026-06-10T07:52:36-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:52:37.307560 | controller | 2026-06-10T07:52:37-04:00 INFO 2026-06-10 07:52:37.307636 | controller | - Config: Waiting ... 2026-06-10 07:52:37.307646 | controller | - Gateway: OK 2026-06-10 07:52:37.307654 | controller | - GitServer: OK 2026-06-10 07:52:37.307661 | controller | - HoundSearch: OK 2026-06-10 07:52:37.307668 | controller | - LogJuicer: OK 2026-06-10 07:52:37.307675 | controller | - Logserver: OK 2026-06-10 07:52:37.307682 | controller | - MariaDB: OK 2026-06-10 07:52:37.307689 | controller | - NodePoolBuilder: OK 2026-06-10 07:52:37.307859 | controller | - NodePoolLauncher: OK 2026-06-10 07:52:37.307866 | controller | - Zookeeper: OK 2026-06-10 07:52:37.307873 | controller | - Zuul: Waiting ... 2026-06-10 07:52:37.307880 | controller | - |_ Config Secrets: OK 2026-06-10 07:52:37.307887 | controller | - |_ Executor: OK 2026-06-10 07:52:37.307894 | controller | - |_ Merger: OK 2026-06-10 07:52:37.307901 | controller | - |_ Scheduler: OK 2026-06-10 07:52:37.307907 | controller | - |_ Web: Waiting ... 2026-06-10 07:52:37.307915 | controller | 2026-06-10 07:52:37.307922 | controller | 2026-06-10T07:52:37-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-10 07:52:42.317223 | controller | 2026-06-10T07:52:42-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:52:42.319416 | controller | 2026-06-10T07:52:42-04:00 INFO Nothing to clean up. 2026-06-10 07:52:42.936644 | controller | 2026-06-10T07:52:42-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:52:42.936765 | controller | 2026-06-10T07:52:42-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:52:42.952374 | controller | 2026-06-10T07:52:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:52:42.952489 | controller | 2026-06-10T07:52:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:52:43.336780 | controller | 2026-06-10T07:52:43-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:52:43.728203 | controller | 2026-06-10T07:52:43-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:52:43.728268 | controller | 2026-06-10T07:52:43-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:52:44.742255 | controller | 2026-06-10T07:52:44-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:52:44.742370 | controller | 2026-06-10T07:52:44-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:52:45.721201 | controller | 2026-06-10T07:52:45-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:52:45.721276 | controller | 2026-06-10T07:52:45-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 07:52:46.531604 | controller | 2026-06-10T07:52:46-04:00 INFO 2026-06-10 07:52:46.531673 | controller | - Config: Waiting ... 2026-06-10 07:52:46.531680 | controller | - Gateway: OK 2026-06-10 07:52:46.531686 | controller | - GitServer: OK 2026-06-10 07:52:46.531691 | controller | - HoundSearch: OK 2026-06-10 07:52:46.531695 | controller | - LogJuicer: OK 2026-06-10 07:52:46.531699 | controller | - Logserver: OK 2026-06-10 07:52:46.531704 | controller | - MariaDB: OK 2026-06-10 07:52:46.531709 | controller | - NodePoolBuilder: OK 2026-06-10 07:52:46.531713 | controller | - NodePoolLauncher: OK 2026-06-10 07:52:46.531718 | controller | - Zookeeper: OK 2026-06-10 07:52:46.531723 | controller | - Zuul: Waiting ... 2026-06-10 07:52:46.531728 | controller | - |_ Config Secrets: OK 2026-06-10 07:52:46.531733 | controller | - |_ Executor: OK 2026-06-10 07:52:46.531738 | controller | - |_ Merger: OK 2026-06-10 07:52:46.531742 | controller | - |_ Scheduler: OK 2026-06-10 07:52:46.531747 | controller | - |_ Web: Waiting ... 2026-06-10 07:52:46.531752 | controller | 2026-06-10 07:52:46.531763 | controller | 2026-06-10T07:52:46-04:00 INFO [attempt #13] Waiting 5s for the next reconcile call ... 2026-06-10 07:52:51.540827 | controller | 2026-06-10T07:52:51-04:00 INFO Deleting object, name: ca-cert 2026-06-10 07:52:51.542615 | controller | 2026-06-10T07:52:51-04:00 INFO Nothing to clean up. 2026-06-10 07:52:52.155627 | controller | 2026-06-10T07:52:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 07:52:52.155710 | controller | 2026-06-10T07:52:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 07:52:52.172448 | controller | 2026-06-10T07:52:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 07:52:52.172525 | controller | 2026-06-10T07:52:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 07:52:52.555558 | controller | 2026-06-10T07:52:52-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 07:52:52.953959 | controller | 2026-06-10T07:52:52-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 07:52:52.954034 | controller | 2026-06-10T07:52:52-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 07:52:53.957832 | controller | 2026-06-10T07:52:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 07:52:53.957900 | controller | 2026-06-10T07:52:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 07:52:54.943705 | controller | 2026-06-10T07:52:54-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 07:52:54.943788 | controller | 2026-06-10T07:52:54-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 07:52:56.151199 | controller | 2026-06-10T07:52:56-04:00 INFO 2026-06-10 07:52:56.151266 | controller | - Config: OK 2026-06-10 07:52:56.151274 | controller | - Gateway: OK 2026-06-10 07:52:56.151280 | controller | - GitServer: OK 2026-06-10 07:52:56.151285 | controller | - HoundSearch: OK 2026-06-10 07:52:56.151290 | controller | - LogJuicer: OK 2026-06-10 07:52:56.151294 | controller | - Logserver: OK 2026-06-10 07:52:56.151299 | controller | - MariaDB: OK 2026-06-10 07:52:56.151304 | controller | - NodePoolBuilder: OK 2026-06-10 07:52:56.151310 | controller | - NodePoolLauncher: OK 2026-06-10 07:52:56.151315 | controller | - Zookeeper: OK 2026-06-10 07:52:56.151320 | controller | - Zuul: OK 2026-06-10 07:52:56.151325 | controller | 2026-06-10 07:52:56.151360 | controller | 2026-06-10T07:52:56-04:00 INFO Updating controller configmap ... 2026-06-10 07:52:56.542822 | controller | 2026-06-10T07:52:56-04:00 INFO Standalone reconcile done. 2026-06-10 11:52:56.831768 | controller | ok: Runtime: 0:04:39.334867 2026-06-10 11:52:56.848629 | 2026-06-10 11:52:56.848770 | TASK [health-check/test-backup-restore : Fetch the last build ID reported by Zuul web (after restore)] 2026-06-10 11:52:57.433851 | controller | ok: OK (698 bytes) 2026-06-10 11:52:57.444819 | 2026-06-10 11:52:57.444990 | TASK [health-check/test-backup-restore : Ensure Zuul SQL DB well restored by checking last build ID] 2026-06-10 11:52:57.484180 | controller | ok: All assertions passed 2026-06-10 11:52:57.493185 | 2026-06-10 11:52:57.493310 | TASK [Run a Zuul job workflow in order to validate the restored system] 2026-06-10 11:52:57.523464 | controller | ok 2026-06-10 11:52:57.555287 | 2026-06-10 11:52:57.555405 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 11:52:57.589341 | controller | ok 2026-06-10 11:52:57.614999 | 2026-06-10 11:52:57.615094 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the internal tenant] 2026-06-10 11:52:58.167225 | controller | ok: OK (2 bytes) 2026-06-10 11:52:58.179928 | 2026-06-10 11:52:58.180074 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the internal tenant] 2026-06-10 11:52:58.218030 | controller | skipping: Conditional result was False 2026-06-10 11:52:58.238947 | 2026-06-10 11:52:58.239087 | TASK [health-check/config-update-zuul : Set random value] 2026-06-10 11:52:58.450411 | controller | ok 2026-06-10 11:52:58.459159 | 2026-06-10 11:52:58.459317 | TASK [health-check/config-update-zuul : Create vars to generate and submit resources addition] 2026-06-10 11:52:58.505723 | controller | ok 2026-06-10 11:52:58.514961 | 2026-06-10 11:52:58.515084 | TASK [ansible.builtin.include_role : create-gerrit-repository] 2026-06-10 11:52:58.551543 | controller | ok 2026-06-10 11:52:58.591549 | 2026-06-10 11:52:58.591642 | TASK [create-gerrit-repository : Create gerrit repository named ichiban-config-gAN85AcW] 2026-06-10 07:53:00.047310 | controller | 2026-06-10 11:53:00,046 INFO managesf.model.yamlbkd.engine - Resource engine is using /tmp/direct-apply as workdir 2026-06-10 07:53:00.047543 | controller | 2026-06-10 11:53:00,046 INFO managesf.model.yamlbkd.engine - Resources engine: direct apply resources requested 2026-06-10 07:53:00.067818 | controller | 2026-06-10 11:53:00,064 INFO managesf.GerritClient - ichiban-config-gAN85AcW-core: creating group (Team core for the ichiban-config-gAN85AcW repo) 2026-06-10 07:53:00.148733 | controller | 2026-06-10 11:53:00,147 INFO managesf.GerritClient - ichiban-config-gAN85AcW-core: deleting group member admin 2026-06-10 07:53:00.173095 | controller | 2026-06-10 11:53:00,172 INFO managesf.GerritClient - ichiban-config-gAN85AcW-ptl: creating group (Team lead for the ichiban-config-gAN85AcW repo) 2026-06-10 07:53:00.266842 | controller | 2026-06-10 11:53:00,265 INFO managesf.GerritClient - ichiban-config-gAN85AcW-ptl: deleting group member admin 2026-06-10 07:53:00.287876 | controller | 2026-06-10 11:53:00,287 INFO managesf.GerritClient - ichiban-config-gAN85AcW-ptl: adding group member admin@sfop.me 2026-06-10 07:53:00.363364 | controller | 2026-06-10 11:53:00,362 INFO managesf.GerritClient - ichiban-config-gAN85AcW: creating project (ichiban-config-gAN85AcW repository) 2026-06-10 07:53:00.473883 | controller | 2026-06-10 11:53:00,472 INFO managesf.services.gerrit.utils - [gerrit] Clone repository ichiban-config-gAN85AcW 2026-06-10 07:53:00.751233 | controller | 2026-06-10 11:53:00,747 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'clone', 'ssh://admin@gerrit-sshd:29418/ichiban-config-gAN85AcW', '/tmp/tmpbqygdy1s/clone-ichiban-config-gAN85AcW'] output 2026-06-10 07:53:00.751296 | controller | 2026-06-10 11:53:00,747 INFO managesf.services.gerrit.utils - Cloning into '/tmp/tmpbqygdy1s/clone-ichiban-config-gAN85AcW'... 2026-06-10 07:53:00.751307 | controller | Total 2 (delta 0), reused 0 (delta 0) 2026-06-10 07:53:00.751317 | controller | 2026-06-10 07:53:01.036023 | controller | 2026-06-10 11:53:01,034 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', '-a'] output 2026-06-10 07:53:01.036093 | controller | 2026-06-10 11:53:01,035 INFO managesf.services.gerrit.utils - 2026-06-10 07:53:01.106507 | controller | 2026-06-10 11:53:01,105 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on config for repository ichiban-config-gAN85AcW 2026-06-10 07:53:01.310794 | controller | 2026-06-10 11:53:01,309 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', 'origin', 'refs/meta/config:refs/remotes/origin/meta/config'] output 2026-06-10 07:53:01.310845 | controller | 2026-06-10 11:53:01,310 INFO managesf.services.gerrit.utils - Total 4 (delta 0), reused 0 (delta 0) 2026-06-10 07:53:01.310853 | controller | From ssh://gerrit-sshd:29418/ichiban-config-gAN85AcW 2026-06-10 07:53:01.310860 | controller | * [new ref] refs/meta/config -> origin/meta/config 2026-06-10 07:53:01.310865 | controller | 2026-06-10 07:53:01.317290 | controller | 2026-06-10 11:53:01,316 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'meta/config'] output 2026-06-10 07:53:01.317312 | controller | 2026-06-10 11:53:01,316 INFO managesf.services.gerrit.utils - Switched to a new branch 'meta/config' 2026-06-10 07:53:01.317318 | controller | branch 'meta/config' set up to track 'origin/meta/config'. 2026-06-10 07:53:01.317323 | controller | 2026-06-10 07:53:01.317337 | controller | 2026-06-10 11:53:01,316 INFO managesf.services.gerrit.utils - [gerrit] Add file project.config to index 2026-06-10 07:53:01.321908 | controller | 2026-06-10 11:53:01,321 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'project.config'] output 2026-06-10 07:53:01.321927 | controller | 2026-06-10 11:53:01,321 INFO managesf.services.gerrit.utils - 2026-06-10 07:53:01.321935 | controller | 2026-06-10 11:53:01,321 INFO managesf.services.gerrit.utils - [gerrit] Add file groups to index 2026-06-10 07:53:01.326926 | controller | 2026-06-10 11:53:01,326 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'groups'] output 2026-06-10 07:53:01.326950 | controller | 2026-06-10 11:53:01,326 INFO managesf.services.gerrit.utils - 2026-06-10 07:53:01.331834 | controller | 2026-06-10 11:53:01,331 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-10 07:53:01.331857 | controller | 2026-06-10 11:53:01,331 INFO managesf.services.gerrit.utils - M groups 2026-06-10 07:53:01.331873 | controller | M project.config 2026-06-10 07:53:01.331878 | controller | 2026-06-10 07:53:01.340588 | controller | 2026-06-10 11:53:01,340 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mProvides ACL and Groups'] output 2026-06-10 07:53:01.340705 | controller | 2026-06-10 11:53:01,340 INFO managesf.services.gerrit.utils - [meta/config bc33e19] Provides ACL and Groups 2026-06-10 07:53:01.340715 | controller | 2 files changed, 31 insertions(+), 9 deletions(-) 2026-06-10 07:53:01.340720 | controller | 2026-06-10 07:53:01.529885 | controller | 2026-06-10 11:53:01,528 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'meta/config:meta/config'] output 2026-06-10 07:53:01.529947 | controller | 2026-06-10 11:53:01,528 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 07:53:01.529955 | controller | To ssh://gerrit-sshd:29418/ichiban-config-gAN85AcW 2026-06-10 07:53:01.529961 | controller | 42140f6..bc33e19 meta/config -> refs/meta/config 2026-06-10 07:53:01.529966 | controller | 2026-06-10 07:53:01.529973 | controller | 2026-06-10 11:53:01,528 INFO managesf.services.gerrit.utils - [gerrit] Push on config for repository ichiban-config-gAN85AcW 2026-06-10 07:53:01.529978 | controller | 2026-06-10 11:53:01,529 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on master for repository ichiban-config-gAN85AcW 2026-06-10 07:53:01.539420 | controller | 2026-06-10 11:53:01,538 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'master'] output 2026-06-10 07:53:01.539469 | controller | 2026-06-10 11:53:01,538 INFO managesf.services.gerrit.utils - Switched to branch 'master' 2026-06-10 07:53:01.539480 | controller | Your branch is up to date with 'origin/master'. 2026-06-10 07:53:01.539489 | controller | 2026-06-10 07:53:01.545659 | controller | 2026-06-10 11:53:01,544 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'reset', '--hard', 'origin/master'] output 2026-06-10 07:53:01.545721 | controller | 2026-06-10 11:53:01,544 INFO managesf.services.gerrit.utils - HEAD is now at 80540d2 Initial empty repository 2026-06-10 07:53:01.545733 | controller | 2026-06-10 07:53:01.545742 | controller | 2026-06-10 11:53:01,544 INFO managesf.services.gerrit.utils - [gerrit] Add file .gitreview to index 2026-06-10 07:53:01.550783 | controller | 2026-06-10 11:53:01,549 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', '.gitreview'] output 2026-06-10 07:53:01.550823 | controller | 2026-06-10 11:53:01,550 INFO managesf.services.gerrit.utils - 2026-06-10 07:53:01.554873 | controller | 2026-06-10 11:53:01,554 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-10 07:53:01.554915 | controller | 2026-06-10 11:53:01,554 INFO managesf.services.gerrit.utils - A .gitreview 2026-06-10 07:53:01.554922 | controller | 2026-06-10 07:53:01.562902 | controller | 2026-06-10 11:53:01,562 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mManageSF commit'] output 2026-06-10 07:53:01.562943 | controller | 2026-06-10 11:53:01,562 INFO managesf.services.gerrit.utils - [master 1d5258e] ManageSF commit 2026-06-10 07:53:01.562951 | controller | 1 file changed, 5 insertions(+) 2026-06-10 07:53:01.562957 | controller | create mode 100644 .gitreview 2026-06-10 07:53:01.562962 | controller | 2026-06-10 07:53:01.771809 | controller | 2026-06-10 11:53:01,770 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'master'] output 2026-06-10 07:53:01.771857 | controller | 2026-06-10 11:53:01,771 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 07:53:01.771866 | controller | To ssh://gerrit-sshd:29418/ichiban-config-gAN85AcW 2026-06-10 07:53:01.771871 | controller | 80540d2..1d5258e master -> master 2026-06-10 07:53:01.771875 | controller | 2026-06-10 07:53:01.771881 | controller | 2026-06-10 11:53:01,771 INFO managesf.services.gerrit.utils - [gerrit] Push on master for repository ichiban-config-gAN85AcW 2026-06-10 07:53:01.771885 | controller | 2026-06-10 11:53:01,771 INFO managesf.services.gerrit.utils - [gerrit] List remote branches 2026-06-10 07:53:01.779355 | controller | 2026-06-10 11:53:01,778 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'branch', '-rv', '--abbrev=40'] output 2026-06-10 07:53:01.779389 | controller | 2026-06-10 11:53:01,778 INFO managesf.services.gerrit.utils - origin/HEAD -> origin/master 2026-06-10 07:53:01.779396 | controller | origin/master 1d5258e2c509792641bafb08e652927368b5de2b ManageSF commit 2026-06-10 07:53:01.779402 | controller | origin/meta/config 42140f6adf6592ce341838302e46f225cbd13efc Created project 2026-06-10 07:53:01.779407 | controller | 2026-06-10 07:53:01.779416 | controller | 2026-06-10 11:53:01,778 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-gAN85AcW-acl] is going to be created. 2026-06-10 07:53:01.779430 | controller | 2026-06-10 11:53:01,778 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-gAN85AcW-core] is going to be created. 2026-06-10 07:53:01.779443 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-gAN85AcW-ptl] is going to be created. 2026-06-10 07:53:01.779620 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-gAN85AcW] is going to be created. 2026-06-10 07:53:01.779643 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-gAN85AcW-core] will be created. 2026-06-10 07:53:01.779648 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-gAN85AcW-core] has been created. 2026-06-10 07:53:01.779658 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-gAN85AcW-ptl] will be created. 2026-06-10 07:53:01.779663 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-gAN85AcW-ptl] has been created. 2026-06-10 07:53:01.779668 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-gAN85AcW-acl] will be created. 2026-06-10 07:53:01.779910 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-gAN85AcW-acl] has been created. 2026-06-10 07:53:01.779930 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-gAN85AcW] will be created. 2026-06-10 07:53:01.779935 | controller | 2026-06-10 11:53:01,779 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-gAN85AcW] has been created. 2026-06-10 07:53:01.780244 | controller | 2026-06-10 07:53:01.780261 | controller | === Resources actions === 2026-06-10 07:53:01.780266 | controller | 2026-06-10 07:53:01.780271 | controller | Resource [type: acls, ID: ichiban-config-gAN85AcW-acl] is going to be created. 2026-06-10 07:53:01.780277 | controller | Resource [type: groups, ID: ichiban-config-gAN85AcW-core] is going to be created. 2026-06-10 07:53:01.780282 | controller | Resource [type: groups, ID: ichiban-config-gAN85AcW-ptl] is going to be created. 2026-06-10 07:53:01.780287 | controller | Resource [type: repos, ID: ichiban-config-gAN85AcW] is going to be created. 2026-06-10 07:53:01.780291 | controller | Resource [type: groups, ID: ichiban-config-gAN85AcW-core] will be created. 2026-06-10 07:53:01.780296 | controller | Resource [type: groups, ID: ichiban-config-gAN85AcW-core] has been created. 2026-06-10 07:53:01.780301 | controller | Resource [type: groups, ID: ichiban-config-gAN85AcW-ptl] will be created. 2026-06-10 07:53:01.780306 | controller | Resource [type: groups, ID: ichiban-config-gAN85AcW-ptl] has been created. 2026-06-10 07:53:01.780311 | controller | Resource [type: acls, ID: ichiban-config-gAN85AcW-acl] will be created. 2026-06-10 07:53:01.780322 | controller | Resource [type: acls, ID: ichiban-config-gAN85AcW-acl] has been created. 2026-06-10 07:53:01.780328 | controller | Resource [type: repos, ID: ichiban-config-gAN85AcW] will be created. 2026-06-10 07:53:01.780332 | controller | Resource [type: repos, ID: ichiban-config-gAN85AcW] has been created. 2026-06-10 11:53:02.131985 | controller | ok: Runtime: 0:00:03.146960 2026-06-10 11:53:02.143599 | 2026-06-10 11:53:02.143725 | TASK [create-gerrit-repository : Ensure repository exists in Gerrit] 2026-06-10 11:53:02.684967 | controller | ok: OK (752 bytes) 2026-06-10 11:53:02.696404 | 2026-06-10 11:53:02.696529 | TASK [health-check/config-update-zuul : Set bad Zuul tenant config] 2026-06-10 11:53:02.956104 | controller | changed: Block inserted 2026-06-10 11:53:02.966025 | 2026-06-10 11:53:02.966163 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 11:53:03.279691 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.008101 2026-06-10 11:53:03.280221 | controller | changed: All items complete 2026-06-10 11:53:03.280283 | 2026-06-10 11:53:03.615850 | controller | [master 6c3d79c] Add zuul/main.yaml 2026-06-10 11:53:03.615954 | controller | 1 file changed, 2 insertions(+), 5 deletions(-) 2026-06-10 11:53:03.616017 | controller | ok: Item: git commit -m 'Add zuul/main.yaml' Runtime: 0:00:00.073396 2026-06-10 11:53:03.636003 | 2026-06-10 11:53:03.636261 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 11:53:03.672096 | controller | ok 2026-06-10 11:53:03.723390 | 2026-06-10 11:53:03.723561 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 11:53:03.756643 | controller | ok 2026-06-10 11:53:03.765906 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 11:53:03.774621 | 2026-06-10 11:53:03.774706 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 07:53:04.794174 | controller | remote: 2026-06-10 07:53:04.794234 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:53:04.794240 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:53:04.794245 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 07:53:04.794250 | controller | remote: 2026-06-10 07:53:04.794254 | controller | remote: SUCCESS 2026-06-10 07:53:04.794259 | controller | remote: 2026-06-10 07:53:04.794265 | controller | remote: https://gerrit.sfop.me/c/config/+/7 Add zuul/main.yaml [NEW] 2026-06-10 07:53:04.794270 | controller | remote: 2026-06-10 07:53:04.794274 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 07:53:04.794278 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:53:04.943260 | controller | ok: Runtime: 0:00:00.629820 2026-06-10 11:53:04.953215 | 2026-06-10 11:53:04.953381 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 07:53:05.280270 | controller | Ib1531789be7e6bcedc85e4a205c4a611323fc146 2026-06-10 11:53:05.504608 | controller | ok: Runtime: 0:00:00.008746 2026-06-10 11:53:05.514744 | 2026-06-10 11:53:05.514879 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 11:53:06.079825 | controller | ok: OK (47 bytes) 2026-06-10 11:53:06.093585 | 2026-06-10 11:53:06.093723 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 11:53:06.140065 | controller | ok 2026-06-10 11:53:06.155454 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 11:53:06.173113 | 2026-06-10 11:53:06.173261 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 11:53:06.677200 | controller | ok: OK (7142 bytes) 2026-06-10 11:53:06.687889 | 2026-06-10 11:53:06.688034 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 11:53:06.735177 | controller | ok 2026-06-10 11:53:06.751556 | 2026-06-10 11:53:06.751731 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 11:53:59.704549 | controller | ok: OK (1163 bytes) 2026-06-10 11:53:59.715858 | 2026-06-10 11:53:59.716011 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 11:53:59.752863 | controller | skipping: Conditional result was False 2026-06-10 11:53:59.766728 | 2026-06-10 11:53:59.766877 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 11:53:59.795473 | controller | skipping: Conditional result was False 2026-06-10 11:53:59.810671 | 2026-06-10 11:53:59.810810 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 11:53:59.839978 | controller | skipping: Conditional result was False 2026-06-10 11:53:59.861281 | 2026-06-10 11:53:59.861503 | TASK [health-check/config-update-zuul : Set Zuul tenant config] 2026-06-10 11:54:00.116846 | controller | changed: Block inserted 2026-06-10 11:54:00.126613 | 2026-06-10 11:54:00.126747 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 11:54:00.394420 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.005156 2026-06-10 11:54:00.685618 | controller | [master 762eac1] Add zuul/main.yaml 2026-06-10 11:54:00.685800 | controller | Date: Wed Jun 10 07:53:03 2026 -0400 2026-06-10 11:54:00.685872 | controller | 1 file changed, 2 insertions(+), 2 deletions(-) 2026-06-10 11:54:00.685931 | controller | ok: Item: git commit --amend --no-edit Runtime: 0:00:00.043697 2026-06-10 11:54:00.706402 | 2026-06-10 11:54:00.706570 | TASK [Submit change and verify Zuul console output] 2026-06-10 11:54:00.747565 | controller | ok 2026-06-10 11:54:00.798119 | 2026-06-10 11:54:00.798216 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 11:54:00.831435 | controller | ok 2026-06-10 11:54:00.840648 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 11:54:00.849030 | 2026-06-10 11:54:00.849114 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 07:54:01.571923 | controller | remote: 2026-06-10 07:54:01.571981 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 07:54:01.571987 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 07:54:01.571992 | controller | remote: Processing changes: refs: 1, updated: 1, done 2026-06-10 07:54:01.571997 | controller | remote: 2026-06-10 07:54:01.572001 | controller | remote: SUCCESS 2026-06-10 07:54:01.572005 | controller | remote: 2026-06-10 07:54:01.572010 | controller | remote: https://gerrit.sfop.me/c/config/+/7 Add zuul/main.yaml 2026-06-10 07:54:01.572015 | controller | remote: 2026-06-10 07:54:01.572018 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 07:54:01.572022 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:54:01.889770 | controller | ok: Runtime: 0:00:00.487984 2026-06-10 11:54:01.900290 | 2026-06-10 11:54:01.900455 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 07:54:02.157327 | controller | Ib1531789be7e6bcedc85e4a205c4a611323fc146 2026-06-10 11:54:02.446601 | controller | ok: Runtime: 0:00:00.006179 2026-06-10 11:54:02.456549 | 2026-06-10 11:54:02.456683 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 11:54:02.991160 | controller | ok: OK (47 bytes) 2026-06-10 11:54:03.003624 | 2026-06-10 11:54:03.003761 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 11:54:03.033324 | controller | skipping: Conditional result was False 2026-06-10 11:54:03.046192 | 2026-06-10 11:54:03.046332 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 11:54:03.092528 | controller | ok 2026-06-10 11:54:03.104938 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-zuul-console-success.yaml 2026-06-10 11:54:03.122633 | 2026-06-10 11:54:03.122716 | TASK [health-check/repo-submit-change : Wait for config-update job] 2026-06-10 11:54:54.739684 | controller | ok: Runtime: 0:00:00.041986 2026-06-10 11:54:54.749426 | 2026-06-10 11:54:54.749553 | TASK [health-check/repo-submit-change : Connect and validate console stream] 2026-06-10 07:54:55.142460 | controller | 2026-06-10 11:54:51.391871 | Job console starting 2026-06-10 07:54:55.142568 | controller | 2026-06-10 11:54:51.392371 | Updating git repos 2026-06-10 07:54:55.142578 | controller | 2026-06-10 11:54:51.447079 | Cloning repos into workspace 2026-06-10 07:54:55.142585 | controller | 2026-06-10 11:54:52.162947 | Restoring repo states 2026-06-10 07:54:55.142593 | controller | 2026-06-10 11:54:52.168224 | Merging changes 2026-06-10 07:54:55.142601 | controller | 2026-06-10 11:54:52.168237 | Checking out repos 2026-06-10 07:54:55.142607 | controller | 2026-06-10 11:54:52.188225 | Preparing playbooks 2026-06-10 07:54:55.142613 | controller | 2026-06-10 11:54:52.726121 | Running Ansible setup 2026-06-10 07:54:55.142619 | controller | 2026-06-10 07:54:55.642805 | controller | 2026-06-10 11:54:55.541727 | PRE-RUN START: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-10 07:54:55.642871 | controller | 2026-06-10 07:54:57.143691 | controller | 2026-06-10 11:54:57.045302 | 2026-06-10 07:54:57.143738 | controller | 2026-06-10 11:54:57.045471 | PLAY [localhost] 2026-06-10 07:54:57.143744 | controller | 2026-06-10 11:54:57.053900 | 2026-06-10 07:54:57.143748 | controller | 2026-06-10 11:54:57.054017 | TASK [Gathering Facts] 2026-06-10 07:54:57.143752 | controller | 2026-06-10 07:54:59.144370 | controller | 2026-06-10 11:54:58.840727 | localhost | ok 2026-06-10 07:54:59.144431 | controller | 2026-06-10 11:54:58.849419 | 2026-06-10 07:54:59.144438 | controller | 2026-06-10 11:54:58.849576 | TASK [Setup log path fact] 2026-06-10 07:54:59.144443 | controller | 2026-06-10 11:54:58.865716 | localhost | ok 2026-06-10 07:54:59.144447 | controller | 2026-06-10 11:54:58.877490 | 2026-06-10 07:54:59.144452 | controller | 2026-06-10 11:54:58.877624 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 07:54:59.144456 | controller | 2026-06-10 11:54:58.943532 | localhost | ok 2026-06-10 07:54:59.144461 | controller | 2026-06-10 11:54:58.950674 | 2026-06-10 07:54:59.144472 | controller | 2026-06-10 11:54:58.950823 | TASK [emit-job-header : Print job information] 2026-06-10 07:54:59.144477 | controller | 2026-06-10 11:54:58.976791 | # Job Information 2026-06-10 07:54:59.144482 | controller | 2026-06-10 11:54:58.976944 | Ansible Version: 2.16.18 2026-06-10 07:54:59.144488 | controller | 2026-06-10 11:54:58.976974 | Job: config-update 2026-06-10 07:54:59.144493 | controller | 2026-06-10 11:54:58.976993 | Pipeline: post 2026-06-10 07:54:59.144514 | controller | 2026-06-10 11:54:58.977010 | Executor: zuul-executor-0.zuul-executor-headless.sf.svc.cluster.local 2026-06-10 07:54:59.144519 | controller | 2026-06-10 11:54:58.977027 | Triggered by: https://gerrit.sfop.me/plugins/gitiles/config/+/762eac1635c75f4518122986735a6a67fc2ff2e9^!/ 2026-06-10 07:54:59.144524 | controller | 2026-06-10 11:54:58.977046 | Log URL (when completed): https://sfop.me/logs/96a/internal/96a1f117da9743318cdaa083f4bb2d5e/ 2026-06-10 07:54:59.144529 | controller | 2026-06-10 11:54:58.977062 | Event 2026-06-10 07:54:59.144539 | controller | ID: 59a1cd02963249c1845f711fabececd0 2026-06-10 07:54:59.144544 | controller | 2026-06-10 11:54:58.979736 | 2026-06-10 07:54:59.144548 | controller | 2026-06-10 11:54:58.979829 | LOOP [emit-job-header : Print node information] 2026-06-10 07:54:59.144552 | controller | 2026-06-10 11:54:59.046824 | 2026-06-10 07:54:59.144556 | controller | 2026-06-10 11:54:59.046981 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-10 07:54:59.144561 | controller | 2026-06-10 11:55:00.299693 | controller | ok: Runtime: 0:00:05.020887 2026-06-10 11:55:00.312705 | 2026-06-10 11:55:00.312874 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 11:55:00.359077 | controller | ok 2026-06-10 11:55:00.370882 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 11:55:00.389057 | 2026-06-10 11:55:00.389190 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-10 07:55:00.672458 | controller | 762eac1635c75f4518122986735a6a67fc2ff2e9 2026-06-10 11:55:00.927535 | controller | ok: Runtime: 0:00:00.006503 2026-06-10 11:55:00.938991 | 2026-06-10 11:55:00.939154 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 11:56:14.811944 | controller | ok: OK (1143 bytes) 2026-06-10 11:56:14.824517 | 2026-06-10 11:56:14.824661 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 11:56:14.861734 | controller | skipping: Conditional result was False 2026-06-10 11:56:14.971399 | 2026-06-10 11:56:14.971558 | TASK [health-check/config-update-zuul : ansible.builtin.set_fact] 2026-06-10 11:56:15.000955 | controller | ok 2026-06-10 11:56:15.005491 | 2026-06-10 11:56:15.005558 | TASK [Check that Zuul has pushed the job console on the logserver] 2026-06-10 11:56:15.024391 | controller | ok 2026-06-10 11:56:15.055631 | 2026-06-10 11:56:15.055773 | TASK [health-check/ensure-job-result-artifacts : Ensure job console (job-output.txt.gz) exists in Logserver] 2026-06-10 11:56:15.576096 | controller | ok: OK (4747 bytes) 2026-06-10 11:56:15.586251 | 2026-06-10 11:56:15.586416 | TASK [health-check/ensure-job-result-artifacts : Ensure internal logserver rewrite from job-output.txt to job-output.txt.gz] 2026-06-10 11:56:16.082530 | controller | ok: OK (4747 bytes) 2026-06-10 11:56:16.101767 | 2026-06-10 11:56:16.101980 | TASK [health-check/config-update-zuul : Get Zuul projects list] 2026-06-10 11:56:16.636906 | controller | ok: OK (144 bytes) 2026-06-10 11:56:16.647667 | 2026-06-10 11:56:16.647810 | TASK [health-check/config-update-zuul : Ensure project exists in Zuul (fail if not)] 2026-06-10 11:56:16.701213 | controller | skipping: Conditional result was False 2026-06-10 11:56:16.731200 | 2026-06-10 11:56:16.731389 | LOOP [health-check/test-nodepool-launcher-pod : Reset config dir to HEAD] 2026-06-10 11:56:17.092783 | controller | Fetching origin 2026-06-10 11:56:17.093236 | controller | From https://gerrit.sfop.me/a/demo-project 2026-06-10 11:56:17.093279 | controller | 6f67f48..c8f5c5f master -> origin/master 2026-06-10 11:56:17.093306 | controller | Fetching gerrit 2026-06-10 11:56:17.093374 | controller | ok: Item: git remote update Runtime: 0:00:00.101181 2026-06-10 11:56:17.093429 | controller | changed: All items complete 2026-06-10 11:56:17.093458 | 2026-06-10 11:56:17.321572 | controller | HEAD is now at c8f5c5f Update demo-job to sleep 1 second 2026-06-10 11:56:17.321768 | controller | ok: Item: git reset --hard origin/master Runtime: 0:00:00.005855 2026-06-10 11:56:17.341542 | 2026-06-10 11:56:17.341762 | LOOP [health-check/test-nodepool-launcher-pod : Ensure playbooks and zuul.d dir exists] 2026-06-10 11:56:17.616801 | controller | ok: "playbooks" 2026-06-10 11:56:17.811416 | controller | changed: "zuul.d" 2026-06-10 11:56:17.829182 | 2026-06-10 11:56:17.829326 | TASK [health-check/test-nodepool-launcher-pod : Create new zuul config] 2026-06-10 11:56:18.570260 | controller | changed 2026-06-10 11:56:18.580889 | 2026-06-10 11:56:18.581018 | TASK [health-check/test-nodepool-launcher-pod : Create simple playbook] 2026-06-10 11:56:19.271176 | controller | changed 2026-06-10 11:56:19.284274 | 2026-06-10 11:56:19.284563 | LOOP [health-check/test-nodepool-launcher-pod : Create git commit message] 2026-06-10 11:56:19.590127 | controller | ok: Item: git add . Runtime: 0:00:00.007385 2026-06-10 11:56:19.590699 | controller | changed: All items complete 2026-06-10 11:56:19.590760 | 2026-06-10 11:56:19.862010 | controller | [test_short_job 076029a] Provide a demo job for testing the local pod spawning 2026-06-10 11:56:19.862176 | controller | 2 files changed, 18 insertions(+) 2026-06-10 11:56:19.862228 | controller | create mode 100644 playbooks/microshift-pod.yml 2026-06-10 11:56:19.862268 | controller | create mode 100644 zuul.d/config.yaml 2026-06-10 11:56:19.862324 | controller | ok: Item: git commit -m 'Provide a demo job for testing the local pod spawning' Runtime: 0:00:00.048764 2026-06-10 11:56:19.881825 | 2026-06-10 11:56:19.882031 | TASK [Submit change and validate job success] 2026-06-10 11:56:19.918595 | controller | ok 2026-06-10 11:56:19.955752 | 2026-06-10 11:56:19.955883 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 11:56:19.991545 | controller | ok 2026-06-10 11:56:20.000902 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 11:56:20.008881 | 2026-06-10 11:56:20.008966 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 07:56:20.782096 | controller | remote: 2026-06-10 07:56:20.782240 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:56:20.782252 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:56:20.782258 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 07:56:20.782266 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 07:56:20.782273 | controller | remote: commit 076029a: warning: subject >50 characters; use shorter first paragraph 2026-06-10 07:56:20.782279 | controller | remote: 2026-06-10 07:56:20.782283 | controller | remote: SUCCESS 2026-06-10 07:56:20.782288 | controller | remote: 2026-06-10 07:56:20.782295 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/8 Provide a demo job for testing the local pod spawning [NEW] 2026-06-10 07:56:20.782301 | controller | remote: 2026-06-10 07:56:20.782305 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 07:56:20.782311 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 11:56:21.045318 | controller | ok: Runtime: 0:00:00.543276 2026-06-10 11:56:21.050275 | 2026-06-10 11:56:21.050339 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 07:56:21.415471 | controller | Ic272008b41f232a103bbc922545230d8c2d83e92 2026-06-10 11:56:21.579897 | controller | ok: Runtime: 0:00:00.011785 2026-06-10 11:56:21.589658 | 2026-06-10 11:56:21.589826 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 11:56:21.617469 | controller | skipping: Conditional result was False 2026-06-10 11:56:21.631239 | 2026-06-10 11:56:21.631432 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 11:56:21.659175 | controller | skipping: Conditional result was False 2026-06-10 11:56:21.672089 | 2026-06-10 11:56:21.672243 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 11:56:21.699864 | controller | skipping: Conditional result was False 2026-06-10 11:56:21.716688 | 2026-06-10 11:56:21.716856 | TASK [health-check/repo-submit-change : Ensure a success for microshift-pod post job] 2026-06-10 11:56:21.744854 | controller | skipping: Conditional result was False 2026-06-10 11:56:21.759871 | 2026-06-10 11:56:21.760084 | TASK [health-check/repo-submit-change : Ensure a success for microshift-pod job] 2026-06-10 11:56:21.808749 | controller | ok 2026-06-10 11:56:21.826614 | controller | included: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-success.yaml 2026-06-10 11:56:21.841936 | 2026-06-10 11:56:21.842084 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 11:56:22.355173 | controller | ok: OK (2986 bytes) 2026-06-10 11:56:22.365733 | 2026-06-10 11:56:22.365876 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 11:56:22.426484 | controller | ok 2026-06-10 11:56:22.439446 | 2026-06-10 11:56:22.439583 | TASK [health-check/repo-submit-change : Wait microshift-pod SUCCESS result using zuul-web API] 2026-06-10 11:57:57.191257 | controller | ok: OK (798 bytes) 2026-06-10 11:57:57.203152 | 2026-06-10 11:57:57.203211 | TASK [health-check/test-volumestats-sidecar : ensure logserver deployment is ready] 2026-06-10 11:57:57.731763 | controller | ok: Runtime: 0:00:00.067837 2026-06-10 11:57:57.741062 | 2026-06-10 11:57:57.741189 | TASK [health-check/test-volumestats-sidecar : Start port forwarding to logserver sidecar] 2026-06-10 11:57:58.558596 | controller | ok: ERROR (ignored) 2026-06-10 11:57:58.559022 | controller | { 2026-06-10 11:57:58.559086 | controller | "ansible_job_id": "j302293749571.214691", 2026-06-10 11:57:58.559128 | controller | "finished": 0, 2026-06-10 11:57:58.559164 | controller | "results_file": "/home/zuul-worker/.ansible_async/j302293749571.214691", 2026-06-10 11:57:58.559203 | controller | "started": 1 2026-06-10 11:57:58.559237 | controller | } 2026-06-10 11:57:58.567779 | 2026-06-10 11:57:58.567915 | TASK [health-check/test-volumestats-sidecar : Pause to ensure the port forwarding is up] 2026-06-10 11:58:18.615526 | controller | ok 2026-06-10 11:58:18.624819 | 2026-06-10 11:58:18.624958 | TASK [health-check/test-volumestats-sidecar : Ensure volume info is exposed by metrics] 2026-06-10 11:58:19.229891 | controller | ok: OK (unknown bytes) 2026-06-10 11:58:19.249540 | 2026-06-10 11:58:19.249663 | TASK [Register CLI version] 2026-06-10 07:58:19.798578 | controller | v0.0.67-11-g5feace1e 2026-06-10 11:58:20.292496 | controller | ok: Runtime: 0:00:00.269868 2026-06-10 11:58:20.301404 | 2026-06-10 11:58:20.301539 | TASK [ensure annotations are present on sf-standalone-owner] 2026-06-10 11:58:20.848238 | controller | ok: Runtime: 0:00:00.062215 2026-06-10 11:58:20.850946 | 2026-06-10 11:58:20.851011 | PLAY RECAP 2026-06-10 11:58:20.851067 | controller | ok: 367 changed: 219 unreachable: 0 failed: 0 skipped: 57 rescued: 0 ignored: 0 2026-06-10 11:58:20.851094 | 2026-06-10 11:58:21.140021 | RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-tests.yaml@master] 2026-06-10 11:58:21.142764 | POST-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/get-minikube-logs.yaml@master] 2026-06-10 11:58:21.781650 | 2026-06-10 11:58:21.781872 | PLAY [controller] 2026-06-10 11:58:21.792334 | 2026-06-10 11:58:21.792441 | TASK [Grab minikube logs] 2026-06-10 11:58:26.826632 | controller | ok: Runtime: 0:00:04.060629 2026-06-10 11:58:26.836070 | 2026-06-10 11:58:26.836226 | TASK [ansible.builtin.include_role : post/get-kubectl-logs] 2026-06-10 11:58:26.862097 | controller | ok 2026-06-10 11:58:26.892444 | 2026-06-10 11:58:26.892593 | TASK [post/get-kubectl-logs : Set proper facts] 2026-06-10 11:58:26.922383 | controller | ok 2026-06-10 11:58:26.927266 | 2026-06-10 11:58:26.927373 | TASK [post/get-kubectl-logs : Get pod logs] 2026-06-10 11:58:34.472883 | controller | ok: Runtime: 0:00:06.998274 2026-06-10 11:58:34.482076 | 2026-06-10 11:58:34.482214 | LOOP [post/get-kubectl-logs : Describe namespace resources] 2026-06-10 11:58:37.993653 | controller | ok: Item: pods Runtime: 0:00:03.290862 2026-06-10 11:58:39.427015 | controller | ok: Item: deployments Runtime: 0:00:01.201815 2026-06-10 11:58:41.038367 | controller | ok: Item: statefulsets Runtime: 0:00:01.403389 2026-06-10 11:58:43.439124 | controller | ok: Item: services Runtime: 0:00:02.201040 2026-06-10 11:58:45.761914 | controller | ok: Item: secrets Runtime: 0:00:02.095372 2026-06-10 11:58:49.119974 | controller | ok: Item: configmaps Runtime: 0:00:03.151006 2026-06-10 11:58:50.199853 | controller | ok: Item: jobs Runtime: 0:00:00.859659 2026-06-10 11:58:52.396198 | controller | ok: Item: pvc Runtime: 0:00:01.981217 2026-06-10 11:58:53.914458 | controller | ok: Item: ingress Runtime: 0:00:01.220511 2026-06-10 11:58:53.930212 | 2026-06-10 11:58:53.930454 | TASK [post/get-kubectl-logs : Get events] 2026-06-10 11:58:55.980850 | controller | ok: Runtime: 0:00:01.002093 2026-06-10 11:58:55.991415 | 2026-06-10 11:58:55.991573 | TASK [post/get-kubectl-logs : Fetch decoded secrets] 2026-06-10 11:58:58.542741 | controller | ok: Runtime: 0:00:01.995614 2026-06-10 11:58:58.557751 | 2026-06-10 11:58:58.557889 | TASK [ansible.builtin.include_role : post/get-logserver-content] 2026-06-10 11:58:58.584593 | controller | ok 2026-06-10 11:58:58.618656 | 2026-06-10 11:58:58.618743 | TASK [post/get-logserver-content : Ensure removal of ~/zuul-output/logs/logserver-content] 2026-06-10 11:58:59.006986 | controller | ok 2026-06-10 11:58:59.015069 | 2026-06-10 11:58:59.015219 | TASK [post/get-logserver-content : Get logserver pod name] 2026-06-10 07:58:59.315201 | controller | logserver-0 2026-06-10 11:58:59.563758 | controller | ok: Runtime: 0:00:00.073471 2026-06-10 11:58:59.576931 | 2026-06-10 11:58:59.577075 | TASK [post/get-logserver-content : Fetch logserver content on locally in ~/zuul-output/logs/logserver-content] 2026-06-10 07:58:59.983718 | controller | tar: Removing leading `/' from member names 2026-06-10 11:59:00.124829 | controller | ok: Runtime: 0:00:00.147979 2026-06-10 11:59:00.139858 | 2026-06-10 11:59:00.140006 | TASK [ansible.builtin.include_role : post/get-repos] 2026-06-10 11:59:00.167584 | controller | ok 2026-06-10 11:59:00.198763 | 2026-06-10 11:59:00.198848 | TASK [post/get-repos : Set proper facts] 2026-06-10 11:59:00.241401 | controller | ok 2026-06-10 11:59:00.250815 | 2026-06-10 11:59:00.250971 | TASK [post/get-repos : Ensure ~/zuul-output/logs/repos/] 2026-06-10 11:59:00.544914 | controller | changed 2026-06-10 11:59:00.557946 | 2026-06-10 11:59:00.558168 | TASK [post/get-repos : Copy repositories] 2026-06-10 11:59:01.219997 | controller | changed 2026-06-10 11:59:01.237707 | 2026-06-10 11:59:01.237868 | TASK [ansible.builtin.include_role : post/get-loki-logs] 2026-06-10 11:59:01.275590 | controller | ok 2026-06-10 11:59:01.314906 | 2026-06-10 11:59:01.315045 | TASK [post/get-loki-logs : Ensure output dir exists for loki logs] 2026-06-10 11:59:01.549060 | controller | changed 2026-06-10 11:59:01.555045 | 2026-06-10 11:59:01.555132 | TASK [post/get-loki-logs : Set realpath for the log directory] 2026-06-10 07:59:01.787212 | controller | /home/zuul-worker/zuul-output/logs/loki-logs 2026-06-10 11:59:02.093323 | controller | ok: Runtime: 0:00:00.003363 2026-06-10 11:59:02.102373 | 2026-06-10 11:59:02.102511 | TASK [post/get-loki-logs : Start port forwarding to loki-http if needed] 2026-06-10 11:59:02.791380 | controller | ok: ERROR (ignored) 2026-06-10 11:59:02.791818 | controller | { 2026-06-10 11:59:02.791905 | controller | "ansible_job_id": "j167760829102.221642", 2026-06-10 11:59:02.792057 | controller | "finished": 0, 2026-06-10 11:59:02.792097 | controller | "results_file": "/home/zuul-worker/.ansible_async/j167760829102.221642", 2026-06-10 11:59:02.792124 | controller | "started": 1 2026-06-10 11:59:02.792148 | controller | } 2026-06-10 11:59:02.798522 | 2026-06-10 11:59:02.798630 | TASK [post/get-loki-logs : Ensure loki is reachable] 2026-06-10 11:59:03.522494 | controller | ok: OK (141 bytes) 2026-06-10 11:59:03.532427 | 2026-06-10 11:59:03.532568 | TASK [post/get-loki-logs : Get aggregated logs prefixed by service] 2026-06-10 11:59:05.578116 | controller | ok: Runtime: 0:00:01.536780 2026-06-10 11:59:05.582064 | 2026-06-10 11:59:05.582125 | TASK [post/get-loki-logs : Change owner and group for the log dir] 2026-06-10 11:59:06.123707 | controller | ok: Runtime: 0:00:00.009752 2026-06-10 11:59:06.134474 | 2026-06-10 11:59:06.134575 | PLAY RECAP 2026-06-10 11:59:06.134658 | controller | ok: 18 changed: 14 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 11:59:06.134699 | 2026-06-10 11:59:06.280080 | POST-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/get-minikube-logs.yaml@master] 2026-06-10 11:59:06.282514 | POST-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/post.yaml@main] 2026-06-10 11:59:06.929629 | 2026-06-10 11:59:06.929776 | PLAY [all] 2026-06-10 11:59:06.939713 | 2026-06-10 11:59:06.939786 | TASK [include_role : fetch-output] 2026-06-10 11:59:06.988237 | controller | ok 2026-06-10 11:59:07.000925 | 2026-06-10 11:59:07.001040 | TASK [fetch-output : Set log path for multiple nodes] 2026-06-10 11:59:07.057327 | controller | skipping: Conditional result was False 2026-06-10 11:59:07.067029 | 2026-06-10 11:59:07.067177 | TASK [fetch-output : Set log path for single node] 2026-06-10 11:59:07.131716 | controller | ok 2026-06-10 11:59:07.136824 | 2026-06-10 11:59:07.136947 | LOOP [fetch-output : Ensure local output dirs] 2026-06-10 11:59:07.674327 | controller -> localhost | changed: "/var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/work/logs" 2026-06-10 11:59:07.973558 | controller -> localhost | changed: "/var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/work/artifacts" 2026-06-10 11:59:08.262377 | controller -> localhost | changed: "/var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/work/docs" 2026-06-10 11:59:08.281659 | 2026-06-10 11:59:08.281890 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-06-10 11:59:09.171276 | controller | changed: 2026-06-10 11:59:09.171771 | controller | .d..t...... ./ 2026-06-10 11:59:09.171944 | controller | >f+++++++++ dmesg.txt 2026-06-10 11:59:09.172014 | controller | >f+++++++++ minikube.logs 2026-06-10 11:59:09.172050 | controller | >f+++++++++ sf-operator-dry-run-lifecycle.log 2026-06-10 11:59:09.172081 | controller | >f+++++++++ sf-operator-dry-run-update.log 2026-06-10 11:59:09.172107 | controller | >f+++++++++ sf-operator.log 2026-06-10 11:59:09.172131 | controller | cd+++++++++ decoded-secrets/ 2026-06-10 11:59:09.172154 | controller | >f+++++++++ decoded-secrets/ingress-nginx--ingress-nginx-admission 2026-06-10 11:59:09.172179 | controller | >f+++++++++ decoded-secrets/kube-system--bootstrap-token-bgy9nz 2026-06-10 11:59:09.172203 | controller | >f+++++++++ decoded-secrets/nodepool--nodepool-token 2026-06-10 11:59:09.172226 | controller | >f+++++++++ decoded-secrets/sf--admin-ssh-key 2026-06-10 11:59:09.172249 | controller | >f+++++++++ decoded-secrets/sf--config-update-secrets 2026-06-10 11:59:09.172272 | controller | >f+++++++++ decoded-secrets/sf--dummygerritsecret 2026-06-10 11:59:09.172295 | controller | >f+++++++++ decoded-secrets/sf--es-basicauth 2026-06-10 11:59:09.172318 | controller | >f+++++++++ decoded-secrets/sf--gerrit-admin-api-key 2026-06-10 11:59:09.172344 | controller | >f+++++++++ decoded-secrets/sf--logserver-keys 2026-06-10 11:59:09.172403 | controller | >f+++++++++ decoded-secrets/sf--logserver-uploader-keys 2026-06-10 11:59:09.172428 | controller | >f+++++++++ decoded-secrets/sf--logserver-uploader-spare-keys 2026-06-10 11:59:09.172453 | controller | >f+++++++++ decoded-secrets/sf--mariadb-config-secrets 2026-06-10 11:59:09.172476 | controller | >f+++++++++ decoded-secrets/sf--mariadb-initdb-secrets 2026-06-10 11:59:09.172499 | controller | >f+++++++++ decoded-secrets/sf--mariadb-root-password 2026-06-10 11:59:09.172522 | controller | >f+++++++++ decoded-secrets/sf--nodepool-builder-ssh-key 2026-06-10 11:59:09.172550 | controller | >f+++++++++ decoded-secrets/sf--nodepool-providers-secrets 2026-06-10 11:59:09.172585 | controller | >f+++++++++ decoded-secrets/sf--self-signed-cert 2026-06-10 11:59:09.173685 | controller | >f+++++++++ decoded-secrets/sf--smtp-secret 2026-06-10 11:59:09.173735 | controller | >f+++++++++ decoded-secrets/sf--zookeeper-client-tls 2026-06-10 11:59:09.173774 | controller | >f+++++++++ decoded-secrets/sf--zookeeper-server-tls 2026-06-10 11:59:09.173811 | controller | >f+++++++++ decoded-secrets/sf--zuul-auth-secret 2026-06-10 11:59:09.173848 | controller | >f+++++++++ decoded-secrets/sf--zuul-config 2026-06-10 11:59:09.173883 | controller | >f+++++++++ decoded-secrets/sf--zuul-db-connection 2026-06-10 11:59:09.173916 | controller | >f+++++++++ decoded-secrets/sf--zuul-gerrit-api-key 2026-06-10 11:59:09.173948 | controller | >f+++++++++ decoded-secrets/sf--zuul-keystore-password 2026-06-10 11:59:09.173980 | controller | >f+++++++++ decoded-secrets/sf--zuul-spare-ssh-key 2026-06-10 11:59:09.174013 | controller | >f+++++++++ decoded-secrets/sf--zuul-ssh-key 2026-06-10 11:59:09.174049 | controller | cd+++++++++ events/ 2026-06-10 11:59:09.174082 | controller | >f+++++++++ events/default.txt 2026-06-10 11:59:09.174106 | controller | >f+++++++++ events/ingress-nginx.txt 2026-06-10 11:59:09.174129 | controller | >f+++++++++ events/kube-node-lease.txt 2026-06-10 11:59:09.174174 | controller | >f+++++++++ events/kube-public.txt 2026-06-10 11:59:09.174200 | controller | >f+++++++++ events/kube-system.txt 2026-06-10 11:59:09.174228 | controller | >f+++++++++ events/nodepool.txt 2026-06-10 11:59:09.174269 | controller | >f+++++++++ events/sf.txt 2026-06-10 11:59:09.174305 | controller | cd+++++++++ items-describe/ 2026-06-10 11:59:09.174341 | controller | >f+++++++++ items-describe/default--configmaps--kube-root-ca.crt.txt 2026-06-10 11:59:09.174410 | controller | >f+++++++++ items-describe/default--services--kubernetes.txt 2026-06-10 11:59:09.174439 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--ingress-nginx-controller.txt 2026-06-10 11:59:09.174486 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--kube-root-ca.crt.txt 2026-06-10 11:59:09.174518 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--tcp-services.txt 2026-06-10 11:59:09.174542 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--udp-services.txt 2026-06-10 11:59:09.174569 | controller | >f+++++++++ items-describe/ingress-nginx--deployments--ingress-nginx-controller.txt 2026-06-10 11:59:09.174603 | controller | >f+++++++++ items-describe/ingress-nginx--jobs--ingress-nginx-admission-create.txt 2026-06-10 11:59:09.174639 | controller | >f+++++++++ items-describe/ingress-nginx--jobs--ingress-nginx-admission-patch.txt 2026-06-10 11:59:09.174675 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-admission-create-58vqp.txt 2026-06-10 11:59:09.174711 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-admission-patch-zkxtk.txt 2026-06-10 11:59:09.174748 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-controller-86649c845b-c9s5l.txt 2026-06-10 11:59:09.174776 | controller | >f+++++++++ items-describe/ingress-nginx--secrets--ingress-nginx-admission.txt 2026-06-10 11:59:09.174800 | controller | >f+++++++++ items-describe/ingress-nginx--services--ingress-nginx-controller-admission.txt 2026-06-10 11:59:09.174823 | controller | >f+++++++++ items-describe/ingress-nginx--services--ingress-nginx-controller.txt 2026-06-10 11:59:09.174847 | controller | >f+++++++++ items-describe/kube-node-lease--configmaps--kube-root-ca.crt.txt 2026-06-10 11:59:09.174870 | controller | >f+++++++++ items-describe/kube-public--configmaps--cluster-info.txt 2026-06-10 11:59:09.174893 | controller | >f+++++++++ items-describe/kube-public--configmaps--kube-root-ca.crt.txt 2026-06-10 11:59:09.174916 | controller | >f+++++++++ items-describe/kube-system--configmaps--coredns.txt 2026-06-10 11:59:09.174943 | controller | >f+++++++++ items-describe/kube-system--configmaps--extension-apiserver-authentication.txt 2026-06-10 11:59:09.174977 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-apiserver-legacy-service-account-token-tracking.txt 2026-06-10 11:59:09.175014 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-proxy.txt 2026-06-10 11:59:09.175048 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-root-ca.crt.txt 2026-06-10 11:59:09.175083 | controller | >f+++++++++ items-describe/kube-system--configmaps--kubeadm-config.txt 2026-06-10 11:59:09.175116 | controller | >f+++++++++ items-describe/kube-system--configmaps--kubelet-config.txt 2026-06-10 11:59:09.175149 | controller | >f+++++++++ items-describe/kube-system--deployments--coredns.txt 2026-06-10 11:59:09.175183 | controller | >f+++++++++ items-describe/kube-system--pods--coredns-668d6bf9bc-4rn2t.txt 2026-06-10 11:59:09.175209 | controller | >f+++++++++ items-describe/kube-system--pods--etcd-minikube.txt 2026-06-10 11:59:09.175232 | controller | >f+++++++++ items-describe/kube-system--pods--kindnet-ngpf2.txt 2026-06-10 11:59:09.175262 | controller | >f+++++++++ items-describe/kube-system--pods--kube-apiserver-minikube.txt 2026-06-10 11:59:09.175298 | controller | >f+++++++++ items-describe/kube-system--pods--kube-controller-manager-minikube.txt 2026-06-10 11:59:09.175333 | controller | >f+++++++++ items-describe/kube-system--pods--kube-ingress-dns-minikube.txt 2026-06-10 11:59:09.175407 | controller | >f+++++++++ items-describe/kube-system--pods--kube-proxy-952fl.txt 2026-06-10 11:59:09.175434 | controller | >f+++++++++ items-describe/kube-system--pods--kube-scheduler-minikube.txt 2026-06-10 11:59:09.175457 | controller | >f+++++++++ items-describe/kube-system--pods--storage-provisioner.txt 2026-06-10 11:59:09.175481 | controller | >f+++++++++ items-describe/kube-system--secrets--bootstrap-token-bgy9nz.txt 2026-06-10 11:59:09.175504 | controller | >f+++++++++ items-describe/kube-system--services--kube-dns.txt 2026-06-10 11:59:09.175528 | controller | >f+++++++++ items-describe/nodepool--configmaps--kube-root-ca.crt.txt 2026-06-10 11:59:09.175552 | controller | >f+++++++++ items-describe/nodepool--secrets--nodepool-token.txt 2026-06-10 11:59:09.175581 | controller | >f+++++++++ items-describe/sf--configmaps--gateway-config-map.txt 2026-06-10 11:59:09.175618 | controller | >f+++++++++ items-describe/sf--configmaps--git-server-pi-config-map.txt 2026-06-10 11:59:09.175653 | controller | >f+++++++++ items-describe/sf--configmaps--kube-root-ca.crt.txt 2026-06-10 11:59:09.175688 | controller | >f+++++++++ items-describe/sf--configmaps--logserver-config-map.txt 2026-06-10 11:59:09.175713 | controller | >f+++++++++ items-describe/sf--configmaps--loki-config.txt 2026-06-10 11:59:09.175737 | controller | >f+++++++++ items-describe/sf--configmaps--managesf-resources-config-map.txt 2026-06-10 11:59:09.175760 | controller | >f+++++++++ items-describe/sf--configmaps--managesf-resources-tooling-config-map.txt 2026-06-10 11:59:09.175783 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-builder-extra-config-config-map.txt 2026-06-10 11:59:09.175806 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-launcher-extra-config-config-map.txt 2026-06-10 11:59:09.175828 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-tooling-config-map.txt 2026-06-10 11:59:09.175854 | controller | >f+++++++++ items-describe/sf--configmaps--np-statsd-config-map.txt 2026-06-10 11:59:09.175888 | controller | >f+++++++++ items-describe/sf--configmaps--pymod-sf-operator-config-map.txt 2026-06-10 11:59:09.175923 | controller | >f+++++++++ items-describe/sf--configmaps--sf-standalone-owner.txt 2026-06-10 11:59:09.175952 | controller | >f+++++++++ items-describe/sf--configmaps--test-fluentbit-config.txt 2026-06-10 11:59:09.175976 | controller | >f+++++++++ items-describe/sf--configmaps--zookeeper-pi-config-map.txt 2026-06-10 11:59:09.175999 | controller | >f+++++++++ items-describe/sf--configmaps--zs-internal-tenant-reconfigure.txt 2026-06-10 11:59:09.176022 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-extra-config-map.txt 2026-06-10 11:59:09.176044 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-logging-config-map.txt 2026-06-10 11:59:09.176067 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-scheduler-tooling-config-map.txt 2026-06-10 11:59:09.176089 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-statsd-config-map.txt 2026-06-10 11:59:09.176117 | controller | >f+++++++++ items-describe/sf--deployments--gateway.txt 2026-06-10 11:59:09.176153 | controller | >f+++++++++ items-describe/sf--deployments--logjuicer.txt 2026-06-10 11:59:09.176187 | controller | >f+++++++++ items-describe/sf--deployments--nodepool-launcher.txt 2026-06-10 11:59:09.176222 | controller | >f+++++++++ items-describe/sf--deployments--test-loki.txt 2026-06-10 11:59:09.176256 | controller | >f+++++++++ items-describe/sf--deployments--zuul-web.txt 2026-06-10 11:59:09.176288 | controller | >f+++++++++ items-describe/sf--deployments--zuul-weeder.txt 2026-06-10 11:59:09.176319 | controller | >f+++++++++ items-describe/sf--ingress--gerrit-ingress.txt 2026-06-10 11:59:09.176383 | controller | >f+++++++++ items-describe/sf--ingress--sf-ingress.txt 2026-06-10 11:59:09.176427 | controller | >f+++++++++ items-describe/sf--jobs--config-base-secret-1-9189-ns.txt 2026-06-10 11:59:09.176461 | controller | >f+++++++++ items-describe/sf--jobs--post-init.txt 2026-06-10 11:59:09.176500 | controller | >f+++++++++ items-describe/sf--jobs--zuul-db-provision.txt 2026-06-10 11:59:09.176532 | controller | >f+++++++++ items-describe/sf--pods--config-base-secret-1-917-ns-rz8lm.txt 2026-06-10 11:59:09.176566 | controller | >f+++++++++ items-describe/sf--pods--config-base-secret-1-9189-ns-dlqqt.txt 2026-06-10 11:59:09.176599 | controller | >f+++++++++ items-describe/sf--pods--gateway-6bbbd758d9-fngqp.txt 2026-06-10 11:59:09.176633 | controller | >f+++++++++ items-describe/sf--pods--gerrit-0.txt 2026-06-10 11:59:09.176666 | controller | >f+++++++++ items-describe/sf--pods--git-server-0.txt 2026-06-10 11:59:09.176697 | controller | >f+++++++++ items-describe/sf--pods--hound-search-0.txt 2026-06-10 11:59:09.176730 | controller | >f+++++++++ items-describe/sf--pods--logjuicer-6ff84764b7-pfvlf.txt 2026-06-10 11:59:09.176761 | controller | >f+++++++++ items-describe/sf--pods--logserver-0.txt 2026-06-10 11:59:09.176794 | controller | >f+++++++++ items-describe/sf--pods--mariadb-0.txt 2026-06-10 11:59:09.176828 | controller | >f+++++++++ items-describe/sf--pods--nodepool-builder-0.txt 2026-06-10 11:59:09.176864 | controller | >f+++++++++ items-describe/sf--pods--nodepool-launcher-58bc794b9d-l9cn4.txt 2026-06-10 11:59:09.176899 | controller | >f+++++++++ items-describe/sf--pods--post-init-97qhg.txt 2026-06-10 11:59:09.176935 | controller | >f+++++++++ items-describe/sf--pods--test-fluentbit.txt 2026-06-10 11:59:09.176970 | controller | >f+++++++++ items-describe/sf--pods--test-loki-8cd4fd56-2v44p.txt 2026-06-10 11:59:09.177003 | controller | >f+++++++++ items-describe/sf--pods--zookeeper-0.txt 2026-06-10 11:59:09.177037 | controller | >f+++++++++ items-describe/sf--pods--zookeeper-1.txt 2026-06-10 11:59:09.177069 | controller | >f+++++++++ items-describe/sf--pods--zookeeper-2.txt 2026-06-10 11:59:09.177102 | controller | >f+++++++++ items-describe/sf--pods--zuul-db-provision-sj6px.txt 2026-06-10 11:59:09.177136 | controller | >f+++++++++ items-describe/sf--pods--zuul-db-provision-zlqvm.txt 2026-06-10 11:59:09.177169 | controller | >f+++++++++ items-describe/sf--pods--zuul-executor-0.txt 2026-06-10 11:59:09.177203 | controller | >f+++++++++ items-describe/sf--pods--zuul-merger-0.txt 2026-06-10 11:59:09.177236 | controller | >f+++++++++ items-describe/sf--pods--zuul-scheduler-0.txt 2026-06-10 11:59:09.177273 | controller | >f+++++++++ items-describe/sf--pods--zuul-web-5c5c9bf66b-ngltn.txt 2026-06-10 11:59:09.177306 | controller | >f+++++++++ items-describe/sf--pods--zuul-weeder-57c6567f5f-wfrfs.txt 2026-06-10 11:59:09.177338 | controller | >f+++++++++ items-describe/sf--pvc--gerrit-gerrit-0.txt 2026-06-10 11:59:09.177414 | controller | >f+++++++++ items-describe/sf--pvc--git-server-git-server-0.txt 2026-06-10 11:59:09.177450 | controller | >f+++++++++ items-describe/sf--pvc--hound-search-data-hound-search-0.txt 2026-06-10 11:59:09.177479 | controller | >f+++++++++ items-describe/sf--pvc--logjuicer-pvc.txt 2026-06-10 11:59:09.177507 | controller | >f+++++++++ items-describe/sf--pvc--logserver-logserver-0.txt 2026-06-10 11:59:09.177535 | controller | >f+++++++++ items-describe/sf--pvc--mariadb-logs-mariadb-0.txt 2026-06-10 11:59:09.177563 | controller | >f+++++++++ items-describe/sf--pvc--mariadb-mariadb-0.txt 2026-06-10 11:59:09.177596 | controller | >f+++++++++ items-describe/sf--pvc--nodepool-builder-nodepool-builder-0.txt 2026-06-10 11:59:09.177629 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-data-zookeeper-0.txt 2026-06-10 11:59:09.177661 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-data-zookeeper-1.txt 2026-06-10 11:59:09.177689 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-data-zookeeper-2.txt 2026-06-10 11:59:09.177717 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-logs-zookeeper-0.txt 2026-06-10 11:59:09.177744 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-logs-zookeeper-1.txt 2026-06-10 11:59:09.177772 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-logs-zookeeper-2.txt 2026-06-10 11:59:09.177812 | controller | >f+++++++++ items-describe/sf--pvc--zuul-executor-zuul-executor-0.txt 2026-06-10 11:59:09.177845 | controller | >f+++++++++ items-describe/sf--pvc--zuul-merger-zuul-merger-0.txt 2026-06-10 11:59:09.177873 | controller | >f+++++++++ items-describe/sf--pvc--zuul-scheduler-zuul-scheduler-0.txt 2026-06-10 11:59:09.177901 | controller | >f+++++++++ items-describe/sf--secrets--admin-ssh-key.txt 2026-06-10 11:59:09.177928 | controller | >f+++++++++ items-describe/sf--secrets--config-update-secrets.txt 2026-06-10 11:59:09.177956 | controller | >f+++++++++ items-describe/sf--secrets--dummygerritsecret.txt 2026-06-10 11:59:09.177983 | controller | >f+++++++++ items-describe/sf--secrets--es-basicauth.txt 2026-06-10 11:59:09.178012 | controller | >f+++++++++ items-describe/sf--secrets--gerrit-admin-api-key.txt 2026-06-10 11:59:09.178046 | controller | >f+++++++++ items-describe/sf--secrets--logserver-keys.txt 2026-06-10 11:59:09.178081 | controller | >f+++++++++ items-describe/sf--secrets--logserver-uploader-keys.txt 2026-06-10 11:59:09.178112 | controller | >f+++++++++ items-describe/sf--secrets--logserver-uploader-spare-keys.txt 2026-06-10 11:59:09.178136 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-config-secrets.txt 2026-06-10 11:59:09.178158 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-initdb-secrets.txt 2026-06-10 11:59:09.178181 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-root-password.txt 2026-06-10 11:59:09.178203 | controller | >f+++++++++ items-describe/sf--secrets--nodepool-builder-ssh-key.txt 2026-06-10 11:59:09.178225 | controller | >f+++++++++ items-describe/sf--secrets--nodepool-providers-secrets.txt 2026-06-10 11:59:09.178248 | controller | >f+++++++++ items-describe/sf--secrets--self-signed-cert.txt 2026-06-10 11:59:09.178271 | controller | >f+++++++++ items-describe/sf--secrets--smtp-secret.txt 2026-06-10 11:59:09.178303 | controller | >f+++++++++ items-describe/sf--secrets--zookeeper-client-tls.txt 2026-06-10 11:59:09.178337 | controller | >f+++++++++ items-describe/sf--secrets--zookeeper-server-tls.txt 2026-06-10 11:59:09.178405 | controller | >f+++++++++ items-describe/sf--secrets--zuul-auth-secret.txt 2026-06-10 11:59:09.178441 | controller | >f+++++++++ items-describe/sf--secrets--zuul-config.txt 2026-06-10 11:59:09.178494 | controller | >f+++++++++ items-describe/sf--secrets--zuul-db-connection.txt 2026-06-10 11:59:09.178531 | controller | >f+++++++++ items-describe/sf--secrets--zuul-gerrit-api-key.txt 2026-06-10 11:59:09.178566 | controller | >f+++++++++ items-describe/sf--secrets--zuul-keystore-password.txt 2026-06-10 11:59:09.178601 | controller | >f+++++++++ items-describe/sf--secrets--zuul-spare-ssh-key.txt 2026-06-10 11:59:09.178634 | controller | >f+++++++++ items-describe/sf--secrets--zuul-ssh-key.txt 2026-06-10 11:59:09.178667 | controller | >f+++++++++ items-describe/sf--services--fluent-bit-forward-input.txt 2026-06-10 11:59:09.178699 | controller | >f+++++++++ items-describe/sf--services--gateway.txt 2026-06-10 11:59:09.178732 | controller | >f+++++++++ items-describe/sf--services--gerrit-httpd.txt 2026-06-10 11:59:09.178766 | controller | >f+++++++++ items-describe/sf--services--gerrit-sshd.txt 2026-06-10 11:59:09.178801 | controller | >f+++++++++ items-describe/sf--services--git-server-rw.txt 2026-06-10 11:59:09.178836 | controller | >f+++++++++ items-describe/sf--services--git-server.txt 2026-06-10 11:59:09.178868 | controller | >f+++++++++ items-describe/sf--services--hound-search.txt 2026-06-10 11:59:09.178902 | controller | >f+++++++++ items-describe/sf--services--logjuicer.txt 2026-06-10 11:59:09.178935 | controller | >f+++++++++ items-describe/sf--services--logserver.txt 2026-06-10 11:59:09.178969 | controller | >f+++++++++ items-describe/sf--services--loki-http.txt 2026-06-10 11:59:09.179003 | controller | >f+++++++++ items-describe/sf--services--mariadb.txt 2026-06-10 11:59:09.179038 | controller | >f+++++++++ items-describe/sf--services--nodepool-builder.txt 2026-06-10 11:59:09.179087 | controller | >f+++++++++ items-describe/sf--services--nodepool-launcher.txt 2026-06-10 11:59:09.179124 | controller | >f+++++++++ items-describe/sf--services--zookeeper-headless.txt 2026-06-10 11:59:09.179160 | controller | >f+++++++++ items-describe/sf--services--zookeeper.txt 2026-06-10 11:59:09.179195 | controller | >f+++++++++ items-describe/sf--services--zuul-capacity.txt 2026-06-10 11:59:09.179220 | controller | >f+++++++++ items-describe/sf--services--zuul-executor-headless.txt 2026-06-10 11:59:09.179243 | controller | >f+++++++++ items-describe/sf--services--zuul-web.txt 2026-06-10 11:59:09.179266 | controller | >f+++++++++ items-describe/sf--services--zuul-weeder.txt 2026-06-10 11:59:09.179288 | controller | >f+++++++++ items-describe/sf--statefulsets--gerrit.txt 2026-06-10 11:59:09.179310 | controller | >f+++++++++ items-describe/sf--statefulsets--git-server.txt 2026-06-10 11:59:09.179333 | controller | >f+++++++++ items-describe/sf--statefulsets--hound-search.txt 2026-06-10 11:59:09.179398 | controller | >f+++++++++ items-describe/sf--statefulsets--logserver.txt 2026-06-10 11:59:09.179436 | controller | >f+++++++++ items-describe/sf--statefulsets--mariadb.txt 2026-06-10 11:59:09.179472 | controller | >f+++++++++ items-describe/sf--statefulsets--nodepool-builder.txt 2026-06-10 11:59:09.179509 | controller | >f+++++++++ items-describe/sf--statefulsets--zookeeper.txt 2026-06-10 11:59:09.179545 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-executor.txt 2026-06-10 11:59:09.179580 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-merger.txt 2026-06-10 11:59:09.179615 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-scheduler.txt 2026-06-10 11:59:09.179650 | controller | cd+++++++++ k8s-resources/ 2026-06-10 11:59:09.179683 | controller | cd+++++++++ k8s-resources/sf/ 2026-06-10 11:59:09.179714 | controller | cd+++++++++ k8s-resources/sf/pod/ 2026-06-10 11:59:09.179747 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-executor-0-zuul-executor-logs0.txt 2026-06-10 11:59:09.179779 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-merger-0-zuul-merger-logs0.txt 2026-06-10 11:59:09.179814 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-scheduler-0-zuul-scheduler-logs0.txt 2026-06-10 11:59:09.179940 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-web-0-zuul-web-logs0.txt 2026-06-10 11:59:09.180036 | controller | cd+++++++++ logserver-content/ 2026-06-10 11:59:09.180085 | controller | cd+++++++++ logserver-content/442/ 2026-06-10 11:59:09.180119 | controller | cd+++++++++ logserver-content/442/internal/ 2026-06-10 11:59:09.180237 | controller | cd+++++++++ logserver-content/442/internal/44277dc0016e4398954df7968548f1de/ 2026-06-10 11:59:09.180277 | controller | >f+++++++++ logserver-content/442/internal/44277dc0016e4398954df7968548f1de/job-output.json.gz 2026-06-10 11:59:09.180314 | controller | >f+++++++++ logserver-content/442/internal/44277dc0016e4398954df7968548f1de/job-output.txt.gz 2026-06-10 11:59:09.180375 | controller | >f+++++++++ logserver-content/442/internal/44277dc0016e4398954df7968548f1de/workspace-repos.json 2026-06-10 11:59:09.180423 | controller | >f+++++++++ logserver-content/442/internal/44277dc0016e4398954df7968548f1de/zuul-manifest.json 2026-06-10 11:59:09.180460 | controller | cd+++++++++ logserver-content/442/internal/44277dc0016e4398954df7968548f1de/zuul-info/ 2026-06-10 11:59:09.180494 | controller | >f+++++++++ logserver-content/442/internal/44277dc0016e4398954df7968548f1de/zuul-info/inventory.yaml 2026-06-10 11:59:09.180528 | controller | cd+++++++++ logserver-content/96a/ 2026-06-10 11:59:09.180564 | controller | cd+++++++++ logserver-content/96a/internal/ 2026-06-10 11:59:09.180600 | controller | cd+++++++++ logserver-content/96a/internal/96a1f117da9743318cdaa083f4bb2d5e/ 2026-06-10 11:59:09.180635 | controller | >f+++++++++ logserver-content/96a/internal/96a1f117da9743318cdaa083f4bb2d5e/job-output.json.gz 2026-06-10 11:59:09.180669 | controller | >f+++++++++ logserver-content/96a/internal/96a1f117da9743318cdaa083f4bb2d5e/job-output.txt.gz 2026-06-10 11:59:09.180711 | controller | >f+++++++++ logserver-content/96a/internal/96a1f117da9743318cdaa083f4bb2d5e/workspace-repos.json 2026-06-10 11:59:09.180746 | controller | >f+++++++++ logserver-content/96a/internal/96a1f117da9743318cdaa083f4bb2d5e/zuul-manifest.json 2026-06-10 11:59:09.180780 | controller | cd+++++++++ logserver-content/96a/internal/96a1f117da9743318cdaa083f4bb2d5e/zuul-info/ 2026-06-10 11:59:09.180814 | controller | >f+++++++++ logserver-content/96a/internal/96a1f117da9743318cdaa083f4bb2d5e/zuul-info/inventory.yaml 2026-06-10 11:59:09.180849 | controller | cd+++++++++ logserver-content/b68/ 2026-06-10 11:59:09.180882 | controller | cd+++++++++ logserver-content/b68/internal/ 2026-06-10 11:59:09.180916 | controller | cd+++++++++ logserver-content/b68/internal/b68f1b7951e144d29094ae44d06e0691/ 2026-06-10 11:59:09.180954 | controller | >f+++++++++ logserver-content/b68/internal/b68f1b7951e144d29094ae44d06e0691/job-output.json.gz 2026-06-10 11:59:09.180989 | controller | >f+++++++++ logserver-content/b68/internal/b68f1b7951e144d29094ae44d06e0691/job-output.txt.gz 2026-06-10 11:59:09.181023 | controller | >f+++++++++ logserver-content/b68/internal/b68f1b7951e144d29094ae44d06e0691/workspace-repos.json 2026-06-10 11:59:09.181057 | controller | >f+++++++++ logserver-content/b68/internal/b68f1b7951e144d29094ae44d06e0691/zuul-manifest.json 2026-06-10 11:59:09.181092 | controller | cd+++++++++ logserver-content/b68/internal/b68f1b7951e144d29094ae44d06e0691/zuul-info/ 2026-06-10 11:59:09.181126 | controller | >f+++++++++ logserver-content/b68/internal/b68f1b7951e144d29094ae44d06e0691/zuul-info/inventory.yaml 2026-06-10 11:59:09.181159 | controller | cd+++++++++ loki-logs/ 2026-06-10 11:59:09.181194 | controller | >f+++++++++ loki-logs/all.log 2026-06-10 11:59:09.181258 | controller | cd+++++++++ pods-logs/ 2026-06-10 11:59:09.181298 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-admission-create-58vqp--create.txt 2026-06-10 11:59:09.181334 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-admission-patch-zkxtk--patch.txt 2026-06-10 11:59:09.181411 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-controller-86649c845b-c9s5l--controller.txt 2026-06-10 11:59:09.181450 | controller | >f+++++++++ pods-logs/kube-system--coredns-668d6bf9bc-4rn2t--coredns.txt 2026-06-10 11:59:09.181485 | controller | >f+++++++++ pods-logs/kube-system--etcd-minikube--etcd.txt 2026-06-10 11:59:09.181519 | controller | >f+++++++++ pods-logs/kube-system--kindnet-ngpf2--kindnet-cni.txt 2026-06-10 11:59:09.181554 | controller | >f+++++++++ pods-logs/kube-system--kube-apiserver-minikube--kube-apiserver.txt 2026-06-10 11:59:09.181591 | controller | >f+++++++++ pods-logs/kube-system--kube-controller-manager-minikube--kube-controller-manager.txt 2026-06-10 11:59:09.181626 | controller | >f+++++++++ pods-logs/kube-system--kube-ingress-dns-minikube--minikube-ingress-dns.txt 2026-06-10 11:59:09.181660 | controller | >f+++++++++ pods-logs/kube-system--kube-proxy-952fl--kube-proxy.txt 2026-06-10 11:59:09.181694 | controller | >f+++++++++ pods-logs/kube-system--kube-scheduler-minikube--kube-scheduler.txt 2026-06-10 11:59:09.181726 | controller | >f+++++++++ pods-logs/kube-system--storage-provisioner--storage-provisioner.txt 2026-06-10 11:59:09.181758 | controller | >f+++++++++ pods-logs/sf--config-base-secret-1-917-ns-rz8lm--sf-operator.txt 2026-06-10 11:59:09.181807 | controller | >f+++++++++ pods-logs/sf--config-base-secret-1-9189-ns-dlqqt--sf-operator.txt 2026-06-10 11:59:09.181853 | controller | >f+++++++++ pods-logs/sf--gateway-6bbbd758d9-fngqp--gateway.txt 2026-06-10 11:59:09.181898 | controller | >f+++++++++ pods-logs/sf--gerrit-0--gerrit.txt 2026-06-10 11:59:09.181942 | controller | >f+++++++++ pods-logs/sf--gerrit-0--managesf-resources.txt 2026-06-10 11:59:09.181987 | controller | >f+++++++++ pods-logs/sf--git-server-0--git-server-rw.txt 2026-06-10 11:59:09.182033 | controller | >f+++++++++ pods-logs/sf--git-server-0--git-server.txt 2026-06-10 11:59:09.182083 | controller | >f+++++++++ pods-logs/sf--hound-search-0--hound-search.txt 2026-06-10 11:59:09.182126 | controller | >f+++++++++ pods-logs/sf--logjuicer-6ff84764b7-pfvlf--logjuicer.txt 2026-06-10 11:59:09.182170 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver-nodeexporter.txt 2026-06-10 11:59:09.182214 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver-sshd.txt 2026-06-10 11:59:09.182259 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver.txt 2026-06-10 11:59:09.182302 | controller | >f+++++++++ pods-logs/sf--logserver-0--purgelogs.txt 2026-06-10 11:59:09.182345 | controller | >f+++++++++ pods-logs/sf--mariadb-0--mariadb-nodeexporter.txt 2026-06-10 11:59:09.182438 | controller | >f+++++++++ pods-logs/sf--mariadb-0--mariadb.txt 2026-06-10 11:59:09.182564 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--build-logs-httpd.txt 2026-06-10 11:59:09.182617 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--nodepool-builder-nodeexporter.txt 2026-06-10 11:59:09.182662 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--nodepool-builder.txt 2026-06-10 11:59:09.182703 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--np-statsd.txt 2026-06-10 11:59:09.182743 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-58bc794b9d-l9cn4--launcher.txt 2026-06-10 11:59:09.182786 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-58bc794b9d-l9cn4--np-statsd.txt 2026-06-10 11:59:09.182822 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-58bc794b9d-l9cn4--zuul-capacity.txt 2026-06-10 11:59:09.182865 | controller | >f+++++++++ pods-logs/sf--post-init-97qhg--post-init-container.txt 2026-06-10 11:59:09.182909 | controller | >f+++++++++ pods-logs/sf--test-fluentbit--test-fluentbit.txt 2026-06-10 11:59:09.182947 | controller | >f+++++++++ pods-logs/sf--test-loki-8cd4fd56-2v44p--loki.txt 2026-06-10 11:59:09.182977 | controller | >f+++++++++ pods-logs/sf--zookeeper-0--zookeeper-nodeexporter.txt 2026-06-10 11:59:09.183007 | controller | >f+++++++++ pods-logs/sf--zookeeper-0--zookeeper.txt 2026-06-10 11:59:09.183036 | controller | >f+++++++++ pods-logs/sf--zookeeper-1--zookeeper-nodeexporter.txt 2026-06-10 11:59:09.183077 | controller | >f+++++++++ pods-logs/sf--zookeeper-1--zookeeper.txt 2026-06-10 11:59:09.183120 | controller | >f+++++++++ pods-logs/sf--zookeeper-2--zookeeper-nodeexporter.txt 2026-06-10 11:59:09.183164 | controller | >f+++++++++ pods-logs/sf--zookeeper-2--zookeeper.txt 2026-06-10 11:59:09.183208 | controller | >f+++++++++ pods-logs/sf--zuul-db-provision-sj6px--mariadb-client.txt 2026-06-10 11:59:09.183279 | controller | >f+++++++++ pods-logs/sf--zuul-db-provision-zlqvm--mariadb-client.txt 2026-06-10 11:59:09.183329 | controller | >f+++++++++ pods-logs/sf--zuul-executor-0--zuul-executor-nodeexporter.txt 2026-06-10 11:59:09.183402 | controller | >f+++++++++ pods-logs/sf--zuul-executor-0--zuul-executor.txt 2026-06-10 11:59:09.183435 | controller | >f+++++++++ pods-logs/sf--zuul-merger-0--zuul-merger-nodeexporter.txt 2026-06-10 11:59:09.183465 | controller | >f+++++++++ pods-logs/sf--zuul-merger-0--zuul-merger.txt 2026-06-10 11:59:09.183493 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-scheduler-nodeexporter.txt 2026-06-10 11:59:09.183522 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-scheduler.txt 2026-06-10 11:59:09.183551 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-statsd.txt 2026-06-10 11:59:09.183587 | controller | >f+++++++++ pods-logs/sf--zuul-web-5c5c9bf66b-ngltn--zuul-web.txt 2026-06-10 11:59:09.183630 | controller | >f+++++++++ pods-logs/sf--zuul-weeder-57c6567f5f-wfrfs--zuul-weeder.txt 2026-06-10 11:59:09.183758 | controller | cd+++++++++ repos/ 2026-06-10 11:59:09.183806 | controller | >f+++++++++ repos/test-ssh-key 2026-06-10 11:59:09.183851 | controller | >f+++++++++ repos/test-ssh-key.pub 2026-06-10 11:59:09.183889 | controller | >f+++++++++ repos/zuul-ssh-key 2026-06-10 11:59:09.183914 | controller | cd+++++++++ repos/config/ 2026-06-10 11:59:09.183943 | controller | >f+++++++++ repos/config/.gitreview 2026-06-10 11:59:09.183973 | controller | cd+++++++++ repos/config/.git/ 2026-06-10 11:59:09.184002 | controller | >f+++++++++ repos/config/.git/COMMIT_EDITMSG 2026-06-10 11:59:09.184030 | controller | >f+++++++++ repos/config/.git/FETCH_HEAD 2026-06-10 11:59:09.184058 | controller | >f+++++++++ repos/config/.git/HEAD 2026-06-10 11:59:09.184086 | controller | >f+++++++++ repos/config/.git/ORIG_HEAD 2026-06-10 11:59:09.184114 | controller | >f+++++++++ repos/config/.git/config 2026-06-10 11:59:09.184142 | controller | >f+++++++++ repos/config/.git/description 2026-06-10 11:59:09.184171 | controller | >f+++++++++ repos/config/.git/index 2026-06-10 11:59:09.184199 | controller | >f+++++++++ repos/config/.git/packed-refs 2026-06-10 11:59:09.184227 | controller | cd+++++++++ repos/config/.git/hooks/ 2026-06-10 11:59:09.184256 | controller | >f+++++++++ repos/config/.git/hooks/applypatch-msg.sample 2026-06-10 11:59:09.184287 | controller | >f+++++++++ repos/config/.git/hooks/commit-msg 2026-06-10 11:59:09.184316 | controller | >f+++++++++ repos/config/.git/hooks/commit-msg.sample 2026-06-10 11:59:09.184343 | controller | >f+++++++++ repos/config/.git/hooks/fsmonitor-watchman.sample 2026-06-10 11:59:09.184399 | controller | >f+++++++++ repos/config/.git/hooks/post-update.sample 2026-06-10 11:59:09.184429 | controller | >f+++++++++ repos/config/.git/hooks/pre-applypatch.sample 2026-06-10 11:59:09.184458 | controller | >f+++++++++ repos/config/.git/hooks/pre-commit.sample 2026-06-10 11:59:09.184486 | controller | >f+++++++++ repos/config/.git/hooks/pre-merge-commit.sample 2026-06-10 11:59:09.184515 | controller | >f+++++++++ repos/config/.git/hooks/pre-push.sample 2026-06-10 11:59:09.184543 | controller | >f+++++++++ repos/config/.git/hooks/pre-rebase.sample 2026-06-10 11:59:09.184576 | controller | >f+++++++++ repos/config/.git/hooks/pre-receive.sample 2026-06-10 11:59:09.184619 | controller | >f+++++++++ repos/config/.git/hooks/prepare-commit-msg.sample 2026-06-10 11:59:09.184662 | controller | >f+++++++++ repos/config/.git/hooks/push-to-checkout.sample 2026-06-10 11:59:09.184706 | controller | >f+++++++++ repos/config/.git/hooks/sendemail-validate.sample 2026-06-10 11:59:09.184747 | controller | >f+++++++++ repos/config/.git/hooks/update.sample 2026-06-10 11:59:09.184787 | controller | cd+++++++++ repos/config/.git/info/ 2026-06-10 11:59:09.184829 | controller | >f+++++++++ repos/config/.git/info/exclude 2026-06-10 11:59:09.184871 | controller | cd+++++++++ repos/config/.git/logs/ 2026-06-10 11:59:09.184914 | controller | >f+++++++++ repos/config/.git/logs/HEAD 2026-06-10 11:59:09.184957 | controller | cd+++++++++ repos/config/.git/logs/refs/ 2026-06-10 11:59:09.185000 | controller | cd+++++++++ repos/config/.git/logs/refs/heads/ 2026-06-10 11:59:09.185033 | controller | >f+++++++++ repos/config/.git/logs/refs/heads/master 2026-06-10 11:59:09.185063 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/ 2026-06-10 11:59:09.185094 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/gerrit/ 2026-06-10 11:59:09.185134 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/gerrit/HEAD 2026-06-10 11:59:09.185178 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/gerrit/master 2026-06-10 11:59:09.185222 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/origin/ 2026-06-10 11:59:09.185266 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/origin/HEAD 2026-06-10 11:59:09.185308 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/origin/master 2026-06-10 11:59:09.185380 | controller | cd+++++++++ repos/config/.git/objects/ 2026-06-10 11:59:09.185579 | controller | cd+++++++++ repos/config/.git/objects/06/ 2026-06-10 11:59:09.185636 | controller | >f+++++++++ repos/config/.git/objects/06/5460e3e4fba2934d333152681c02f5bafe09c9 2026-06-10 11:59:09.185673 | controller | cd+++++++++ repos/config/.git/objects/12/ 2026-06-10 11:59:09.185708 | controller | >f+++++++++ repos/config/.git/objects/12/fa3ad1c2d24d0302fb6e47d5bd2e779f1ac3b0 2026-06-10 11:59:09.185750 | controller | cd+++++++++ repos/config/.git/objects/2d/ 2026-06-10 11:59:09.185785 | controller | >f+++++++++ repos/config/.git/objects/2d/85b4df99ffe2a683f4c691e7c6e224b7a2eb28 2026-06-10 11:59:09.185822 | controller | >f+++++++++ repos/config/.git/objects/2d/9f8c30b41d150bebdca388ea05848650a1ce4e 2026-06-10 11:59:09.185856 | controller | cd+++++++++ repos/config/.git/objects/34/ 2026-06-10 11:59:09.185892 | controller | >f+++++++++ repos/config/.git/objects/34/d1bce832cc1df8d44b98d95ea747d427f139cf 2026-06-10 11:59:09.185926 | controller | cd+++++++++ repos/config/.git/objects/46/ 2026-06-10 11:59:09.185959 | controller | >f+++++++++ repos/config/.git/objects/46/ac42172645ad18ac5877befbb59519538ac4e5 2026-06-10 11:59:09.185994 | controller | cd+++++++++ repos/config/.git/objects/4e/ 2026-06-10 11:59:09.186028 | controller | >f+++++++++ repos/config/.git/objects/4e/834974b0ffc484c06ad2e51fbbce69f65508e9 2026-06-10 11:59:09.186061 | controller | cd+++++++++ repos/config/.git/objects/58/ 2026-06-10 11:59:09.186093 | controller | >f+++++++++ repos/config/.git/objects/58/56df933aae228bf3d3e5c85781a29f36681f4c 2026-06-10 11:59:09.186126 | controller | cd+++++++++ repos/config/.git/objects/5d/ 2026-06-10 11:59:09.186157 | controller | >f+++++++++ repos/config/.git/objects/5d/63f319cde15093cd91f0478efeac0689ba2150 2026-06-10 11:59:09.186188 | controller | cd+++++++++ repos/config/.git/objects/6a/ 2026-06-10 11:59:09.186219 | controller | >f+++++++++ repos/config/.git/objects/6a/906645df0c95e3c0f28d78f55392f0c0ef8c10 2026-06-10 11:59:09.186254 | controller | cd+++++++++ repos/config/.git/objects/6c/ 2026-06-10 11:59:09.186286 | controller | >f+++++++++ repos/config/.git/objects/6c/3d79c5c4e6d881408460a3e77faeb1d0c4ab08 2026-06-10 11:59:09.186320 | controller | cd+++++++++ repos/config/.git/objects/6f/ 2026-06-10 11:59:09.186383 | controller | >f+++++++++ repos/config/.git/objects/6f/7878e39e14e63cce1c9959f1638354c421eddc 2026-06-10 11:59:09.186430 | controller | cd+++++++++ repos/config/.git/objects/75/ 2026-06-10 11:59:09.186468 | controller | >f+++++++++ repos/config/.git/objects/75/026a20fdbeb3f69d8d51c8357b94cab9b33ae7 2026-06-10 11:59:09.186523 | controller | cd+++++++++ repos/config/.git/objects/76/ 2026-06-10 11:59:09.186561 | controller | >f+++++++++ repos/config/.git/objects/76/2eac1635c75f4518122986735a6a67fc2ff2e9 2026-06-10 11:59:09.186597 | controller | >f+++++++++ repos/config/.git/objects/76/4761191d8eeab58e50efae3f9d507d2d2294d3 2026-06-10 11:59:09.186632 | controller | cd+++++++++ repos/config/.git/objects/7b/ 2026-06-10 11:59:09.186667 | controller | >f+++++++++ repos/config/.git/objects/7b/502ed9681fab04fd5f79aa6a7528d52c69527c 2026-06-10 11:59:09.186702 | controller | cd+++++++++ repos/config/.git/objects/7d/ 2026-06-10 11:59:09.186738 | controller | >f+++++++++ repos/config/.git/objects/7d/75b95c582dba13e9a0a71267b5289898e8408e 2026-06-10 11:59:09.186773 | controller | cd+++++++++ repos/config/.git/objects/8f/ 2026-06-10 11:59:09.186810 | controller | >f+++++++++ repos/config/.git/objects/8f/3fdbd15aacab68911d9317340327344cc2b421 2026-06-10 11:59:09.186845 | controller | cd+++++++++ repos/config/.git/objects/af/ 2026-06-10 11:59:09.186880 | controller | >f+++++++++ repos/config/.git/objects/af/b38036ce517c054a2b7679ea42d4a724d3c8b5 2026-06-10 11:59:09.186916 | controller | cd+++++++++ repos/config/.git/objects/b4/ 2026-06-10 11:59:09.186953 | controller | >f+++++++++ repos/config/.git/objects/b4/8c64172c9d244e7dea5a851a55fa075c667c27 2026-06-10 11:59:09.186989 | controller | cd+++++++++ repos/config/.git/objects/c5/ 2026-06-10 11:59:09.187025 | controller | >f+++++++++ repos/config/.git/objects/c5/1f88eb0031ad26c880750c9bf9b45ccdad17c9 2026-06-10 11:59:09.187062 | controller | cd+++++++++ repos/config/.git/objects/c8/ 2026-06-10 11:59:09.187098 | controller | >f+++++++++ repos/config/.git/objects/c8/c94da96e5043450bcc719e889be06556a538b6 2026-06-10 11:59:09.187133 | controller | cd+++++++++ repos/config/.git/objects/cc/ 2026-06-10 11:59:09.187167 | controller | >f+++++++++ repos/config/.git/objects/cc/6450d887904229c514a2a58f48d4f6231d6d67 2026-06-10 11:59:09.187209 | controller | cd+++++++++ repos/config/.git/objects/d1/ 2026-06-10 11:59:09.187247 | controller | >f+++++++++ repos/config/.git/objects/d1/d7925a11fc1e119fceab7019d37c870cd877f0 2026-06-10 11:59:09.187283 | controller | cd+++++++++ repos/config/.git/objects/db/ 2026-06-10 11:59:09.187319 | controller | >f+++++++++ repos/config/.git/objects/db/99c46d9145bacbf5061adeb6c276c325950f24 2026-06-10 11:59:09.187390 | controller | >f+++++++++ repos/config/.git/objects/db/b4fa5be20399b6563bb6246ad7b83f5195b899 2026-06-10 11:59:09.187431 | controller | cd+++++++++ repos/config/.git/objects/e6/ 2026-06-10 11:59:09.187467 | controller | >f+++++++++ repos/config/.git/objects/e6/a2e27c77913f57098f9f2b0d546594d343c988 2026-06-10 11:59:09.187504 | controller | cd+++++++++ repos/config/.git/objects/eb/ 2026-06-10 11:59:09.187540 | controller | >f+++++++++ repos/config/.git/objects/eb/1416aeffe694a26c8784a3753731c42e99ba30 2026-06-10 11:59:09.187576 | controller | cd+++++++++ repos/config/.git/objects/f7/ 2026-06-10 11:59:09.187612 | controller | >f+++++++++ repos/config/.git/objects/f7/d04d5eafabb07a825120cdf276a128c7b1d46c 2026-06-10 11:59:09.187646 | controller | cd+++++++++ repos/config/.git/objects/info/ 2026-06-10 11:59:09.187679 | controller | cd+++++++++ repos/config/.git/objects/pack/ 2026-06-10 11:59:09.187712 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-60f8f1975bdc75ac8a316e826a9c828a3cf3138c.idx 2026-06-10 11:59:09.187747 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-60f8f1975bdc75ac8a316e826a9c828a3cf3138c.pack 2026-06-10 11:59:09.187780 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-60f8f1975bdc75ac8a316e826a9c828a3cf3138c.rev 2026-06-10 11:59:09.187815 | controller | cd+++++++++ repos/config/.git/refs/ 2026-06-10 11:59:09.187850 | controller | cd+++++++++ repos/config/.git/refs/heads/ 2026-06-10 11:59:09.187885 | controller | >f+++++++++ repos/config/.git/refs/heads/master 2026-06-10 11:59:09.187921 | controller | cd+++++++++ repos/config/.git/refs/remotes/ 2026-06-10 11:59:09.187957 | controller | cd+++++++++ repos/config/.git/refs/remotes/gerrit/ 2026-06-10 11:59:09.187992 | controller | >f+++++++++ repos/config/.git/refs/remotes/gerrit/HEAD 2026-06-10 11:59:09.188026 | controller | >f+++++++++ repos/config/.git/refs/remotes/gerrit/master 2026-06-10 11:59:09.188062 | controller | cd+++++++++ repos/config/.git/refs/remotes/origin/ 2026-06-10 11:59:09.188097 | controller | >f+++++++++ repos/config/.git/refs/remotes/origin/HEAD 2026-06-10 11:59:09.188132 | controller | >f+++++++++ repos/config/.git/refs/remotes/origin/master 2026-06-10 11:59:09.188167 | controller | cd+++++++++ repos/config/.git/refs/rewritten/ 2026-06-10 11:59:09.188201 | controller | cd+++++++++ repos/config/.git/refs/tags/ 2026-06-10 11:59:09.188236 | controller | cd+++++++++ repos/config/nodepool/ 2026-06-10 11:59:09.188271 | controller | >f+++++++++ repos/config/nodepool/nodepool.yaml 2026-06-10 11:59:09.188303 | controller | cd+++++++++ repos/config/zuul/ 2026-06-10 11:59:09.188337 | controller | >f+++++++++ repos/config/zuul/main.yaml 2026-06-10 11:59:09.188411 | controller | cd+++++++++ repos/demo-project/ 2026-06-10 11:59:09.188450 | controller | >f+++++++++ repos/demo-project/.gitreview 2026-06-10 11:59:09.188486 | controller | cd+++++++++ repos/demo-project/.git/ 2026-06-10 11:59:09.188523 | controller | >f+++++++++ repos/demo-project/.git/COMMIT_EDITMSG 2026-06-10 11:59:09.188559 | controller | >f+++++++++ repos/demo-project/.git/FETCH_HEAD 2026-06-10 11:59:09.188593 | controller | >f+++++++++ repos/demo-project/.git/HEAD 2026-06-10 11:59:09.188627 | controller | >f+++++++++ repos/demo-project/.git/ORIG_HEAD 2026-06-10 11:59:09.188662 | controller | >f+++++++++ repos/demo-project/.git/config 2026-06-10 11:59:09.188697 | controller | >f+++++++++ repos/demo-project/.git/description 2026-06-10 11:59:09.188730 | controller | >f+++++++++ repos/demo-project/.git/index 2026-06-10 11:59:09.188764 | controller | >f+++++++++ repos/demo-project/.git/packed-refs 2026-06-10 11:59:09.188809 | controller | cd+++++++++ repos/demo-project/.git/hooks/ 2026-06-10 11:59:09.192267 | controller | >f+++++++++ repos/demo-project/.git/hooks/applypatch-msg.sample 2026-06-10 11:59:09.192336 | controller | >f+++++++++ repos/demo-project/.git/hooks/commit-msg 2026-06-10 11:59:09.192407 | controller | >f+++++++++ repos/demo-project/.git/hooks/commit-msg.sample 2026-06-10 11:59:09.192533 | controller | >f+++++++++ repos/demo-project/.git/hooks/fsmonitor-watchman.sample 2026-06-10 11:59:09.192575 | controller | >f+++++++++ repos/demo-project/.git/hooks/post-update.sample 2026-06-10 11:59:09.192611 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-applypatch.sample 2026-06-10 11:59:09.192645 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-commit.sample 2026-06-10 11:59:09.192678 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-merge-commit.sample 2026-06-10 11:59:09.192712 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-push.sample 2026-06-10 11:59:09.192744 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-rebase.sample 2026-06-10 11:59:09.192777 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-receive.sample 2026-06-10 11:59:09.192811 | controller | >f+++++++++ repos/demo-project/.git/hooks/prepare-commit-msg.sample 2026-06-10 11:59:09.192847 | controller | >f+++++++++ repos/demo-project/.git/hooks/push-to-checkout.sample 2026-06-10 11:59:09.192881 | controller | >f+++++++++ repos/demo-project/.git/hooks/sendemail-validate.sample 2026-06-10 11:59:09.192914 | controller | >f+++++++++ repos/demo-project/.git/hooks/update.sample 2026-06-10 11:59:09.193049 | controller | cd+++++++++ repos/demo-project/.git/info/ 2026-06-10 11:59:09.193266 | controller | >f+++++++++ repos/demo-project/.git/info/exclude 2026-06-10 11:59:09.193296 | controller | cd+++++++++ repos/demo-project/.git/logs/ 2026-06-10 11:59:09.193326 | controller | >f+++++++++ repos/demo-project/.git/logs/HEAD 2026-06-10 11:59:09.193393 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/ 2026-06-10 11:59:09.193432 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/heads/ 2026-06-10 11:59:09.193465 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/master 2026-06-10 11:59:09.193499 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/test_check_url 2026-06-10 11:59:09.193530 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/test_short_job 2026-06-10 11:59:09.193564 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/ 2026-06-10 11:59:09.193601 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/ 2026-06-10 11:59:09.193635 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/HEAD 2026-06-10 11:59:09.193671 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/master 2026-06-10 11:59:09.193707 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/ 2026-06-10 11:59:09.193741 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/HEAD 2026-06-10 11:59:09.193776 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/master 2026-06-10 11:59:09.193811 | controller | cd+++++++++ repos/demo-project/.git/objects/ 2026-06-10 11:59:09.193847 | controller | cd+++++++++ repos/demo-project/.git/objects/07/ 2026-06-10 11:59:09.193882 | controller | >f+++++++++ repos/demo-project/.git/objects/07/6029a779c368a7dcb34abb1ab3c64dbf15caf8 2026-06-10 11:59:09.193918 | controller | cd+++++++++ repos/demo-project/.git/objects/10/ 2026-06-10 11:59:09.193952 | controller | >f+++++++++ repos/demo-project/.git/objects/10/6cf75c4c81970c7e2d871c6abb704d62ebc7f4 2026-06-10 11:59:09.193987 | controller | cd+++++++++ repos/demo-project/.git/objects/1f/ 2026-06-10 11:59:09.194022 | controller | >f+++++++++ repos/demo-project/.git/objects/1f/6e184de5b75dea653c90252bd2a6517e80be85 2026-06-10 11:59:09.194056 | controller | cd+++++++++ repos/demo-project/.git/objects/26/ 2026-06-10 11:59:09.194096 | controller | >f+++++++++ repos/demo-project/.git/objects/26/a3816fbd763ed648023f02849f8c3b0db3966c 2026-06-10 11:59:09.194128 | controller | cd+++++++++ repos/demo-project/.git/objects/2b/ 2026-06-10 11:59:09.194159 | controller | >f+++++++++ repos/demo-project/.git/objects/2b/f9cc206fec899bca2e61b1bafeff8e993529d8 2026-06-10 11:59:09.194190 | controller | cd+++++++++ repos/demo-project/.git/objects/3e/ 2026-06-10 11:59:09.194220 | controller | >f+++++++++ repos/demo-project/.git/objects/3e/f50d9ef392cd509f0f4d1d324f7465d2b29d04 2026-06-10 11:59:09.194252 | controller | cd+++++++++ repos/demo-project/.git/objects/45/ 2026-06-10 11:59:09.194284 | controller | >f+++++++++ repos/demo-project/.git/objects/45/d7af879c2d8367402de8718c0be637fecbc8a5 2026-06-10 11:59:09.194317 | controller | cd+++++++++ repos/demo-project/.git/objects/48/ 2026-06-10 11:59:09.194378 | controller | >f+++++++++ repos/demo-project/.git/objects/48/c7e8a6339bdda2526991aa9fb3707cb253e2c4 2026-06-10 11:59:09.194423 | controller | cd+++++++++ repos/demo-project/.git/objects/4c/ 2026-06-10 11:59:09.194460 | controller | >f+++++++++ repos/demo-project/.git/objects/4c/df6f9f6741b99bb0f1405b622a0671eb9c694d 2026-06-10 11:59:09.194518 | controller | cd+++++++++ repos/demo-project/.git/objects/4d/ 2026-06-10 11:59:09.194556 | controller | >f+++++++++ repos/demo-project/.git/objects/4d/d05ea0a58950ee887bfe1b67cfd7f44b8bba35 2026-06-10 11:59:09.194594 | controller | cd+++++++++ repos/demo-project/.git/objects/51/ 2026-06-10 11:59:09.194630 | controller | >f+++++++++ repos/demo-project/.git/objects/51/febfc882d29214e8ddb275b148abb77a18bac4 2026-06-10 11:59:09.194665 | controller | cd+++++++++ repos/demo-project/.git/objects/5f/ 2026-06-10 11:59:09.194698 | controller | >f+++++++++ repos/demo-project/.git/objects/5f/a965ccd5f8194e7865f9e02143de7481c0ea8c 2026-06-10 11:59:09.194732 | controller | cd+++++++++ repos/demo-project/.git/objects/60/ 2026-06-10 11:59:09.194765 | controller | >f+++++++++ repos/demo-project/.git/objects/60/4a228bad7e3a0aeb33c04c5f610ca1774ff81c 2026-06-10 11:59:09.194804 | controller | cd+++++++++ repos/demo-project/.git/objects/67/ 2026-06-10 11:59:09.194841 | controller | >f+++++++++ repos/demo-project/.git/objects/67/f112f627ce8ef651c2bb3c0f08aa4b8af7fcf7 2026-06-10 11:59:09.194876 | controller | cd+++++++++ repos/demo-project/.git/objects/6a/ 2026-06-10 11:59:09.194911 | controller | >f+++++++++ repos/demo-project/.git/objects/6a/5efc2360af79010addc99fc233385f250583f2 2026-06-10 11:59:09.194944 | controller | cd+++++++++ repos/demo-project/.git/objects/6e/ 2026-06-10 11:59:09.194973 | controller | >f+++++++++ repos/demo-project/.git/objects/6e/83e8e36a032c915b2a63db99231271d5903804 2026-06-10 11:59:09.195002 | controller | cd+++++++++ repos/demo-project/.git/objects/6f/ 2026-06-10 11:59:09.195030 | controller | >f+++++++++ repos/demo-project/.git/objects/6f/67f4852e045e768a87debd9780f6f03ebe1a70 2026-06-10 11:59:09.195058 | controller | cd+++++++++ repos/demo-project/.git/objects/72/ 2026-06-10 11:59:09.195087 | controller | >f+++++++++ repos/demo-project/.git/objects/72/dfc9c9553ebbd3d3f399036f015fd433cea17c 2026-06-10 11:59:09.195119 | controller | cd+++++++++ repos/demo-project/.git/objects/74/ 2026-06-10 11:59:09.195154 | controller | >f+++++++++ repos/demo-project/.git/objects/74/32b55d7db19f8c788f8fe355819a630f28906d 2026-06-10 11:59:09.195187 | controller | cd+++++++++ repos/demo-project/.git/objects/76/ 2026-06-10 11:59:09.195221 | controller | >f+++++++++ repos/demo-project/.git/objects/76/329e0d45f54dea4bdf76328e8597e999225689 2026-06-10 11:59:09.195255 | controller | cd+++++++++ repos/demo-project/.git/objects/7e/ 2026-06-10 11:59:09.195289 | controller | >f+++++++++ repos/demo-project/.git/objects/7e/612a5b8cd007488e7ccf127fc11b187cebf2b8 2026-06-10 11:59:09.195322 | controller | cd+++++++++ repos/demo-project/.git/objects/91/ 2026-06-10 11:59:09.195392 | controller | >f+++++++++ repos/demo-project/.git/objects/91/f03153690540edb808e62863bd58e78242a750 2026-06-10 11:59:09.195439 | controller | cd+++++++++ repos/demo-project/.git/objects/93/ 2026-06-10 11:59:09.195474 | controller | >f+++++++++ repos/demo-project/.git/objects/93/e9e8acefcdbd3e8547db88c5a1c02665f398a9 2026-06-10 11:59:09.195508 | controller | cd+++++++++ repos/demo-project/.git/objects/98/ 2026-06-10 11:59:09.195541 | controller | >f+++++++++ repos/demo-project/.git/objects/98/d81f09fe1cd3e3df123e0f5812b335036b5980 2026-06-10 11:59:09.195667 | controller | cd+++++++++ repos/demo-project/.git/objects/9d/ 2026-06-10 11:59:09.195715 | controller | >f+++++++++ repos/demo-project/.git/objects/9d/139d568fb3c661b5a0c54ac6e96a1c87231bbc 2026-06-10 11:59:09.195752 | controller | cd+++++++++ repos/demo-project/.git/objects/a1/ 2026-06-10 11:59:09.195786 | controller | >f+++++++++ repos/demo-project/.git/objects/a1/cf2718d30acfe1c72df067ba45715c6e6b7d28 2026-06-10 11:59:09.195821 | controller | cd+++++++++ repos/demo-project/.git/objects/ab/ 2026-06-10 11:59:09.195855 | controller | >f+++++++++ repos/demo-project/.git/objects/ab/870c85c38400f61b7b23c74055cc9641d5eb5a 2026-06-10 11:59:09.195889 | controller | cd+++++++++ repos/demo-project/.git/objects/af/ 2026-06-10 11:59:09.195923 | controller | >f+++++++++ repos/demo-project/.git/objects/af/b9c51ff287f6408185ec86d9f4781cbe40472e 2026-06-10 11:59:09.195955 | controller | cd+++++++++ repos/demo-project/.git/objects/c8/ 2026-06-10 11:59:09.195988 | controller | >f+++++++++ repos/demo-project/.git/objects/c8/f5c5f9ea4b45f06003754c50dfca1bf26077f0 2026-06-10 11:59:09.196022 | controller | cd+++++++++ repos/demo-project/.git/objects/cc/ 2026-06-10 11:59:09.196055 | controller | >f+++++++++ repos/demo-project/.git/objects/cc/771e1c5a19a0d0aa69b51ad908856824ba12c7 2026-06-10 11:59:09.196089 | controller | cd+++++++++ repos/demo-project/.git/objects/d0/ 2026-06-10 11:59:09.196122 | controller | >f+++++++++ repos/demo-project/.git/objects/d0/2db411aaa63f2a85daf1c6c871d8ca080778ba 2026-06-10 11:59:09.196154 | controller | >f+++++++++ repos/demo-project/.git/objects/d0/e66ab409ee4347449c43a1e3773363c72488b5 2026-06-10 11:59:09.196188 | controller | cd+++++++++ repos/demo-project/.git/objects/d4/ 2026-06-10 11:59:09.196222 | controller | >f+++++++++ repos/demo-project/.git/objects/d4/ad8d0fd75f2c12b671a0a497958c9d366b672c 2026-06-10 11:59:09.196254 | controller | cd+++++++++ repos/demo-project/.git/objects/db/ 2026-06-10 11:59:09.196284 | controller | >f+++++++++ repos/demo-project/.git/objects/db/f005f0f2fd8dba8616e57aa8b0639289245960 2026-06-10 11:59:09.196309 | controller | cd+++++++++ repos/demo-project/.git/objects/e7/ 2026-06-10 11:59:09.196335 | controller | >f+++++++++ repos/demo-project/.git/objects/e7/5d37d0e529b222732a993e37a6ad10b51febe6 2026-06-10 11:59:09.196417 | controller | cd+++++++++ repos/demo-project/.git/objects/info/ 2026-06-10 11:59:09.196456 | controller | cd+++++++++ repos/demo-project/.git/objects/pack/ 2026-06-10 11:59:09.196491 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-885320b5811f6c6c07121f72ae5bee31ba5ea554.idx 2026-06-10 11:59:09.196524 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-885320b5811f6c6c07121f72ae5bee31ba5ea554.pack 2026-06-10 11:59:09.196555 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-885320b5811f6c6c07121f72ae5bee31ba5ea554.rev 2026-06-10 11:59:09.196589 | controller | cd+++++++++ repos/demo-project/.git/refs/ 2026-06-10 11:59:09.196625 | controller | cd+++++++++ repos/demo-project/.git/refs/heads/ 2026-06-10 11:59:09.196660 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/master 2026-06-10 11:59:09.196691 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/test_check_url 2026-06-10 11:59:09.196721 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/test_short_job 2026-06-10 11:59:09.196751 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/ 2026-06-10 11:59:09.196780 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/gerrit/ 2026-06-10 11:59:09.196812 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/gerrit/HEAD 2026-06-10 11:59:09.196854 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/gerrit/master 2026-06-10 11:59:09.196889 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/origin/ 2026-06-10 11:59:09.196922 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/origin/HEAD 2026-06-10 11:59:09.196952 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/origin/master 2026-06-10 11:59:09.196982 | controller | cd+++++++++ repos/demo-project/.git/refs/rewritten/ 2026-06-10 11:59:09.197012 | controller | cd+++++++++ repos/demo-project/.git/refs/tags/ 2026-06-10 11:59:09.197042 | controller | cd+++++++++ repos/demo-project/.zuul.d/ 2026-06-10 11:59:09.197071 | controller | >f+++++++++ repos/demo-project/.zuul.d/jobs.yaml 2026-06-10 11:59:09.197101 | controller | >f+++++++++ repos/demo-project/.zuul.d/project.yaml 2026-06-10 11:59:09.197131 | controller | >f+++++++++ repos/demo-project/.zuul.d/secrets.yaml 2026-06-10 11:59:09.197160 | controller | cd+++++++++ repos/demo-project/playbooks/ 2026-06-10 11:59:09.197183 | controller | >f+++++++++ repos/demo-project/playbooks/microshift-pod.yml 2026-06-10 11:59:09.197205 | controller | >f+++++++++ repos/demo-project/playbooks/run-check-url.yaml 2026-06-10 11:59:09.197228 | controller | >f+++++++++ repos/demo-project/playbooks/run.yaml 2026-06-10 11:59:09.197250 | controller | cd+++++++++ repos/demo-project/zuul.d/ 2026-06-10 11:59:09.197272 | controller | >f+++++++++ repos/demo-project/zuul.d/config.yaml 2026-06-10 11:59:09.197294 | controller | cd+++++++++ repos/demo-tenant-config/ 2026-06-10 11:59:09.197316 | controller | >f+++++++++ repos/demo-tenant-config/.gitreview 2026-06-10 11:59:09.197338 | controller | cd+++++++++ repos/demo-tenant-config/.git/ 2026-06-10 11:59:09.197401 | controller | >f+++++++++ repos/demo-tenant-config/.git/COMMIT_EDITMSG 2026-06-10 11:59:09.197427 | controller | >f+++++++++ repos/demo-tenant-config/.git/HEAD 2026-06-10 11:59:09.197450 | controller | >f+++++++++ repos/demo-tenant-config/.git/ORIG_HEAD 2026-06-10 11:59:09.197473 | controller | >f+++++++++ repos/demo-tenant-config/.git/config 2026-06-10 11:59:09.197496 | controller | >f+++++++++ repos/demo-tenant-config/.git/description 2026-06-10 11:59:09.197519 | controller | >f+++++++++ repos/demo-tenant-config/.git/index 2026-06-10 11:59:09.197542 | controller | >f+++++++++ repos/demo-tenant-config/.git/packed-refs 2026-06-10 11:59:09.197566 | controller | cd+++++++++ repos/demo-tenant-config/.git/hooks/ 2026-06-10 11:59:09.197599 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/applypatch-msg.sample 2026-06-10 11:59:09.197635 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/commit-msg.sample 2026-06-10 11:59:09.197670 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/fsmonitor-watchman.sample 2026-06-10 11:59:09.197703 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/post-update.sample 2026-06-10 11:59:09.197727 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-applypatch.sample 2026-06-10 11:59:09.197750 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-commit.sample 2026-06-10 11:59:09.197773 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-merge-commit.sample 2026-06-10 11:59:09.197796 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-push.sample 2026-06-10 11:59:09.197819 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-rebase.sample 2026-06-10 11:59:09.197841 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-receive.sample 2026-06-10 11:59:09.197864 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/prepare-commit-msg.sample 2026-06-10 11:59:09.197889 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/push-to-checkout.sample 2026-06-10 11:59:09.197924 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/sendemail-validate.sample 2026-06-10 11:59:09.197961 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/update.sample 2026-06-10 11:59:09.198003 | controller | cd+++++++++ repos/demo-tenant-config/.git/info/ 2026-06-10 11:59:09.198038 | controller | >f+++++++++ repos/demo-tenant-config/.git/info/exclude 2026-06-10 11:59:09.198075 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/ 2026-06-10 11:59:09.198111 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/HEAD 2026-06-10 11:59:09.198146 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/ 2026-06-10 11:59:09.198180 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/heads/ 2026-06-10 11:59:09.198214 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/heads/master 2026-06-10 11:59:09.198249 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/ 2026-06-10 11:59:09.198283 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/ 2026-06-10 11:59:09.198317 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/HEAD 2026-06-10 11:59:09.198382 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/master 2026-06-10 11:59:09.198428 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/ 2026-06-10 11:59:09.198464 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/35/ 2026-06-10 11:59:09.198519 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/35/b27848a551ba9940a7d29be8ac540ae6bf76c6 2026-06-10 11:59:09.198558 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/4f/ 2026-06-10 11:59:09.198596 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/4f/c04c476415cd04bae93b5a977b4d05425bf61f 2026-06-10 11:59:09.198630 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/71/ 2026-06-10 11:59:09.198664 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/71/766da1ed79ae81b8510541bc42a17123e9f4c7 2026-06-10 11:59:09.198697 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/7e/ 2026-06-10 11:59:09.198729 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/7e/56626d48d97eee76893616729b62b50ce1d6aa 2026-06-10 11:59:09.198757 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/82/ 2026-06-10 11:59:09.198780 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/82/7fb999b752e98c420a55a6844053be01b085ed 2026-06-10 11:59:09.198803 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/9e/ 2026-06-10 11:59:09.198825 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/9e/d5bc9a591ecf02f8b90e37ef0395c13113ad9d 2026-06-10 11:59:09.198848 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/b2/ 2026-06-10 11:59:09.198870 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/b2/9ffbd8fb2947c43ddcd388735096ad3451acaa 2026-06-10 11:59:09.198893 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/e4/ 2026-06-10 11:59:09.198915 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/e4/641b65f1174ec39331eff28a4e1223a9b05c89 2026-06-10 11:59:09.198938 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/info/ 2026-06-10 11:59:09.198960 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/pack/ 2026-06-10 11:59:09.198983 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-a38868ed34ec8cc116db3bea192903f9d0ba9536.idx 2026-06-10 11:59:09.199008 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-a38868ed34ec8cc116db3bea192903f9d0ba9536.pack 2026-06-10 11:59:09.199042 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-a38868ed34ec8cc116db3bea192903f9d0ba9536.rev 2026-06-10 11:59:09.199074 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/ 2026-06-10 11:59:09.199104 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/heads/ 2026-06-10 11:59:09.199136 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/heads/master 2026-06-10 11:59:09.199167 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/remotes/ 2026-06-10 11:59:09.199198 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/ 2026-06-10 11:59:09.199238 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/HEAD 2026-06-10 11:59:09.199276 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/master 2026-06-10 11:59:09.199309 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/tags/ 2026-06-10 11:59:09.199342 | controller | cd+++++++++ repos/demo-tenant-config/playbooks/ 2026-06-10 11:59:09.199413 | controller | >f+++++++++ repos/demo-tenant-config/playbooks/gerrit-pre.yaml 2026-06-10 11:59:09.199449 | controller | cd+++++++++ repos/demo-tenant-config/zuul.d/ 2026-06-10 11:59:09.199484 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-base-jobs.yaml 2026-06-10 11:59:09.199519 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-pipeline.yaml 2026-06-10 11:59:09.199549 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-project-pipeline.yaml 2026-06-10 11:59:09.199733 | controller | changed: All items complete 2026-06-10 11:59:09.199772 | 2026-06-10 11:59:09.731808 | controller | changed: .d..t...... ./ 2026-06-10 11:59:10.319074 | controller | changed: .d..t...... ./ 2026-06-10 11:59:10.347963 | 2026-06-10 11:59:10.348192 | TASK [include_role : fetch-output-openshift] 2026-06-10 11:59:10.378924 | controller | skipping: Conditional result was False 2026-06-10 11:59:10.386517 | 2026-06-10 11:59:10.386706 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-06-10 11:59:10.934684 | controller -> localhost | ok: Item: artifacts Runtime: 0:00:00.012803 2026-06-10 11:59:11.233534 | controller -> localhost | ok: Item: docs Runtime: 0:00:00.012381 2026-06-10 11:59:11.262319 | 2026-06-10 11:59:11.262539 | PLAY [localhost] 2026-06-10 11:59:11.319850 | 2026-06-10 11:59:11.320037 | TASK [add-fileserver : Create SSH private key tempfile] 2026-06-10 11:59:11.719342 | localhost | changed 2026-06-10 11:59:11.730809 | 2026-06-10 11:59:11.730981 | TASK [add-fileserver : Create SSH private key from secret] 2026-06-10 11:59:12.514812 | localhost | changed 2026-06-10 11:59:12.523407 | 2026-06-10 11:59:12.523550 | TASK [add-fileserver : Add fileserver ssh key] 2026-06-10 11:59:12.962470 | localhost | Identity added: /var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/work/tmp/ansible.9jnnenh_ (/var/lib/zuul/builds/74204cee34be4cddbbd3b400f2a79c3d/work/tmp/ansible.9jnnenh_) 2026-06-10 11:59:12.962736 | localhost | ok: Runtime: 0:00:00.022984 2026-06-10 11:59:12.968024 | 2026-06-10 11:59:12.968122 | TASK [add-fileserver : Remove SSH private key from disk] 2026-06-10 11:59:13.314087 | localhost | ok: Runtime: 0:00:00.005492 2026-06-10 11:59:13.317766 | 2026-06-10 11:59:13.317826 | TASK [add-fileserver : Add fileserver to inventory] 2026-06-10 11:59:13.365334 | localhost | changed 2026-06-10 11:59:13.368536 | 2026-06-10 11:59:13.368597 | TASK [add-fileserver : Add fileserver server to known hosts] 2026-06-10 11:59:13.799110 | localhost | changed 2026-06-10 11:59:13.816079 | 2026-06-10 11:59:13.816205 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-06-10 11:59:14.265194 | localhost | changed 2026-06-10 11:59:14.289522 | 2026-06-10 11:59:14.289686 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-06-10 11:59:14.324947 | localhost | ok 2026-06-10 11:59:14.339327 | 2026-06-10 11:59:14.339485 | TASK [zuul_return] 2026-06-10 11:59:14.406516 | localhost | skipping: Conditional result was False 2026-06-10 11:59:14.411760 | 2026-06-10 11:59:14.411875 | PLAY [a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com] 2026-06-10 11:59:14.421989 | 2026-06-10 11:59:14.422122 | TASK [Set zuul-log-path fact] 2026-06-10 11:59:14.446967 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | ok 2026-06-10 11:59:14.460475 | 2026-06-10 11:59:14.460608 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 11:59:14.499642 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | ok 2026-06-10 11:59:14.506802 | 2026-06-10 11:59:14.506930 | TASK [upload-logs : Create log directories] 2026-06-10 11:59:15.317291 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | changed 2026-06-10 11:59:15.321089 | 2026-06-10 11:59:15.321227 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-06-10 11:59:15.685462 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com -> localhost | ok: Runtime: 0:00:00.020062 2026-06-10 11:59:15.689872 | 2026-06-10 11:59:15.690009 | TASK [upload-logs : Upload logs to log server] 2026-06-10 11:59:37.143322 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | changed: 2026-06-10 11:59:37.143617 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | .d..tp..... ./ 2026-06-10 11:59:37.143669 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com |