kubevirt make cluster-sync pull 镜像timeout

尝试在本地编译 kubevirt 源码，在执行 make cluster-sync 发生报错。
环境信息:
root@kubevirt:~/kubevirt# lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 22.04 LTS
Release: 22.04
Codename: jammy

操作记录

# pull kubevirt
git clone git@github.com:kubevirt/kubevirt.git
# 切换tag
git checkout v1.1.1

# 使用kind创建k8s
export KUBEVIRT_PROVIDER=kind-1.27
make cluster-up

# 编译源码并同步到k8s
make cluster-sync
...
...
INFO: Repository fedora_with_test_tooling instantiated at:
  /root/go/src/kubevirt.io/kubevirt/WORKSPACE:308:15: in <toplevel>
Repository rule container_pull defined at:
  /root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/io_bazel_rules_docker/container/pull.bzl:243:33: in <toplevel>
ERROR: An error occurred during the fetch of repository 'fedora_with_test_tooling':
   Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/io_bazel_rules_docker/container/pull.bzl", line 189, column 13, in _impl
                fail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))
Error in fail: Pull command failed: Timed out (/root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/go_puller_linux_amd64/file/downloaded -directory /root/.cache/bazel/_bazel_root/6f347497f91
c9a385dcd9294645b76e0/external/fedora_with_test_tooling/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name quay.io/kubevirtci/fedora-with-test-tooling@sha256:23dfbf35feb0eb
fd2bcf85fe94c94f5ad50bca8762001ee0230f8381f79f52ef)
ERROR: /root/go/src/kubevirt.io/kubevirt/WORKSPACE:308:15: fetching container_pull rule //external:fedora_with_test_tooling: Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/io_bazel_rules_docker/container/pull.bzl", line 189, column 13, in _impl
                fail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))
Error in fail: Pull command failed: Timed out (/root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/go_puller_linux_amd64/file/downloaded -directory /root/.cache/bazel/_bazel_root/6f347497f91
c9a385dcd9294645b76e0/external/fedora_with_test_tooling/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name quay.io/kubevirtci/fedora-with-test-tooling@sha256:23dfbf35feb0eb
fd2bcf85fe94c94f5ad50bca8762001ee0230f8381f79f52ef)
ERROR: /root/go/src/kubevirt.io/kubevirt/containerimages/BUILD.bazel:86:16: //containerimages:fedora-with-test-tooling depends on @fedora_with_test_tooling//image:image in repository @fedora_with_test_tooling wh
ich failed to fetch. no such package '@fedora_with_test_tooling//image': Pull command failed: Timed out (/root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/go_puller_linux_amd64/file/downlo
aded -directory /root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/fedora_with_test_tooling/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name quay.io
/kubevirtci/fedora-with-test-tooling@sha256:23dfbf35feb0ebfd2bcf85fe94c94f5ad50bca8762001ee0230f8381f79f52ef)
ERROR: Analysis of target '//:push-fedora-with-test-tooling-container-disk' failed; build aborted:
INFO: Elapsed time: 600.464s
INFO: 0 processes.

分析

报错信息是可以看到是 pull command failed: timed out. Elapsed time: 600.464s
下载镜像超时有两种可能
1.无法访问仓库导致的超时(https握手失败会立即报错不会等待600s这么久，排除);
2.因为镜像太大导致的下载镜像超时；

因为镜像太大导致的下载镜像超时

root@kubevirt:~/kubevirt# ps -ef|grep download
root     3162711 3146984  0 09:17 ?        00:00:00 /root/.cache/bazel/_bazel_root/install/7e78030801a38f836b1891b02e25eaaa/process-wrapper --timeout=800 --kill_delay=15 /root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/go_puller_linux_amd64/file/downloaded -directory /root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/go_image_base/image -os linux -os-version  -os-features -architecture amd64 -variant  -features  -name gcr.io/distroless/base@sha256:6c871aa3c9019984dfd7f520635bd658d740ad20c6268a82faa433f69dfc9a0b

# 进入到bazel server 查看镜像是否下载
root@kubevirt:~/kubevirt# docker exec -it -uroot kubevirt-bazel-server bash

[root@kubevirt kubevirt]# ls -lh /root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/fedora_with_test_tooling/image/
total 671M
-rwxr-xr-x 1 root root   64 Feb 20 01:20 000.sha256
-rw-r--r-- 1 root root 601M Feb 20 01:30 000.tar.gz
-rwxr-xr-x 1 root root  492 Feb 20 01:30 BUILD
-rwxr-xr-x 1 root root 1.2K Feb 20 01:20 config.json
-rwxr-xr-x 1 root root   71 Feb 20 01:20 digest
lrwxrwxrwx 1 root root  110 Feb 20 01:30 image.digest -> /root/.cache/bazel/_bazel_root/6f347497f91c9a385dcd9294645b76e0/external/fedora_with_test_tooling/image/digest
-rwxr-xr-x 1 root root  530 Feb 20 01:20 manifest.json
[root@kubevirt kubevirt]#

总结

可以看到镜像下载了一部分与我们分析情况一致，镜像太大或着网速太慢导致下载超时。查询rules_docker文档，为了防止其他镜像下载超时这里更改 PULLER_TIMEOUT 环境变量。
修改 .bazelrc 文件增加 –action_env=PULLER_TIMEOUT=10000
build:x86_64 --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64_cgo --compilation_mode=dbg --action_env=PULLER_TIMEOUT=10000