Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] copa CLI gets to a deadlock if package validation fails #503

Closed
1 task
mhrabovcin opened this issue Feb 14, 2024 · 2 comments · Fixed by #508
Closed
1 task

[BUG] copa CLI gets to a deadlock if package validation fails #503

mhrabovcin opened this issue Feb 14, 2024 · 2 comments · Fixed by #508
Assignees
@ashnamehrotra
Labels
bug Something isn't working

Comments

@mhrabovcin
Copy link

Version of copa

v0.6.0

Expected Behavior

CLI should propagate error and exist with appropriate status code.

Actual Behavior

CLI hangs until the provided patch timeout is reached.

copacetic/pkg/patch/patch.go

Lines 46 to 61 in 2f621d1

ch := make(chan error)
go func() {
ch <- patchWithContext(timeoutCtx, image, reportFile, patchedTag, workingFolder, scanner, format, output, ignoreError, bkOpts)
}()
select {
case err := <-ch:
return err
case <-timeoutCtx.Done():
// add a grace period for long running deferred cleanup functions to complete
<-time.After(1 * time.Second)
err := fmt.Errorf("patch exceeded timeout %v", timeout)
log.Error(err)
return err
}

Steps To Reproduce

Trivy scan:

trivy image cr.fluentbit.io/fluent/fluent-bit:2.1.4 --vuln-type os --ignore-unfixed -f json -o fluent-bit.2.1.4.json

Patch:

copa patch -i cr.fluentbit.io/fluent/fluent-bit:2.1.4 -r fluent-bit.2.1.4.json -t 2.1.4-patched

Output:

time="2024-02-14T10:56:44Z" level=debug msg="updates to apply: &{{{debian 11.7} {amd64}} [{libc6 2.31-13+deb11u6 2.31-13+deb11u7 CVE-2023-4911} {libgnutls30 3.7.1-5+deb11u3 3.7.1-5+deb11u5 CVE-2024-0567} {libgnutls30 3.7.1-5+deb11u3 3.7.1-5+deb11u5 CVE-2023-5981} {libgssapi-krb5-2 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libk5crypto3 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libkrb5-3 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libkrb5support0 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libpq5 13.11-0+deb11u1 13.13-0+deb11u1 CVE-2023-39417} {libpq5 13.11-0+deb11u1 13.13-0+deb11u1 CVE-2023-5869} {libpq5 13.11-0+deb11u1 13.13-0+deb11u1 CVE-2023-5868} {libpq5 13.11-0+deb11u1 13.13-0+deb11u1 CVE-2023-5870} {libssl1.1 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0464} {libssl1.1 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0465} {libssl1.1 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0466} {libssl1.1 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-2650} {libssl1.1 1.1.1n-0+deb11u4 1.1.1v-0~deb11u1 CVE-2023-3446} {libssl1.1 1.1.1n-0+deb11u4 1.1.1v-0~deb11u1 CVE-2023-3817} {openssl 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0464} {openssl 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0465} {openssl 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0466} {openssl 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-2650} {openssl 1.1.1n-0+deb11u4 1.1.1v-0~deb11u1 CVE-2023-3446} {openssl 1.1.1n-0+deb11u4 1.1.1v-0~deb11u1 CVE-2023-3817}]}"
time="2024-02-14T10:56:44Z" level=debug msg="Trying docker driver"
time="2024-02-14T10:56:44Z" level=debug msg="serving grpc connection"
time="2024-02-14T10:56:44Z" level=debug msg="stopping session"
time="2024-02-14T10:56:44Z" level=debug msg="Could not use docker driver" error="failed to solve: rpc error: code = Unknown desc = requested experimental feature mergeop  is not supported by build server, please update \nrequested experimental feature diffop  is not supported by build server, please update \nmissing required buildkit functionality"
time="2024-02-14T10:56:44Z" level=debug msg="Trying buildx driver"
time="2024-02-14T10:56:48Z" level=debug msg="Connect to buildx instance" driver=docker-container endpoint="unix:///var/run/docker.sock" name=copacetic-action0
time="2024-02-14T10:56:48Z" level=debug msg="serving grpc connection"
time="2024-02-14T10:56:48Z" level=debug msg="stopping session"
time="2024-02-14T10:56:48Z" level=debug msg="serving grpc connection"
#1 resolve image config for cr.fluentbit.io/fluent/fluent-bit:2.1.4
#1 DONE 0.4s
time="2024-02-14T10:56:49Z" level=debug msg="Using debian:11-slim as basis for tooling image"

#1 resolve image config for cr.fluentbit.io/fluent/fluent-bit:2.1.4
#1 DONE 1.0s

#2 docker-image://cr.fluentbit.io/fluent/fluent-bit:2.1.4
#2 resolve cr.fluentbit.io/fluent/fluent-bit:2.1.4 0.1s done
#2 DONE 0.4s

#3 docker-image://docker.io/library/debian:11-slim
#3 resolve docker.io/library/debian:11-slim
#3 resolve docker.io/library/debian:11-slim 1.1s done
#3 sha256:5d0aeceef7eeb53c3f853fb229ea7fd13a5a56f4ba371ca48f0477493046b702 5.24MB / 31.42MB 0.2s
#3 sha256:5d0aeceef7eeb53c3f853fb229ea7fd13a5a56f4ba371ca48f0477493046b702 31.42MB / 31.42MB 0.4s done
#3 ...

#2 docker-image://cr.fluentbit.io/fluent/fluent-bit:2.1.4
#2 sha256:51bb3c3cd8081d66f32e6102e9092512245826fe75af6ee705dc727e02c4dbe9 14.56MB / 14.56MB 0.5s done
#2 sha256:852c2006aee6e783c44098bd02554ed55be750c534680e4dd2e2a6bea0e58293 5.82MB / 5.82MB 0.5s done
#2 sha256:a5bce306db56202cff4b1f22b5a3347c56ff1f377a8391f7a3039b8f51a82f3f 123.70kB / 123.70kB 0.4s done
#2 sha256:4eec690774a46467a912715848c71dbbdb049008b2252432155522a7f9ccfa92 52.43kB / 52.43kB 0.8s done
#2 sha256:52907d314ddce378f3f36e26629baef60c71d72a0620b9d31c47c8cb9de6467e 741.34kB / 741.34kB 0.1s done
#2 sha256:dfc02eb7708f919bb3b56c008561e4430ea87cd33bc93cb65c2c3c7f0908e5cf 131.16kB / 131.16kB 0.1s done
#2 sha256:f51c8fa85103a5c3ac5ec1969b693ccc0dc2b409dec106e4025e31c436c0c740 961.97kB / 961.97kB 0.1s done
#2 sha256:96266735468f361ae6828901a80fc15a7f75e26640351df9e0f0f9824f36cf92 5.85MB / 5.85MB 0.1s done
#2 sha256:80ec95682aee84c463cbf19b82eef5e1707c5e31b7be9750e1a90bff8b4183ee 2.06MB / 2.06MB 0.1s done
#2 sha256:5627a970d25e752d971a501ec7e35d0d6fdcd4a3ce9e958715a686853024794a 130.56kB / 130.56kB 0.1s done
#2 sha256:7c881f9ab25e0d86562a123b5fb56aebf8aa0ddd7d48ef602faf8d1e7cf43d8c 355B / 355B 0.1s done
#2 sha256:4aa0ea1413d37a58615488592a0b827ea4b2e48fa5a77cf707d0e35f025e613f 385B / 385B 0.1s done
#2 sha256:1e3d9b7d145208fa8fa3ee1c9612d0adaac7255f1bbc9ddea7e461e0b317805c 113B / 113B 0.1s done
#2 sha256:e8c73c638ae9ec5ad70c49df7e484040d889cca6b4a9af056579c3d058ea93f0 198B / 198B 0.1s done
#2 sha256:fcb6f6d2c9986d9cd6a2ea3cc2936e5fc613e09f1af9042329011e43057f3265 317B / 317B 0.1s done
#2 sha256:fe5ca62666f04366c8e7f605aa82997d71320183e99962fa76b3209fdfbb8b58 21.20kB / 21.20kB 0.1s done
#2 sha256:a7ca0d9ba68fdce7e15bc0952d3e898e970548ca24d57698725836c039086639 103.73kB / 103.73kB 0.1s done
#2 extracting sha256:a7ca0d9ba68fdce7e15bc0952d3e898e970548ca24d57698725836c039086639 0.0s done
#2 sha256:b02a7525f878e61fc1ef8a7405a2cc17f866e8de222c1c98fd6681aff6e509db 716.49kB / 716.49kB 0.2s done
#2 extracting sha256:fe5ca62666f04366c8e7f605aa82997d71320183e99962fa76b3209fdfbb8b58 0.0s done
#2 extracting sha256:b02a7525f878e61fc1ef8a7405a2cc17f866e8de222c1c98fd6681aff6e509db 0.2s done
#2 extracting sha256:fcb6f6d2c9986d9cd6a2ea3cc2936e5fc613e09f1af9042329011e43057f3265 0.0s done
#2 DONE 1.6s

#3 docker-image://docker.io/library/debian:11-slim
#3 extracting sha256:5d0aeceef7eeb53c3f853fb229ea7fd13a5a56f4ba371ca48f0477493046b702
#3 extracting sha256:5d0aeceef7eeb53c3f853fb229ea7fd13a5a56f4ba371ca48f0477493046b702 0.8s done
#3 DONE 2.3s

#2 docker-image://cr.fluentbit.io/fluent/fluent-bit:2.1.4
#2 extracting sha256:e8c73c638ae9ec5ad70c49df7e484040d889cca6b4a9af056579c3d058ea93f0 0.0s done
#2 extracting sha256:1e3d9b7d145208fa8fa3ee1c9612d0adaac7255f1bbc9ddea7e461e0b317805c 0.0s done
#2 extracting sha256:4aa0ea1413d37a58615488592a0b827ea4b2e48fa5a77cf707d0e35f025e613f 0.0s done
#2 extracting sha256:7c881f9ab25e0d86562a123b5fb56aebf8aa0ddd7d48ef602faf8d1e7cf43d8c 0.0s done
#2 extracting sha256:5627a970d25e752d971a501ec7e35d0d6fdcd4a3ce9e958715a686853024794a 0.0s done
#2 extracting sha256:96266735468f361ae6828901a80fc15a7f75e26640351df9e0f0f9824f36cf92 0.1s done
#2 extracting sha256:80ec95682aee84c463cbf19b82eef5e1707c5e31b7be9750e1a90bff8b4183ee 0.0s done
#2 extracting sha256:f51c8fa85103a5c3ac5ec1969b693ccc0dc2b409dec106e4025e31c436c0c740 0.0s done
#2 extracting sha256:dfc02eb7708f919bb3b56c008561e4430ea87cd33bc93cb65c2c3c7f0908e5cf 0.0s done
#2 extracting sha256:52907d314ddce378f3f36e26629baef60c71d72a0620b9d31c47c8cb9de6467e 0.0s done
#2 extracting sha256:4eec690774a46467a912715848c71dbbdb049008b2252432155522a7f9ccfa92 0.0s done
#2 extracting sha256:852c2006aee6e783c44098bd02554ed55be750c534680e4dd2e2a6bea0e58293 0.2s done
#2 extracting sha256:a5bce306db56202cff4b1f22b5a3347c56ff1f377a8391f7a3039b8f51a82f3f 0.0s done
#2 extracting sha256:51bb3c3cd8081d66f32e6102e9092512245826fe75af6ee705dc727e02c4dbe9
#2 extracting sha256:51bb3c3cd8081d66f32e6102e9092512245826fe75af6ee705dc727e02c4dbe9 0.2s done
#2 DONE 2.4s

#4 apt update
#4 0.062 
#4 0.062 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#4 0.062 
#4 0.103 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
#4 0.131 Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
#4 0.142 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
#4 0.202 Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8068 kB]
#4 0.298 Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [268 kB]
#4 0.323 Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [18.8 kB]
#4 1.213 Fetched 8562 kB in 1s (7506 kB/s)
#4 1.213 Reading package lists...
#4 1.589 Building dependency tree...
#4 1.700 Reading state information...
#4 1.706 All packages are up to date.
#4 DONE 2.3s

#5 apt install busybox-static
#5 0.080 
#5 0.080 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#5 0.080 
#5 0.085 Reading package lists...
#5 0.468 Building dependency tree...
#5 0.561 Reading state information...
#5 0.658 The following NEW packages will be installed:
#5 0.658   busybox-static
#5 0.690 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
#5 0.690 Need to get 920 kB of archives.
#5 0.690 After this operation, 2111 kB of additional disk space will be used.
#5 0.690 Get:1 http://deb.debian.org/debian bullseye/main amd64 busybox-static amd64 1:1.30.1-6+b3 [920 kB]
#5 0.809 debconf: delaying package configuration, since apt-utils is not installed
#5 0.827 Fetched 920 kB in 0s (14.5 MB/s)
#5 0.843 Selecting previously unselected package busybox-static.
#5 0.843 (Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 6670 files and directories currently installed.)
#5 0.847 Preparing to unpack .../busybox-static_1%3a1.30.1-6+b3_amd64.deb ...
#5 0.851 Unpacking busybox-static (1:1.30.1-6+b3) ...
#5 0.943 Setting up busybox-static (1:1.30.1-6+b3) ...
#5 DONE 1.0s

#6 copy /bin/busybox /bin/busybox
#6 DONE 0.0s

#7 mkdir /copa-out
#7 DONE 0.0s

#8 /bin/busybox sh -c 
                status="$DPKG_STATUS_IS_UNKNOWN"
                if [ -f "$DPKG_STATUS_PATH" ]; then
                    status="$DPKG_STATUS_IS_FILE"
                    cp "$DPKG_STATUS_PATH" "$RESULTS_PATH"
                elif [ -d "$DPKG_STATUS_FOLDER" ]; then
                    status="$DPKG_STATUS_IS_DIRECTORY"
                    ls -1 "$DPKG_STATUS_FOLDER" > "$RESULT_STATUSD_PATH"
                fi
                echo -n "$status" > "${RESULTS_PATH}/${STATUSD_OUTPUT_FILENAME}"
        
#8 DONE 0.1s
time="2024-02-14T10:56:55Z" level=info msg="Processed status.d: base-files base-files.md5sums ca-certificates libatomic1 libc6 libc6.md5sums libcom-err2 libffi7 libgcc-s1 libgcc-s1.md5sums libgcrypt20 libgmp10 libgnutls30 libgomp1 libgomp1.md5sums libgpg-error0 libgssapi-krb5-2 libhogweed6 libidn2-0 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap-2.4-2 liblz4-1 liblzma5 libnettle8 libp11-kit0 libpq5 libsasl2-2 libssl1.1 libssl1.1.md5sums libstdc++6 libstdc++6.md5sums libsystemd0 libtasn1-6 libunistring2 libyaml-0-2 libzstd1 netbase netbase.md5sums openssl openssl.md5sums pkg-config tzdata tzdata.md5sums zlib1g"

#9 apt download --no-install-recommends libkrb5support0 libpq5 libc6 libgnutls30 libgssapi-krb5-2 libk5crypto3 libkrb5-3 libssl1.1 openssl
#9 0.078 
#9 0.078 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#9 0.078 
#9 0.647 Get:1 http://deb.debian.org/debian bullseye/main amd64 libc6 amd64 2.31-13+deb11u8 [2823 kB]
#9 0.708 Get:2 http://deb.debian.org/debian bullseye/main amd64 libgnutls30 amd64 3.7.1-5+deb11u4 [1340 kB]
#9 0.714 Get:3 http://deb.debian.org/debian bullseye/main amd64 libgssapi-krb5-2 amd64 1.18.3-6+deb11u4 [165 kB]
#9 0.715 Get:4 http://deb.debian.org/debian bullseye/main amd64 libk5crypto3 amd64 1.18.3-6+deb11u4 [114 kB]
#9 0.716 Get:5 http://deb.debian.org/debian bullseye/main amd64 libkrb5-3 amd64 1.18.3-6+deb11u4 [363 kB]
#9 0.718 Get:6 http://deb.debian.org/debian bullseye/main amd64 libkrb5support0 amd64 1.18.3-6+deb11u4 [65.6 kB]
#9 0.719 Get:7 http://deb.debian.org/debian bullseye/main amd64 libssl1.1 amd64 1.1.1w-0+deb11u1 [1566 kB]
#9 0.726 Get:8 http://deb.debian.org/debian bullseye/main amd64 openssl amd64 1.1.1w-0+deb11u1 [859 kB]
#9 0.730 Get:9 http://deb.debian.org/debian bullseye/main amd64 libpq5 amd64 13.13-0+deb11u1 [182 kB]
#9 0.735 Fetched 7477 kB in 0s (65.5 MB/s)
#9 0.735 W: Download is performed unsandboxed as root as file '/copa-downloads/libc6_2.31-13+deb11u8_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
#9 DONE 0.8s

#10 mkdir /copa-out
#10 DONE 0.0s

#11 mkdir /var/lib/dpkg/status.d
#11 DONE 0.0s

#12 find . -name *.deb -exec sh -c dpkg-deb -f {} > /copa-out/{}.fields ;
#12 DONE 0.1s

#13 find . -name *.fields -exec sh -c grep "^Package:\|^Version:" {} >> results.manifest ;
#13 DONE 0.1s

#14 diff (find . -name *.deb -exec sh -c dpkg-deb -f {} > /copa-out/{}.fields ;) -> (find . -name *.fields -exec sh -c grep "^Package:\|^Version:" {} >> results.manifest ;)
#14 DONE 0.0s
time="2024-02-14T10:56:56Z" level=info msg="Validated package libkrb5support0 version 1.18.3-6+deb11u4 meets requested version 1.18.3-6+deb11u4"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libpq5 version 13.13-0+deb11u1 meets requested version 13.13-0+deb11u1"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libc6 version 2.31-13+deb11u8 meets requested version 2.31-13+deb11u7"
time="2024-02-14T10:56:56Z" level=error msg="downloaded package libgnutls30 version 3.7.1-5+deb11u4 lower than required 3.7.1-5+deb11u5 for update"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libgssapi-krb5-2 version 1.18.3-6+deb11u4 meets requested version 1.18.3-6+deb11u4"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libk5crypto3 version 1.18.3-6+deb11u4 meets requested version 1.18.3-6+deb11u4"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libkrb5-3 version 1.18.3-6+deb11u4 meets requested version 1.18.3-6+deb11u4"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libssl1.1 version 1.1.1w-0+deb11u1 meets requested version 1.1.1v-0~deb11u1"
time="2024-02-14T10:56:56Z" level=info msg="Validated package openssl version 1.1.1w-0+deb11u1 meets requested version 1.1.1v-0~deb11u1"
time="2024-02-14T10:56:56Z" level=debug msg="stopping session"

#14 diff (find . -name *.deb -exec sh -c dpkg-deb -f {} > /copa-out/{}.fields ;) -> (find . -name *.fields -exec sh -c grep "^Package:\|^Version:" {} >> results.manifest ;)
#14 diffing 0.0s done
#14 DONE 0.0s
time="2024-02-14T11:01:45Z" level=error msg="patch exceeded timeout 5m0s"
Error: patch exceeded timeout 5m0s

The build completes if I provide --ignore-errors flag.

Are you willing to submit PRs to contribute to this bug fix?

  • Yes, I am willing to implement it.
@mhrabovcin mhrabovcin added the bug Something isn't working label Feb 14, 2024
@sozercan sozercan mentioned this issue Feb 15, 2024
Closed
1 task
@sozercan
Copy link
Member

sozercan commented Feb 15, 2024 •
edited
Loading

Thanks for the report! This seems like a debian db bug since there's no libgnutls30 3.7.1-5+deb11u5 https://salsa.debian.org/security-tracker-team/security-tracker/-/blob/master/data/CVE/list?ref_type=heads#L5744
https://packages.debian.org/bullseye/libgnutls30 latest is 3.7.1-5+deb11u4 as of now
Mitigation is to set up a policy to ignore CVE-2024-0567 or use --ignore-errors for now

This should not lock copa though. That is a regression from v0.5.0 (guessing from #381)
@ashnamehrotra can you ptal at this?

@mhrabovcin
Copy link
Author

Thanks for providing the workaround. I can confirm that --ignore-errors flag works for given image.

@ashnamehrotra ashnamehrotra mentioned this issue Feb 16, 2024
Merged
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Copacetic Workboard Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ashnamehrotra ashnamehrotra

Labels
bug Something isn't working
Projects
Copacetic Workboard
Status: ✅ Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Copa hangs when InstallUpdates fails ashnamehrotra/copacetic
3 participants
@mhrabovcin @sozercan @ashnamehrotra