First take at p4tc automated tests #5011

vbnogueira · 2024-11-11T20:33:55Z

First take at implementing the automated testing code for tc backend. In a nutshell, what it's doing is expanding on the already existing run-tc-test.py script. The expansion consists of:

Compiling the generated C files to eBPF bytecode
Extracting a P4TC kernel image from github
Compiling P4TC's version of iproute2
Spawning a VM using virtme to boot the P4TC kernel
Executing the template script generated by the compiler
Loading the generated eBPF parser and control blocks binaries using a TC P4 filter
Loading the runtime rules specified in the samples directory (*.rules)
Parsing an STF in the samples directory detailing what packets to send/expect
Sending the packets using scapy
Verifying that the sent packets (and eventual received packets from the p4tc pipeline) and correct according to the STF file
Opening it as a draft PR because we still need to see how this works with github actions
Locally it's doing fine, however we never know

I created a separate directory for the tests that involve STFs testdata/p4tc_samples_stf/ to make it cleaner
However that might've been overkill, I can undo that if reviewers think it's best

fruffy · 2024-11-11T22:31:24Z

The installation might be missing gcc-multilib.

I recommend adding a TC-specific section here: https://github.com/p4lang/p4c/blob/main/tools/ci-build.sh#L170

vbnogueira · 2024-11-12T13:47:28Z

The installation might be missing gcc-multilib.

I recommend adding a TC-specific section here: https://github.com/p4lang/p4c/blob/main/tools/ci-build.sh#L170

Thank you, will do

fruffy

Some initial comments

.github/workflows/ci-test-mac.yml

backends/tc/run-tc-test.py

backends/tc/runtime/build-simple-p4

backends/tc/runtime/cleanup

backends/tc/runtime/clone-iproute2

tools/install_fedora_deps.sh

fruffy

Couple nits left.

.github/workflows/ci-test-mac.yml

fruffy · 2024-12-12T10:57:38Z

tools/install_fedora_deps.sh

@@ -26,7 +26,7 @@ sudo dnf install -y -q \
    boost-test \
    boost-thread \
    ccache \
-    clang \
+    clang-15 \


Any reason why clang-15 is required?

It's because ubuntu and fedora's clang versions are too old and generated eBPF code that was rejected by the eBPF verifier. clang-15 was the oldest version that worked with the kernel version we are using for the tests

Does the TC back end check the clang version? It should either throw an error or disable itself if clang-15 is not available.

I pushed a change for this yesterday, which checks whether we have at least clang-15 (anything above that should also work for the current p4tc kernel). However the tests are failing because they are not being able to clone p4c. Can you try to run them again? I don't think I have permission to do so.

fruffy · 2024-12-18T12:09:10Z

The code looks okay from my side, I just have final reservations on the CI process. The installation of the VM etc adds significant overhead to our CI. Maybe we should make this a separate workflow that is optional and only runs on code that touches p4tc parts or PRs tagged with p4tc.

Let me think about how to introduce this.

fruffy

Approving to unblock things. We should consider putting the TC tests in a separate (nightly) workflow that is also triggered when code in the tc folder changes or when a PR is tagged with p4tc. We have a similar workflow for the sanitizer for example (https://github.com/p4lang/p4c/blob/main/.github/workflows/ci-ubuntu-20-sanitizer-nightly.yml)

vbnogueira · 2025-01-07T12:22:05Z

Approving to unblock things. We should consider putting the TC tests in a separate (nightly) workflow that is also triggered when code in the tc folder changes or when a PR is tagged with p4tc. We have a similar workflow for the sanitizer for example (https://github.com/p4lang/p4c/blob/main/.github/workflows/ci-ubuntu-20-sanitizer-nightly.yml)

Sorry for the delay, we had a busy couple of weeks
Will get into that as soon as possible

vbnogueira · 2025-02-26T10:34:20Z

@fruffy please take another look when you have some time

fruffy · 2025-02-27T00:28:41Z

@fruffy please take another look when you have some time

Any major change from the version I approved or was this just the rebase?

vbnogueira · 2025-02-27T00:32:16Z

@fruffy please take another look when you have some time

Any major change from the version I approved or was this just the rebase?

The biggest changes are running the more heavy tests only on PRs that target P4TC, and stopping these same tests from being run in other circumstances

fruffy

@fruffy please take another look when you have some time

Any major change from the version I approved or was this just the rebase?

The biggest changes are running the more heavy tests only on PRs that target P4TC, and stopping these same tests from being run in other circumstances

Great! And it looks like it works well. Approving again.

fruffy · 2025-03-02T12:43:52Z

.github/workflows/ci-test-mac.yml

@@ -58,7 +58,7 @@ jobs:
      - name: Run tests (MacOS)
        run: |
          source ~/.bash_profile
-          ctest --output-on-failure --schedule-random -E "bpf|ubpf|testgen|smith"
+          ctest --output-on-failure --schedule-random -E "bpf|ubpf|testgen|smith|p4tc"


I think it is best to disable these tests when some conditions are not met (eg CMake detects some runner is missing or the OS is not supported). But we can do this in a separate PR.

.github/workflows/ci-ubuntu-18-nightly.yml

First take at implementing the automated testing code for tc backend. In a nutshell, what it's doing is expanding on the already existing run-tc-test.py script. The expansion consists of: - Compiling the generated C files to eBPF bytecode - Extracting a P4TC kernel image from github - Compiling P4TC's version of iproute2 - Spawning a VM using virtme to boot the P4TC kernel - Executing the template script generated by the compiler - Loading the generated eBPF parser and control blocks binaries using a TC P4 filter - Parsing an STF in the samples directory detailing what packets to send/expect and what runtime rules to load - Loading any specified runtime rules - Sending the packets using scapy - Verifying that the sent packets (and eventual received packets from the p4tc pipeline) and correct according to the STF file The commands are sent to the VM using ssh through a bridge that connects the host to the VM After the test is finished both the bridge and the VM are destroyed We also added an example (arp_responder) to exercise the testing framework Signed-off-by: Victor Nogueira <[email protected]>

Signed-off-by: Victor Nogueira <[email protected]>

fruffy · 2025-03-03T19:35:12Z

@vbnogueira lmk when you want this merged.

vbnogueira · 2025-03-03T20:37:58Z

@vbnogueira lmk when you want this merged.

If it won't cause any issues, the sooner the better

jafingerhut · 2025-03-03T20:43:03Z

I am fine if you want to merge these right away, as is.

I will point out that the portions of these tests that use ubuntu-20.04 will start failing intermittently during March, 2025, and deterministically on April 1, 2025.

#5155

It seems reasonable to me to remove uses of Ubuntu 20.04 in a later PR, after this one is merged.

jafingerhut · 2025-03-03T20:58:02Z

I am fine if you want to merge these right away, as is.

I will point out that the portions of these tests that use ubuntu-20.04 will start failing intermittently during March, 2025, and deterministically on April 1, 2025.

#5155

It seems reasonable to me to remove uses of Ubuntu 20.04 in a later PR, after this one is merged.

I am also happy if this PR is merged first, and I will update #5156 if necessary.

jafingerhut · 2025-03-04T03:02:22Z

Going ahead and merging this, given the approval and the submitter's desire to have it merged.

Signed-off-by: Victor Nogueira <[email protected]> Signed-off-by: Vineet1101 <[email protected]>

fruffy added the p4tc Topics related to the P4-TC back end. On PRs, also triggers p4tc CI tests to run. label Nov 11, 2024

vbnogueira force-pushed the automated_p4tc_tests branch 14 times, most recently from 12ebfd2 to e5ba246 Compare November 19, 2024 17:39

vbnogueira force-pushed the automated_p4tc_tests branch 3 times, most recently from 80d0655 to c915f8e Compare November 27, 2024 17:49

vbnogueira marked this pull request as ready for review November 27, 2024 19:18

komaljai requested a review from fruffy November 28, 2024 14:09

fruffy reviewed Nov 30, 2024

View reviewed changes

vbnogueira force-pushed the automated_p4tc_tests branch 2 times, most recently from 148cd65 to 8b06b32 Compare December 10, 2024 14:21

fruffy reviewed Dec 12, 2024

View reviewed changes

vbnogueira force-pushed the automated_p4tc_tests branch from 8b06b32 to 14f48b6 Compare December 12, 2024 12:48

fruffy mentioned this pull request Dec 17, 2024

Add p4tc backend to p4testgen #5019

Open

vbnogueira force-pushed the automated_p4tc_tests branch from 14f48b6 to ac4b956 Compare December 18, 2024 02:14

fruffy approved these changes Jan 2, 2025

View reviewed changes

vbnogueira mentioned this pull request Jan 28, 2025

[P4TC] Align key components to the nearest 8-bit size #5024

Merged

vbnogueira force-pushed the automated_p4tc_tests branch 2 times, most recently from 7dc45ec to 736c84a Compare February 26, 2025 01:26

vbnogueira force-pushed the automated_p4tc_tests branch from 736c84a to ea08315 Compare February 27, 2025 19:15

fruffy approved these changes Mar 2, 2025

View reviewed changes

vbnogueira added 2 commits March 3, 2025 15:39

Add simple_l3 example

f1db84f

Signed-off-by: Victor Nogueira <[email protected]>

vbnogueira force-pushed the automated_p4tc_tests branch from ea08315 to f1db84f Compare March 3, 2025 15:55

jafingerhut added this pull request to the merge queue Mar 4, 2025

Merged via the queue into p4lang:main with commit f67ab33 Mar 4, 2025
21 checks passed

jafingerhut mentioned this pull request Mar 4, 2025

Replace most mentions of Ubuntu 20.04 with 22.04 #5156

Merged

ChrisDodd mentioned this pull request Mar 9, 2025

Build broken because of dead link (p4tc) #5171

Closed

Vineet1101 pushed a commit to Vineet1101/p4c that referenced this pull request Mar 15, 2025

First take at p4tc automated tests (p4lang#5011)

aad7971

Signed-off-by: Victor Nogueira <[email protected]> Signed-off-by: Vineet1101 <[email protected]>

Vineet1101 pushed a commit to Vineet1101/p4c that referenced this pull request Mar 15, 2025

First take at p4tc automated tests (p4lang#5011)

66f2e2f

Signed-off-by: Victor Nogueira <[email protected]> Signed-off-by: Vineet1101 <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

First take at p4tc automated tests #5011

First take at p4tc automated tests #5011

vbnogueira commented Nov 11, 2024

fruffy commented Nov 11, 2024

vbnogueira commented Nov 12, 2024

fruffy left a comment

fruffy left a comment

fruffy Dec 12, 2024

vbnogueira Dec 12, 2024

fruffy Dec 13, 2024

vbnogueira Dec 18, 2024 •

edited

Loading

fruffy commented Dec 18, 2024

fruffy left a comment

vbnogueira commented Jan 7, 2025

vbnogueira commented Feb 26, 2025

fruffy commented Feb 27, 2025

vbnogueira commented Feb 27, 2025

fruffy left a comment

fruffy Mar 2, 2025

fruffy commented Mar 3, 2025

vbnogueira commented Mar 3, 2025

jafingerhut commented Mar 3, 2025

jafingerhut commented Mar 3, 2025

jafingerhut commented Mar 4, 2025

First take at p4tc automated tests #5011

First take at p4tc automated tests #5011

Conversation

vbnogueira commented Nov 11, 2024

fruffy commented Nov 11, 2024

vbnogueira commented Nov 12, 2024

fruffy left a comment

Choose a reason for hiding this comment

fruffy left a comment

Choose a reason for hiding this comment

fruffy Dec 12, 2024

Choose a reason for hiding this comment

vbnogueira Dec 12, 2024

Choose a reason for hiding this comment

fruffy Dec 13, 2024

Choose a reason for hiding this comment

vbnogueira Dec 18, 2024 • edited Loading

Choose a reason for hiding this comment

fruffy commented Dec 18, 2024

fruffy left a comment

Choose a reason for hiding this comment

vbnogueira commented Jan 7, 2025

vbnogueira commented Feb 26, 2025

fruffy commented Feb 27, 2025

vbnogueira commented Feb 27, 2025

fruffy left a comment

Choose a reason for hiding this comment

fruffy Mar 2, 2025

Choose a reason for hiding this comment

fruffy commented Mar 3, 2025

vbnogueira commented Mar 3, 2025

jafingerhut commented Mar 3, 2025

jafingerhut commented Mar 3, 2025

jafingerhut commented Mar 4, 2025

vbnogueira Dec 18, 2024 •

edited

Loading