macOS 14 - 20250225.952 performance degradation

[arthurkuhn]

Description

We have observed performance degradation for iOS test execution on macOS 14 agents, more specifically since the rollout of version 20250225.952 (#11671). It is now reproing consistently since Feb 28th PST. There has been no significant changes to our setup in > 1 week.

Noted similar issue tracked in (#11509). Creating a new issue since latest version is affected

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• Windows Server 2019
• Windows Server 2022
• Windows Server 2025

Image version and build link

Failing

Image: macos-14-arm64
Version: 20250225.952
Included Software: https://github.com/actions/runner-images/blob/macos-14-arm64/20250225.952/images/macos/macos-14-arm64-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-14-arm64%2F20250225.952

Failing run link

Note performance degradation causing timeout to be hit

Passing

Image: macos-14-arm64
Version: 20250120.774
Included Software: https://github.com/actions/runner-images/blob/macos-14-arm64/20250120.774/images/macos/macos-14-arm64-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-14-arm64%2F20250120.774

Passing run link

Is it regression?

Yes

Expected behavior

No performance regression

Actual behavior

Performance Degradation

Repro steps

Use latest macOS-14 image
Notice runs are slower

[ijunaidm1]

Hi @arthurkuhn , We are looking into this issue.

[ijunaidm1]

@arthurkuhn - We have successfully rolled back the macOS 14 ARM image from version 20250225.952 to 20250120.774. Please try rerunning your tests and let us know if you still continue to face issues.

[arthurkuhn]

Hi @ijunaidm1, thanks for the quick action, can confirm the issue is gone with rollback!

[acc0613]

We're also experiencing performance issues with macos-14-arm64
Image version: 20250304.1018

Xcode build and test tasks are taking a lot of time (twice/thrice slower)

[erik-bershel]

Heads up! 👋

Current summary:

• starting from the end of January, the simulators installed during the image build (Xcode 16.0+ compatible) began to consume more resources, which leads to longer execution of related tasks on standard runners, which in turn leads to problems with previously set timeouts for some users, mainly affected UI/UX tests on standard runners;
• according to some users, the old workaround using yeetd still helps in eliminating the problem; example;
• our investigation showed that the iOS 18.4 simulator when used with Xcode 16.3 Beta on macOS-15 works noticeably faster than previous versions; images with the Xcode 16.3 Beta is in rollout now;
• Larger Runners are significantly less susceptible to the problem due to the larger amount of computing resources

[acc0613]

We're also experiencing performance issues with macos-14-arm64 Image version: 20250304.1018

Xcode build and test tasks are taking a lot of time (twice/thrice slower)

Tests succeed and execution time was faster when we used Xcode 16.2 and simulator version 18.2

[ijunaidm1]

Hi @arthurkuhn @acc0613 - Currently we had a new roll out image version 20250317.1117 for macOS 14 arm64 image and this week we are rolling out new image version with 20250324.1158 version, kindly please test your runs with the new image versions and confirm us if the issue for timeout is fixed or not.

[ijunaidm1]

Hi @arthurkuhn @acc0613 - Can you please re-run the iOS tests on macOS14-arm64 image with the latest image version 20250331.1204 and update us if your tests are still slower?

[ijunaidm1]

Hi @arthurkuhn @acc0613 - Kindly please update us after re-running the iOS tests with the latest image version.