Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build and install swift-testing in toolchains #75515

Merged
merged 9 commits into from
Aug 10, 2024
Merged

Build and install swift-testing in toolchains #75515

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
19910c6
Add swift-testing to update-checkout
stmontgomery Jun 5, 2024
97e85a6
Build script changes to begin building and installing swift-testing i…
stmontgomery Jul 10, 2024
ff5ece1
Add swift-testing to Linux presets too
stmontgomery Jul 12, 2024
c8d7d13
Improve description of new build-script flags
stmontgomery Jul 16, 2024
155e4c8
Fix header comment
stmontgomery Jul 16, 2024
e06d38b
[swift-testing] Build tweak
rintaro Jul 24, 2024
f069aec
[Windows] Build and install swift-testing in Windows toolchains
rintaro Jul 25, 2024
6a38329
[build-preset] Add swift-testing to incrmental bots
rintaro Aug 3, 2024
e98c5ea
[build-script] Separate SwiftTestinMacros from SwiftTesting product
rintaro Aug 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions utils/build-presets.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,8 @@ sourcekit-lsp
# swift-inspect
swiftsyntax
swiftformat
swift-testing
swift-testing-macros

skstresstester

Expand All @@ -406,6 +408,8 @@ install-swiftpm
install-swift-driver
install-swiftsyntax
install-swiftformat
install-swift-testing
install-swift-testing-macros

[preset: buildbot_incremental,tools=RA,stdlib=RA,apple_silicon]
mixin-preset=buildbot_incremental,tools=RA,stdlib=RA
Expand Down Expand Up @@ -839,6 +843,8 @@ mixin-preset=
llbuild
swiftpm
swift-driver
swift-testing
swift-testing-macros
xctest
swiftdocc
swiftformat
Expand All @@ -852,6 +858,8 @@ install-llbuild
install-swiftpm
install-swift-driver
install-swiftsyntax
install-swift-testing
install-swift-testing-macros
install-xctest
install-prefix=/usr
install-sourcekit-lsp
Expand Down Expand Up @@ -1180,12 +1188,15 @@ llbuild
swiftpm
swift-driver
xctest
swift-testing
swift-testing-macros
foundation
libdispatch
swiftsyntax
swiftformat
indexstore-db
sourcekit-lsp

install-llvm
install-static-linux-config
install-swift
Expand All @@ -1196,6 +1207,8 @@ install-swiftsyntax
install-foundation
install-libdispatch
install-xctest
install-swift-testing
install-swift-testing-macros
install-swiftformat

[preset: buildbot_incremental_linux,long_test]
Expand Down Expand Up @@ -1319,6 +1332,8 @@ swift-driver
# Failing to build in CI: rdar://78408440
# swift-inspect
swiftsyntax
swift-testing
swift-testing-macros
swiftformat
playgroundsupport
indexstore-db
Expand Down Expand Up @@ -1363,6 +1378,8 @@ install-llbuild
install-swiftpm
install-swift-driver
install-swiftsyntax
install-swift-testing
install-swift-testing-macros
install-playgroundsupport
install-sourcekit-lsp
install-swiftformat
Expand Down Expand Up @@ -1686,6 +1703,8 @@ swiftpm
# swift-inspect
swift-driver
swiftsyntax
swift-testing
swift-testing-macros
swiftformat
swiftdocc

Expand All @@ -1705,6 +1724,8 @@ install-llbuild
install-swiftpm
install-swift-driver
install-swiftsyntax
install-swift-testing
install-swift-testing-macros
install-swiftdocc
install-swiftformat

Expand Down
91 changes: 78 additions & 13 deletions utils/build.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ $ArchX64 = @{
PlatformInstallRoot = "$BinaryCache\x64\Windows.platform";
SDKInstallRoot = "$BinaryCache\x64\Windows.platform\Developer\SDKs\Windows.sdk";
XCTestInstallRoot = "$BinaryCache\x64\Windows.platform\Developer\Library\XCTest-development";
SwiftTestingInstallRoot = "$BinaryCache\x64\Windows.platform\Developer\Library\Testing-development";
ToolchainInstallRoot = "$BinaryCache\x64\toolchains\$ProductVersion+Asserts";
}

Expand All @@ -230,6 +231,7 @@ $ArchX86 = @{
PlatformInstallRoot = "$BinaryCache\x86\Windows.platform";
SDKInstallRoot = "$BinaryCache\x86\Windows.platform\Developer\SDKs\Windows.sdk";
XCTestInstallRoot = "$BinaryCache\x86\Windows.platform\Developer\Library\XCTest-development";
SwiftTestingInstallRoot = "$BinaryCache\x86\Windows.platform\Developer\Library\Testing-development";
}

$ArchARM64 = @{
Expand All @@ -245,6 +247,7 @@ $ArchARM64 = @{
SDKInstallRoot = "$BinaryCache\arm64\Windows.platform\Developer\SDKs\Windows.sdk";
XCTestInstallRoot = "$BinaryCache\arm64\Windows.platform\Developer\Library\XCTest-development";
ToolchainInstallRoot = "$BinaryCache\arm64\toolchains\$ProductVersion+Asserts";
SwiftTestingInstallRoot = "$BinaryCache\arm64\Windows.platform\Developer\Library\Testing-development";
}

$AndroidARM64 = @{
Expand All @@ -258,6 +261,7 @@ $AndroidARM64 = @{
PlatformInstallRoot = "$BinaryCache\arm64\Android.platform";
SDKInstallRoot = "$BinaryCache\arm64\Android.platform\Developer\SDKs\Android.sdk";
XCTestInstallRoot = "$BinaryCache\arm64\Android.platform\Developer\Library\XCTest-development";
SwiftTestingInstallRoot = "$BinaryCache\arm64\Android.platform\Developer\Library\Testing-development";
}

$AndroidARMv7 = @{
Expand All @@ -271,6 +275,7 @@ $AndroidARMv7 = @{
PlatformInstallRoot = "$BinaryCache\armv7\Android.platform";
SDKInstallRoot = "$BinaryCache\armv7\Android.platform\Developer\SDKs\Android.sdk";
XCTestInstallRoot = "$BinaryCache\armv7\Android.platform\Developer\Library\XCTest-development";
SwiftTestingInstallRoot = "$BinaryCache\armv7\Android.platform\Developer\Library\Testing-development";
}

$AndroidX86 = @{
Expand All @@ -284,6 +289,7 @@ $AndroidX86 = @{
PlatformInstallRoot = "$BinaryCache\x86\Android.platform";
SDKInstallRoot = "$BinaryCache\x86\Android.platform\Developer\SDKs\Android.sdk";
XCTestInstallRoot = "$BinaryCache\x86\Android.platform\Developer\Library\XCTest-development";
SwiftTestingInstallRoot = "$BinaryCache\x86\Android.platform\Developer\Library\Testing-development";
}

$AndroidX64 = @{
Expand All @@ -297,6 +303,7 @@ $AndroidX64 = @{
PlatformInstallRoot = "$BinaryCache\x64\Android.platform";
SDKInstallRoot = "$BinaryCache\x64\Android.platform\Developer\SDKs\Android.sdk";
XCTestInstallRoot = "$BinaryCache\x64\Android.platform\Developer\Library\XCTest-development";
SwiftTestingInstallRoot = "$BinaryCache\x64\Android.platform\Developer\Library\Testing-development";
}

$HostArch = switch ($HostArchName) {
Expand Down Expand Up @@ -387,6 +394,7 @@ enum TargetComponent {
Dispatch
Foundation
XCTest
SwiftTesting
}

function Get-TargetProjectBinaryCache($Arch, [TargetComponent]$Project) {
Expand All @@ -413,6 +421,7 @@ enum HostComponent {
LMDB
SymbolKit
DocC
SwiftTestingMacros
}

function Get-HostProjectBinaryCache([HostComponent]$Project) {
Expand Down Expand Up @@ -1731,11 +1740,43 @@ function Build-XCTest([Platform]$Platform, $Arch, [switch]$Test = $false) {
dispatch_DIR = "$DispatchBinaryCache\cmake\modules";
Foundation_DIR = "$FoundationBinaryCache\cmake\modules";
} + $TestingDefines)
}
}

function Build-SwiftTesting([Platform]$Platform, $Arch, [switch]$Test = $false) {
$SwiftTestingBinaryCache = Get-TargetProjectBinaryCache $Arch SwiftTesting

Isolate-EnvVars {
if ($Test) {
# TODO: Test
return
} else {
$Targets = @("default")
$InstallPath = "$($Arch.SwiftTestingInstallRoot)\usr"
}

Build-CMakeProject `
-Src $SourceCache\swift-testing `
-Bin $SwiftTestingBinaryCache `
-InstallTo $InstallPath `
-Arch $Arch `
-Platform $Platform `
-UseBuiltCompilers C,CXX,Swift `
-BuildTargets $Targets `
-Defines (@{
BUILD_SHARED_LIBS = "YES";
CMAKE_BUILD_WITH_INSTALL_RPATH = "YES";
SwiftSyntax_DIR = (Get-HostProjectCMakeModules Compilers);
# FIXME: Build the plugin for the builder and specify the path.
SwiftTesting_MACRO = "NO";
})
}
}

function Write-PlatformInfoPlist($Arch) {
$PList = Join-Path -Path $Arch.PlatformInstallRoot -ChildPath "Info.plist"
Invoke-Program $python -c "import plistlib; print(str(plistlib.dumps({ 'DefaultProperties': { 'XCTEST_VERSION': 'development', 'SWIFTC_FLAGS': ['-use-ld=lld'] } }), encoding='utf-8'))" `
Invoke-Program $python -c "import plistlib; print(str(plistlib.dumps({ 'DefaultProperties': { 'XCTEST_VERSION': 'development', 'SWIFT_TESTING_VERSION': 'development', 'SWIFTC_FLAGS': ['-use-ld=lld'] } }), encoding='utf-8'))" `
-OutFile "$PList"
}
}

# Copies files installed by CMake from the arch-specific platform root,
Expand Down Expand Up @@ -2167,6 +2208,21 @@ function Build-SourceKitLSP($Arch) {
}
}

function Build-SwiftTestingMacros($Arch) {
Build-CMakeProject `
-Src $SourceCache\swift-testing\Sources\TestingMacros `
-Bin (Get-HostProjectBinaryCache SwiftTestingMacros) `
-InstallTo "$($Arch.ToolchainInstallRoot)\usr" `
-Arch $Arch `
-Platform Windows `
-UseBuiltCompilers Swift `
-SwiftSDK (Get-HostSwiftSDK) `
-BuildTargets default `
-Defines @{
SwiftSyntax_DIR = (Get-HostProjectCMakeModules Compilers);
}
}

function Install-HostToolchain() {
if ($ToBatch) { return }

Expand Down Expand Up @@ -2325,20 +2381,24 @@ if (-not $SkipBuild) {
Invoke-BuildStep Build-Dispatch Windows $Arch
Invoke-BuildStep Build-Foundation Windows $Arch
Invoke-BuildStep Build-XCTest Windows $Arch
Invoke-BuildStep Build-SwiftTesting Windows $Arch
Invoke-BuildStep Write-PlatformInfoPlist $Arch
}

foreach ($Arch in $AndroidSDKArchs) {
Invoke-BuildStep Build-ZLib Android $Arch
Invoke-BuildStep Build-XML2 Android $Arch
Invoke-BuildStep Build-CURL Android $Arch
Invoke-BuildStep Build-LLVM Android $Arch
foreach ($Arch in $AndroidSDKArchs) {
Invoke-BuildStep Build-ZLib Android $Arch
Invoke-BuildStep Build-XML2 Android $Arch
Invoke-BuildStep Build-CURL Android $Arch
Invoke-BuildStep Build-LLVM Android $Arch

# Build platform: SDK, Redist and XCTest
Invoke-BuildStep Build-Runtime Android $Arch
Invoke-BuildStep Build-Dispatch Android $Arch
Invoke-BuildStep Build-Foundation Android $Arch
Invoke-BuildStep Build-XCTest Android $Arch
}
# Build platform: SDK, Redist and XCTest
Invoke-BuildStep Build-Runtime Android $Arch
Invoke-BuildStep Build-Dispatch Android $Arch
Invoke-BuildStep Build-Foundation Android $Arch
Invoke-BuildStep Build-XCTest Android $Arch
Invoke-BuildStep Build-SwiftTesting Android $Arch
Invoke-BuildStep Write-PlatformInfoPlist $Arch
}
}

if (-not $ToBatch) {
Expand All @@ -2360,6 +2420,8 @@ if (-not $ToBatch) {
}

if (-not $SkipBuild) {
# TestingMacros can't be built before the standard library for the host as it is required for the Swift code.
Invoke-BuildStep Build-SwiftTestingMacros $HostArch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this next to the call to Build-SwiftTesting above?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't move this right next to Build-SwiftTesting call because it is for $WindowsSDKArchs not $HostArch. To what line you want to move this exactly?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, why is it being built for each architecture for the SDK? The macros run at compile time, so we should only have to build it for the host that the toolchain is being built for.

Copy link
Member Author

@rintaro rintaro Aug 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, TestingMacros is built for the host only, Testing is built for each SDK arch. That's how I implemented. Am I missing anything?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The macros should be running with the compiler (host) not the SDK environment (target). So, I'm not sure I understand why we are building it in all these cases.

Invoke-BuildStep Build-SQLite $HostArch
Invoke-BuildStep Build-System $HostArch
Invoke-BuildStep Build-ToolsSupportCore $HostArch
Expand Down Expand Up @@ -2414,6 +2476,9 @@ if (-not $IsCrossCompiling) {
if ($Test -contains "xctest") {
Build-XCTest Windows $HostArch -Test
}
if ($Test -contains "testing") {
Build-SwiftTesting Windows $HostArch -Test
}
if ($Test -contains "llbuild") { Build-LLBuild $HostArch -Test }
if ($Test -contains "swiftpm") { Test-PackageManager $HostArch }
}
Expand Down
10 changes: 10 additions & 0 deletions utils/build_swift/build_swift/driver_arguments.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -809,6 +809,16 @@ def create_argument_parser():
option(['--wasmkit'], toggle_true('build_wasmkit'),
help='build WasmKit')

option('--swift-testing', toggle_true('build_swift_testing'),
help='build Swift Testing')
option('--install-swift-testing', toggle_true('install_swift_testing'),
help='install Swift Testing')
option('--swift-testing-macros', toggle_true('build_swift_testing_macros'),
help='build Swift Testing macro plugin')
option('--install-swift-testing-macros',
toggle_true('install_swift_testing_macros'),
help='install Swift Testing macro plugin')

option('--xctest', toggle_true('build_xctest'),
help='build xctest')

Expand Down
9 changes: 9 additions & 0 deletions utils/build_swift/tests/expected_options.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@
'build_swiftpm': False,
'build_swift_driver': False,
'build_swift_libexec': True,
'build_swift_testing': False,
'build_swift_testing_macros': False,
'build_early_swift_driver': True,
'build_early_swiftsyntax': True,
'build_swiftsyntax': False,
Expand All @@ -111,6 +113,8 @@
'install_static_linux_config': False,
'install_swiftpm': False,
'install_swiftsyntax': False,
'install_swift_testing': False,
'install_swift_testing_macros': False,
'install_swift_driver': False,
'install_swiftdocc': False,
'swiftsyntax_verify_generated_files': False,
Expand Down Expand Up @@ -566,6 +570,8 @@ class BuildScriptImplOption(_BaseOption):
SetTrueOption('--swiftpm', dest='build_swiftpm'),
SetTrueOption('--swift-driver', dest='build_swift_driver'),
SetTrueOption('--swiftsyntax', dest='build_swiftsyntax'),
SetTrueOption('--swift-testing', dest='build_swift_testing'),
SetTrueOption('--swift-testing-macros', dest='build_swift_testing_macros'),
SetTrueOption('--skstresstester', dest='build_skstresstester'),
SetTrueOption('--swiftformat', dest='build_swiftformat'),
SetTrueOption('--swiftdocc', dest='build_swiftdocc'),
Expand Down Expand Up @@ -646,6 +652,9 @@ class BuildScriptImplOption(_BaseOption):
EnableOption('--install-llvm', dest='install_llvm'),
EnableOption('--install-static-linux-config', dest='install_static_linux_config'),
EnableOption('--install-swiftsyntax', dest='install_swiftsyntax'),
EnableOption('--install-swift-testing', dest='install_swift_testing'),
EnableOption('--install-swift-testing-macros',
dest='install_swift_testing_macros'),
EnableOption('--swiftsyntax-verify-generated-files',
dest='swiftsyntax_verify_generated_files'),
EnableOption('--swiftsyntax-enable-rawsyntax-validation',
Expand Down
4 changes: 4 additions & 0 deletions utils/swift_build_support/swift_build_support/build_script_invocation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -678,6 +678,10 @@ def compute_product_pipelines(self):
is_enabled=self.args.build_foundation)
builder.add_product(products.SwiftSyntax,
is_enabled=self.args.build_swiftsyntax)
builder.add_product(products.SwiftTestingMacros,
is_enabled=self.args.build_swift_testing_macros)
builder.add_product(products.SwiftTesting,
is_enabled=self.args.build_swift_testing)
builder.add_product(products.SwiftFormat,
is_enabled=self.args.build_swiftformat)
builder.add_product(products.SKStressTester,
Expand Down
2 changes: 1 addition & 1 deletion utils/swift_build_support/swift_build_support/productpipeline_list_builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def add_product(self, product_cls, is_enabled):
self.current_pipeline.append(product_cls, is_enabled)

def add_impl_product(self, product_cls, is_enabled):
"""Add a non-impl product to the current pipeline begin constructed"""
"""Add an impl product to the current pipeline begin constructed"""
assert self.current_pipeline is not None
assert self.is_current_pipeline_impl
assert product_cls.is_build_script_impl_product()
Expand Down
4 changes: 4 additions & 0 deletions utils/swift_build_support/swift_build_support/products/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
from .sourcekitlsp import SourceKitLSP
from .staticswiftlinux import StaticSwiftLinuxConfig
from .swift import Swift
from .swift_testing import SwiftTesting
from .swift_testing_macros import SwiftTestingMacros
from .swiftdocc import SwiftDocC
from .swiftdoccrender import SwiftDocCRender
from .swiftdriver import SwiftDriver
Expand Down Expand Up @@ -68,6 +70,8 @@
'SwiftInspect',
'SwiftPM',
'SwiftDriver',
'SwiftTesting',
'SwiftTestingMacros',
'EarlySwiftDriver',
'XCTest',
'SwiftSyntax',
Expand Down
Loading