Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modernize Windows Modern and Legacy Jenkins CI #4767

Closed
wants to merge 7 commits into from
Closed

Modernize Windows Modern and Legacy Jenkins CI #4767

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8cbb049
Revert "Remove Dev12 CI Builds"
dilijev Mar 1, 2018
bb059ae
Update script logic for forcing detection of a specific msbuild version.
dilijev Mar 1, 2018
b35e066
Update Windows build matrix to Server 2016 + Dev15 / Legacy = Server …
dilijev Mar 1, 2018
484f177
Fix error in groovy script.
dilijev Mar 1, 2018
6a529fb
Remove msbuild version 12 detection logic.
dilijev Mar 27, 2018
d382ecc
Propagate defaultMachineTag throughout netci.groovy
dilijev Mar 27, 2018
22cc5d3
Refactor and update platform flag for testExtra param
dilijev Mar 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions Build/scripts/add_msbuild_path.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
:: add_msbuild_path.cmd
::
:: Locate msbuild.exe and add it to the PATH
set USE_MSBUILD_12=%1
set FORCE_MSBUILD_VERSION=%1

if "%USE_MSBUILD_12%" == "True" (
echo Skipping Dev15 and trying Dev12...
goto :LABEL_USE_MSBUILD_12
if "%FORCE_MSBUILD_VERSION%" == "msbuild14" (
echo Skipping Dev15 and trying Dev14...
goto :LABEL_USE_MSBUILD_14
)

where /q msbuild.exe
Expand All @@ -20,8 +20,10 @@ if "%ERRORLEVEL%" == "0" (

REM Try Dev15 first

set MSBUILD_VERSION=15.0
echo Trying to locate Dev15...

:LABEL_USE_MSBUILD_15
set MSBUILD_VERSION=15.0
set "MSBUILD_PATH=%ProgramFiles(x86)%\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin"

if not exist "%MSBUILD_PATH%\msbuild.exe" (
Expand All @@ -40,8 +42,9 @@ if exist "%MSBUILD_PATH%\msbuild.exe" (
goto :MSBuildFound
)

echo Dev15 not found, trying Dev14...
echo Dev15 not found, trying to locate Dev14...

:LABEL_USE_MSBUILD_14
set MSBUILD_VERSION=14.0
set "MSBUILD_PATH=%ProgramFiles%\msbuild\%MSBUILD_VERSION%\Bin\x86"

Expand All @@ -57,7 +60,7 @@ if exist "%MSBUILD_PATH%\msbuild.exe" (
goto :MSBuildFound
)

echo Dev14 not found, trying Dev12...
echo Dev14 not found, trying to locate Dev12...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we still bother trying to find dev12?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.


:LABEL_USE_MSBUILD_12
set MSBUILD_VERSION=12.0
Expand Down
47 changes: 34 additions & 13 deletions netci.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,28 @@ def msbuildTypeMap = [

// convert `machine` parameter to OS component of PR task name
def machineTypeToOSTagMap = [
'Windows_NT': 'Windows', // Windows Server 2012 R2, equivalent to Windows 8.1 (aka Blue)
'Windows 7': 'Windows 7', // Windows Server 2008 R2 ~= Windows 7
'Windows_NT': 'Windows', // 'latest-or-auto' -> Windows Server 2012 R2 ~= Windows 8.1 aka Blue
// 'latest-dev15-5' -> Windows Server 2016 ~= Windows 10 RS2/RS3, with Dev15.5
'Ubuntu16.04': 'Ubuntu',
'OSX': 'OSX'
]

def defaultMachineTag = 'latest-or-auto'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: defaultWindowsMachineTag


def legacyWindowsMachine = 'Windows 7'
def legacyWindowsMachineTag = defaultMachineTag

def latestWindowsMachine = 'Windows_NT'
def latestWindowsMachineTag = 'latest-dev15-5'

def dailyRegex = 'dailies'

// ---------------
// HELPER CLOSURES
// ---------------

def CreateBuildTask = { isPR, buildArch, buildType, machine, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup ->
def CreateBuildTask = { isPR, buildArch, buildType, machine, machineTag, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup ->
if (excludeConfigIf && excludeConfigIf(isPR, buildArch, buildType)) {
return // early exit: we don't want to create a job for this configuration
}
Expand Down Expand Up @@ -75,7 +85,7 @@ def CreateBuildTask = { isPR, buildArch, buildType, machine, configTag, buildExt
false, // doNotFailIfNothingArchived=false ~= failIfNothingArchived
false) // archiveOnlyIfSuccessful=false ~= archiveAlways

Utilities.setMachineAffinity(newJob, machine, 'latest-or-auto')
Utilities.setMachineAffinity(newJob, machine, machineTag ?: defaultMachineTag)
Utilities.standardJobSetup(newJob, project, isPR, "*/${branch}")

if (nonDefaultTaskSetup == null) {
Expand All @@ -95,11 +105,11 @@ def CreateBuildTask = { isPR, buildArch, buildType, machine, configTag, buildExt
}
}

def CreateBuildTasks = { machine, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup ->
def CreateBuildTasks = { machine, machineTag, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup ->
[true, false].each { isPR ->
['x86', 'x64', 'arm'].each { buildArch ->
['debug', 'test', 'release'].each { buildType ->
CreateBuildTask(isPR, buildArch, buildType, machine, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup)
CreateBuildTask(isPR, buildArch, buildType, machine, machineTag, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup)
}
}
}
Expand Down Expand Up @@ -221,34 +231,45 @@ def CreateStyleCheckTasks = { taskString, taskName, checkName ->
// INNER LOOP TASKS
// ----------------

CreateBuildTasks('Windows_NT', null, null, "-winBlue", true, null, null)
CreateBuildTasks(latestWindowsMachine, latestWindowsMachineTag, null, null, "-winBlue", true, null, null)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we are supposed to be passing "-winBlue" for testExtra anymore. What does that flag even do? Enable WinBlue tests? Is the script so old that they weren't originally enabled by default?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this is passed in a bunch of different places and should be cleaned up. Maybe make the jenkins script pass the appropriate flag for wherever its running so we don't have to specify it here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think -winBlue might enable "not Windows 7" tests which should probably continue running. Could be wrong but seems like we should just remove the winBlue tags and disable some tests on Windows 7.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not mistaken there are significant platform differences across Win7 (Server 2008 R2) Win8 (Server 2012) WinBlue (Server 2012 R2), and Win10 -- and we have test platform configurations and test tags reflecting all of these. Worth an audit on the test tags as a separate work item, but whether or not we currently use this, our tests still "depend" on this on the appropriate platform.

However, I'll make sure that the "testExtra" accurately reflects that platform that each leg will actually run on. If modern = Win10 (Server 2016) then "-winBlue" is incorrect.


// Add some additional daily configs to trigger per-PR as a quality gate:
// x64_debug Slow Tests
CreateBuildTask(true, 'x64', 'debug',
'Windows_NT', 'ci_slow', null, '-winBlue -includeSlow', false, null, null)
latestWindowsMachine, latestWindowsMachineTag, 'ci_slow', null, '-winBlue -includeSlow', false, null, null)
// x64_debug DisableJIT
CreateBuildTask(true, 'x64', 'debug',
'Windows_NT', 'ci_disablejit', '"/p:BuildJIT=false"', '-winBlue -disablejit', false, null, null)
latestWindowsMachine, latestWindowsMachineTag, 'ci_disablejit', '"/p:BuildJIT=false"', '-winBlue -disablejit', false, null, null)
// x64_debug Lite
CreateBuildTask(true, 'x64', 'debug',
'Windows_NT', 'ci_lite', '"/p:BuildLite=true"', '-winBlue -lite', false, null, null)
latestWindowsMachine, latestWindowsMachineTag, 'ci_lite', '"/p:BuildLite=true"', '-winBlue -lite', false, null, null)
// x64_debug Legacy
CreateBuildTask(true, 'x64', 'debug',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So ci_lite and ci_disablejit don't get rolling builds. Is that intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need for rolling builds since they're intended to be daily builds. The PR-only tasks for these daily build configs were added as a minimal sanity check. This is by design. We can revisit this (or the general distinction of daily|rolling) as a separate discussion and work item if you think that's worthwhile.

legacyWindowsMachine, legacyWindowsMachineTag, 'ci_legacy', 'msbuild14', '-win7 -includeSlow', false, null, null)

// -----------------
// DAILY BUILD TASKS
// -----------------

if (!branch.endsWith('-ci')) {
// build and test on the usual configuration (VS 2015) with -includeSlow
CreateBuildTasks('Windows_NT', 'daily_slow', null, '-winBlue -includeSlow', false,
// build and test on the legacy configuration (Windows 7 + VS 2015 (Dev14))
CreateBuildTasks(legacyWindowsMachine, legacyWindowsMachineTag, 'daily_legacy', 'msbuild14', '-win7 -includeSlow', false,
/* excludeConfigIf */ { isPR, buildArch, buildType -> (buildArch == 'arm') },
/* nonDefaultTaskSetup */ { newJob, isPR, config ->
DailyBuildTaskSetup(newJob, isPR,
"Windows 7 ${config}",
'legacy\\s+tests')})

// build and test on the latest configuration (Server 2016 + VS 2017 (Dev15)) with -includeSlow
CreateBuildTasks(latestWindowsMachine, latestWindowsMachineTag, 'daily_slow', null, '-winBlue -includeSlow', false,
/* excludeConfigIf */ null,
/* nonDefaultTaskSetup */ { newJob, isPR, config ->
DailyBuildTaskSetup(newJob, isPR,
"Windows ${config}",
'slow\\s+tests')})

// build and test on the usual configuration (VS 2015) with JIT disabled
CreateBuildTasks('Windows_NT', 'daily_disablejit', '"/p:BuildJIT=false"', '-winBlue -disablejit', true,
// build and test on the latest configuration (Server 2016 + VS 2017 (Dev15)) with JIT disabled
CreateBuildTasks(latestWindowsMachine, latestWindowsMachineTag, 'daily_disablejit', '"/p:BuildJIT=false"', '-winBlue -disablejit', true,
/* excludeConfigIf */ null,
/* nonDefaultTaskSetup */ { newJob, isPR, config ->
DailyBuildTaskSetup(newJob, isPR,
Expand Down
8 changes: 4 additions & 4 deletions test/jenkins.build.init.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ if not "%JENKINS_BUILD%" == "True" (
set REPO_ROOT=%~dp0\..

set JENKINS_BUILD_ARGS=
set JENKINS_USE_MSBUILD_12=
set JENKINS_FORCE_MSBUILD_VERSION=
:ContinueArgParse
if not [%1]==[] (
if [%1]==[msbuild12] (
set JENKINS_USE_MSBUILD_12=True
if [%1]==[msbuild14] (
set JENKINS_FORCE_MSBUILD_VERSION=%1
goto :ContinueArgParseEnd
)

Expand All @@ -36,4 +36,4 @@ if not [%1]==[] (
:: Set up msbuild.exe
:: ========================================

call %REPO_ROOT%\Build\scripts\add_msbuild_path.cmd %JENKINS_USE_MSBUILD_12%
call %REPO_ROOT%\Build\scripts\add_msbuild_path.cmd %JENKINS_FORCE_MSBUILD_VERSION%