Cannot run unit tests in a project.json style project that targets multiple runtimes

[jeremydmiller]

If you want, you can pull down the StructureMap solution at http://github.com/structuremap/structuremap to see what I mean here. When I try to run any test - or all of the tests - with TD.Net I get:

StructureMap.Testing.dll doesn't exist.
Please ensure that the project has been compiled.

When you look at the file system, dotnet is compiling it to: bin/Net451, but it drops all of its dependencies into a platform specific folder underneath that, "win7-x64" in my case.

[gmartinezsan]

Hitting this too, with the following error:
System.Exception: Couldn't find type with name 'DataTypeTests'
at MutantDesign.Xml.Documentation.MemberInfoUtilities.FindMemberInfo(Assembly assembly, String cref)
at TestDriven.Framework.TestRunnerTarget.GetTestRunnerTarget(String assemblyFile, String cref)
at TestDriven.NetCore.TestRunnerFactory.Create(ITestListener testListener, String assemblyFile, String cref, TestRunnerTarget& target)
at TestDriven.NetCore.Program.run(String assemblyFile, String cref)
at TestDriven.NetCore.Program.Remote.Run(String assemblyFile, String cref)

[jcansdale]

@jeremydmiller, @gmartinezsan,

I'm just about to release a new version (release notes always seem to take forever 😉).

Could you try again with this build?
TestDriven.NET-4.0.3425_Extraterrestrial_Beta2.zip

The notes will look something like this:
https://github.com/jcansdale/TestDriven.Net-Issues/blob/master/ReleaseNotes.md

This version appears to work with @jeremydmiller's StructureMap.Testing project.

@gmartinezsan, I don't know if it will make a difference for your project. If it doesn't, do you have some source I could try it with?

[gmartinezsan]

@jcansdale I'll give it a try and let you know.
Thanks a lot!

[jcansdale]

@gmartinezsan,

I've fixed an issue that is possibly related to this:
#63

Could you give this updated build a spin and let me know how you get on?
TestDriven.NET-4.0.3428_Extraterrestrial_Beta2.zip

[gmartinezsan]

hi @jcansdale
Still getting an exception with the extraterrestrial version when trying to run one of our tests. Any suggestions ?

System.Exception: Couldn't find type with name 'DataTypeTests'
at MutantDesign.Xml.Documentation.MemberInfoUtilities.FindMemberInfo(Assembly assembly, String cref)
at TestDriven.Framework.TestRunnerTarget.GetTestRunnerTarget(String assemblyFile, String cref)
at TestDriven.NetCore.TestRunnerFactory.Create(ITestListener testListener, String assemblyFile, String cref, String testRunnerName, TestRunnerTarget& target)
at TestDriven.NetCore.Program.run(String assemblyFile, String cref, String testRunnerName)
at TestDriven.NetCore.Program.Main(String[] args)

[jcansdale]

This is a strange one! Does it work with other test classes in this assembly? Is the class public?

Could you strip down the project so there is only this class and any sensitive code has been removed? Having a repro would really help.

[gmartinezsan]

I saw what the problem is. Is something very simple.

My solution supports .net451 and netcoreapp 1.0. And the issue I think is related to the selected framework in the code editor. There is a point where the framework is not valid, I guess, because I cannot select any in the code editor of Visual Studio.

If I have selected in the code editor the 4.5.1 framework and I run a different one from the Test Driven context menu I got the error I mentioned. If in both cases I use the same framework, everything work as expected.
So I would say this build works fine with my solution.
I will be using it more and if I found more details I'll let you know. Thank you a lot for your quick response!

[jcansdale]

I'm glad you found what the issue is!

Any chance you could share the source file with DataTypeTests in? (or a trimmed down summary) I'd be interested to see exactly what's going on.

[gmartinezsan]

Sure.. actually I think you can reproduce it with any project that targets multiple frameworks. Anyways I will create one and see what are the steps to reproduce it and will share it here. So you can check what the problem is.

[gmartinezsan]

@jcansdale I'm back at checking the unit tests suite of my project, Now I'm getting this error when trying to debug a unit test from any of the classes. Although running them works fine.
My project targets .net 4.5.1 and core.

Is this a known issue?

I'm using latest beta for testdriven.net

System.MethodAccessException: Attempt to access method Xunit.FactAttribute..ctor() in violation of security transparency rules failed.
at System.RuntimeMethodHandle.CheckLinktimeDemands(IRuntimeMethodInfo method, RuntimeModule module, Boolean isDecoratedTargetSecurityTransparent)
at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
at System.Attribute.GetCustomAttributes(MemberInfo element, Boolean inherit)
at System.Reflection.CustomAttributeExtensions.GetCustomAttributes(MemberInfo element)
at TestDriven.NetCore.TestRunnerFactory.hasCustomAttributes(MemberInfo member)
at TestDriven.NetCore.TestRunnerFactory.useAdHoc(MemberInfo member)
at TestDriven.NetCore.TestRunnerFactory.Create(ITestListener testListener, String assemblyFile, String cref, String testRunnerName, TestRunnerTarget& target)
at TestDriven.NetCore.Program.run(String assemblyFile, String cref, String testRunnerName)
at TestDriven.NetCore.Program.Main(String[] args)

[jeremydmiller]

@jcansdale I think this one is fixed, or at least it is for me in later builds.

[jcansdale]

@gmartinezsan,

System.MethodAccessException: Attempt to access method Xunit.FactAttribute..ctor()
     in violation of security transparency rules failed.

This looks looks like a weird one, especially since it only happens when debugging! I'm able to debug a simple xUnit test just fine. I wonder what might be different about your tests? Does this happen for all of your projects or are there any where debugging works?

@jeremydmiller,

I'm pleased to hear your issue seems to be sorted! 😄

[gmartinezsan]

@jcansdale
Thank you for your attention Jamie, the solution I'm working on is a class library. As mentioned here
http://stackoverflow.com/questions/36118567/how-to-switch-between-target-frameworks-for-net-core-projects-in-visual-studio
looks like there are some prerequisites to be switching between frameworks inside the code editor of Visual Studio. This seems to have still an issue. I thought that having the project of the unit tests as startup project would solved it but it doesn't change anything.
If the project targets only one framework everything works fine.
Please let me know if you need further details.

[jcansdale]

Switching views seems to work for me on console and class library projects. It is however only a view and Visual Studio always executes using the first framework defined in the "frameworks" section. I can't even detect which view is selected (if I wanted to make this work differently in TestDriven.Net).

Could you upload a simple repro here? I haven't been able to trigger the exception mentioned.

Thanks, Jamie.

[gmartinezsan]

@jcansdale Finally took the time to create this solution and was able to have the repro steps.

https://github.com/gmartinezsan/ConsoleAppForTrace
Feel free to let me know if you need further details.

Steps to reproduce:

• Open the solution in VS 2015
• Go to the Mytestclass.cs file
• Add a breakpoint inside the FirstTest method
• Select the 4.5.1 framework in the combo box of the code editor.
• Right clic on the context menu in the FirstTest method (same as when starting the debugger).
• Select "Test With" menu and then select .NET Framework
• On the output window you will see the following error:

------ Test started: Assembly: ConsoleTests.dll ------

System.Exception: Couldn't find declaring type with name 'ConsoleTests.Mytestclass'
at MutantDesign.Xml.Documentation.MemberInfoUtilities.FindMemberInfo(Assembly assembly, String cref)
at TestDriven.Framework.TestRunnerTarget.GetTestRunnerTarget(String assemblyFile, String cref)
at TestDriven.NetCore.TestRunnerFactory.Create(ITestListener testListener, String assemblyFile, String cref, String testRunnerName, TestRunnerTarget& target)
at TestDriven.NetCore.Program.run(String assemblyFile, String cref, String testRunnerName)
at TestDriven.NetCore.Program.Remote.Run(String assemblyFile, String cref, String testRunnerName)

• If the net core app is selected instead then it does work.

When using the debugger

I only can debug the first framework of my project.json file. For example if the first framework is .net451 then that's the only framework I can debug even if I have selected the .net core app in the combo of the code editor.
It is debugging correctly the .net451.

Hope this helps!

[kzu]

FWIW, selecting the target framework is implemented now in VS2017 Preview (15.3), and it basically sets an ActiveDebugFramework MSBuild property in the .user file alongside the .csproj.

May (or may not ;)) be useful to @jcansdale ;)

[jcansdale]

@kzu Thanks for the heads up. I'm messing with the 15.3 Preview at the moment...

[jcansdale]

@kzu I've created a multi-target project, but can't find how to select the target framework. Could you give me a clue? 😄

I'm using 15.3 and my .csproj includes:
<TargetFrameworks>net462;net46;netstandard2.0</TargetFrameworks>

[kzu]

It has to be an SDK-style project, and you'd see the dropdown in the Run button:

[jcansdale]

Got it, thanks!

I'd never have thought to look in *.csproj.user.

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <ShowAllFiles>false</ShowAllFiles>
    <ActiveDebugFramework>net46</ActiveDebugFramework>
  </PropertyGroup>
</Project>

I guess that could also be used as a handy place to inject stuff into a build. I'm like to be able to right-click on a VSSDK project and have it install/execute in a different version of VS. Maybe *.csproj.user. could be the gateway. 😄