dotnet test on .NET Framework test suite requires runtime identifier?

[thomhurst]

Is this anything to do with testing platform?

I've just added .NET Standard support to TUnit, and if I do a normal dotnet test it doesn't run unless I pass in a runtime identifier too.

Original issue is here: thomhurst/TUnit#1465

dotnet test:

PS C:\Users\Tom\Downloads\repro1465\repro1465> dotnet test
Restore complete (1.7s)
  repro1465 failed with 1 error(s) (0.4s)
    C:\Program Files\dotnet\sdk\9.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1047: Assets file 'C:\Users\Tom\Downloads\repro1465\repro1465\obj\project.assets.json' doesn't have a target for 'net472/win-x86'. Ensure that restore has run and that you have included 'net472' in the TargetFrameworks for your project. You may also need to include 'win-x86' in your project's RuntimeIdentifiers.

dotnet test -r win-x86:

PS C:\Users\Tom\Downloads\repro1465\repro1465> dotnet test -r win-x86
Restore complete (0.8s)
  repro1465 succeeded (7.2s) → bin\Debug\net472\win-x86\repro1465.exe
  repro1465 test succeeded (1.4s)

Test summary: total: 1, failed: 0, succeeded: 1, skipped: 0, duration: 1.3s

[Evangelink]

Looking at that issue now

[Evangelink]

@thomhurst could you please confirm the following behaviors:

1. From @AArnott sample, doing dotnet build fails with the same issue
2. Building from VS and then running dotnet test --no-build results in green run of tests

It would seems to indicate this is a msbuild/SDK issue on building (which we call when doing dotnet test).

@baronfel @JanKrivanek @YuliiaKovalova is it something you are aware of?

[baronfel]

Got a binlog of what happens when you call MSBuild @Evangelink?

[Evangelink]

Here you go: msbuild.binlog.txt

[thomhurst]

You're right. It happens on the build. dotnet build causes it too:

PS C:\Users\Tom\Downloads\repro1465\repro1465> dotnet build
Restore complete (0.9s)
  repro1465 failed with 1 error(s) (0.4s)
    C:\Program Files\dotnet\sdk\9.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1047: Assets file 'C:\Users\Tom\Downloads\repro1465\repro1465\obj\project.assets.json' doesn't have a target for 'net472/win-x86'. Ensure that restore has run and that you have included 'net472' in the TargetFrameworks for your project. You may also need to include 'win-x86' in your project's RuntimeIdentifiers.

Build failed with 1 error(s) in 6.4s

I managed to build in VS and then run on the CLI with --no-build like you suggested and that works fine:

PS C:\Users\Tom\Downloads\repro1465\repro1465> dotnet test --no-build
  repro1465 test succeeded (2.4s)

Test summary: total: 1, failed: 0, succeeded: 1, skipped: 0, duration: 1.7s
Build succeeded in 3.0s

[thomhurst]

I'm not well versed in the SDK and how msbuild does stuff, but is the problem that the ResolvePackageAssets task is using a win-x86 runtime identifier parameter?

[JanKrivanek]

Unfortunately it's not easily trackable from the binlog where that inital value comes from

@YuliiaKovalova - this is another example where dotnet/msbuild#11106 will help a ton!

[JanKrivanek]

But it seems to go from here:

https://github.com/dotnet/sdk/blob/094238ce2d4f69fbfdf0fdf862ef714b3dd027ea/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.RuntimeIdentifierInference.targets#L63

While not being used during Restore.

@marcpopMSFT - could you suggest who should look at this?

[marcpopMSFT]

This is specific to TUNIT correct? From the binlog, it appears that HasRuntimeOutput is true for the build but not for the restore which is why a RID is getting set for the build but not project assets json gets created during the restore. This is happening because OutputType is ending up as an exe in the build but not during the restore.

The restore downloads the tunit package which ends up setting the OutputType to exe after the restore phase in TUNIT.Engine.props.

@rainersigwald @dsplaisted any suggestions how tunit can work around this? Do we need a property they can disable the implicit RID inference for netfx apps? They could set HasRuntimeOutput to false but I don't know what other side effects that'll have.

[Evangelink]

@thomhurst I confirm that setting <OutputType>Exe</OutputType> directly in the project fixes the issue. So it's neither a dotnet test nor a MTP issue which is good.

[thomhurst]

@thomhurst I confirm that setting <OutputType>Exe</OutputType> directly in the project fixes the issue. So it's neither a dotnet test nor a MTP issue which is good.

Ah that's good!
I've got a buildTransitive props file in my NuGet package that should be setting that though: https://nuget.info/packages/TUnit.Engine/0.6.52

But it looks like it's in an empty nested folder...

Not sure why as it should be pulling through the TargetFramework

<PackagePath>buildTransitive/$(TargetFramework)/</PackagePath>

[thomhurst]

Anyway - This is a me problem. Sorry for the red herring. Thanks for the help everyone!

[baronfel]

@thomhurst just confirming - that structure would 100% interfere with the auto-loading behaviors of NuGet. Check for double-slashes in the PackagePath for the items you're bundling?

[rainersigwald]

The restore downloads the tunit package which ends up setting the OutputType to exe after the restore phase in TUNIT.Engine.props.

This is the core of it--I think the other test packages require that the project itself set OutputType. If it's set in the import, it will be set for "real" post-restore builds, but not the restore itself.

[dsplaisted]

Yes, props and targets files in packages should not be setting any properties which would modify the input to Restore, which includes the OutputType.

[Evangelink]

The restore downloads the tunit package which ends up setting the OutputType to exe after the restore phase in TUNIT.Engine.props.

This is the core of it--I think the other test packages require that the project itself set OutputType. If it's set in the import, it will be set for "real" post-restore builds, but not the restore itself.

One reason why custom MSBuild SDKs are good. When using it, we can actually set the output type in the props and everything works fine while for a normal package we have this error.

[thomhurst]

Yes, props and targets files in packages should not be setting any properties which would modify the input to Restore, which includes the OutputType.

Weird. This has been in TUnit for ages and doesn't seem to affect .NET (Core) builds?

Only for .NET Framework has it not worked.

I'd really like to be able to abstract away all the properties from users to simplify things for them.