Treat project references as binary references when not part of solution
Would like to request an option to treat project references as binary references if they are no part of the current solution or if they have the 'Ignore this project completely' option enabled.
The reason for this is I work on a large codebase (thousands of projects and deep dependency tree) and our build environment requires that we use ProjectReference for our references (with some projects having dozens of references).
We do have a custom tool that copies the project's dependencies (the corresponding binaries for all the project references) from our build shares. We then pass the "/p:BuildProjectReferences=false" option to msbuild so it does not actually build all those project references.
However when I try to use NCrunch on such a project I get a "Missing or ignored project reference" warning. The build then fails because the code from that project reference is missing, i.e. I get an error like this:
The type or namespace name 'MissingType' could not be found (are you missing a using directive or an assembly reference?)
Rafael Nieto commented
Thanks. I can see why this "simple" feature request is actually very complicated to implement. However as you say I don't think this is an unusual situation either. Many developers work on big codebases and it's only practical to work with a subset of the projects at a time.
For now I'll keep using my "hacks" to get around this, or even not using NCrunch in cases where the workarounds are just too much hassle :(
I do look forward to NCrunch features to better support developers working in these kinds of build environments.
Thanks for the extra details. I don't think your situation is unusual, as we do have many customers working with subset solutions spanning wider ranges of projects.
I've had a bit of a think about ways we could approach this. My best idea was to implement a setting that would allow you to specify a list of DLLs that would take the place of project references declared in a project, kind of the reverse of the 'Infer project references using assembly names' setting.
After more consideration, I realised this would actually be extremely difficult to implement reliably under .NET Core/.NET5+. This is because for runtime purposes, a project outputs more than just a DLL - we also need the .deps file that specifies its dependency data. Without this, we need to guess (potentially very incorrectly) at the correct dependencies for the project. .NET Core dependency management is insanely complex and we do our best not to tangle with it, partly because it undergoes significant changes with every major iteration of the platform.
Maybe we could have some setup where we allow the .deps to be included with the binary files (where its relevant), but then there are other strings attached to this, as dependencies can vary considerably depending on which machine is used to build the project.
I guess the bottom line on this is that we don't have a way to work around this problem without breaking some very big abstractions in the build system. Every abstraction we break introduces unintended side effects and weird stuff that can be hard to track down.
Post NCrunch V5, we have some rough plans to do more work in this area. It's my hope that we might find an opportunity to more thoroughly solve this problem in a way that is reliable and doesn't involve messy hacks through everyone's project files. This is a deceptively big problem area that will take time and careful planning to properly solve.
Rafael Nieto commented
Thanks Remco. Yes I do set 'Ignore this project completely' to true by default for all projects. I only unignore those projects that I'm working on to avoid building them and speed up development.
I'm actually using binary references now as a workaround by placing them inside an ItemGroup node with the NCrunch build variable as a condition. I also use 'Infer project references using assembly names' so that NCrunch knows which of my unignored projects to rebuild.
However this workaround is not ideal since I have to maintain this binary reference list manually. I know that using project references and letting NCrunch build the whole dependency chain is ideal. However a lot of the referenced projects are owned by other teams. And a lot of those projects have custom build steps which don't play well with NCrunch.
Even if I fixed all the projects in the dependency chain to work with NCrunch, it would take a very long time to build all of those dependencies each time I create a new branch in git. That is why we have a custom tool to copy the built binaries from the build share.
I was wondering if msbuild has an API to get the target location of a project. I don't know how NCrunch interacts with msbuild so I'm not sure if that is possible. When I run msbuild from our build environment (which uses BuildProjectReferences=false) it somehow knows where the corresponding binaries are and it builds successfully without having to build the dependency chain.
A feature such as this is quite tricky to implement, as NCrunch would need to be able to find the built assembly for the referenced project.
The error you're receiving about a missing or ignored project reference seems strange to me. NCrunch would normally load the referenced projects outside the solution and treat it as a normal project (though not reported in the IDE). Did you switch the project off via the 'Ignore this project completely' setting?
Note that you can significantly reduce the overhead of loaded projects by setting their 'Instrument output assembly' setting to False. In this way, NCrunch will still build the projects as part of the dependency chain, but they won't take much attention from the engine as they won't have any coverage data.