AdminRemco Mulder
(NCrunch Developer, NCrunch Website)
My feedback
34 results found
-
1 vote
An error occurred while saving the comment -
8 votes
An error occurred while saving the comment Hi John,
If you mark the tests involved with NCrunch.Framework.IsolatedAttribute, does this suppress the issue? Is there any chance you can share a code sample that can reproduce the problem? I'm always happy to help in the support forum with issues like this one. There should be a way we can resolve this problem for you.
An error occurred while saving the comment There is a key difference between parallelisation in xUnit vs NCrunch: xUnit runs tests in parallel within the same process (i.e. different thread), where NCrunch runs tests in parallel in different processes (i.e. different process).
This means that a test that cannot be parallelised in xUnit may still be parallelisable using NCrunch. For example, if a test is making use of code that isn't thread-safe and is making use of static state, it can't be parallelised using xUnit, but can still be parallelised using NCrunch. If a test is making use of a shared resource on the file system, it cannot be parallelised by either.
This makes it unwise for NCrunch to simply assume that a test fixture marked with an xUnit collection cannot be parallelised. Really, these are two completely different mechanisms and the only way for NCrunch to know whether the test should have parallelisation restrictions in place for cross-process parallelisation is through specialised attributes targeted towards NCrunch (i.e. ExclusivelyUsesAttribute).
-
3 votes
An error occurred while saving the comment Not at the moment, but if it gets enough attention here we will take a closer look.
-
2 votes
An error occurred while saving the comment Thanks for the extra details. I don't think your situation is unusual, as we do have many customers working with subset solutions spanning wider ranges of projects.
I've had a bit of a think about ways we could approach this. My best idea was to implement a setting that would allow you to specify a list of DLLs that would take the place of project references declared in a project, kind of the reverse of the 'Infer project references using assembly names' setting.
After more consideration, I realised this would actually be extremely difficult to implement reliably under .NET Core/.NET5+. This is because for runtime purposes, a project outputs more than just a DLL - we also need the .deps file that specifies its dependency data. Without this, we need to guess (potentially very incorrectly) at the correct dependencies for the project. .NET Core dependency management is insanely complex and we do our best not to tangle with it, partly because it undergoes significant changes with every major iteration of the platform.
Maybe we could have some setup where we allow the .deps to be included with the binary files (where its relevant), but then there are other strings attached to this, as dependencies can vary considerably depending on which machine is used to build the project.
I guess the bottom line on this is that we don't have a way to work around this problem without breaking some very big abstractions in the build system. Every abstraction we break introduces unintended side effects and weird stuff that can be hard to track down.
Post NCrunch V5, we have some rough plans to do more work in this area. It's my hope that we might find an opportunity to more thoroughly solve this problem in a way that is reliable and doesn't involve messy hacks through everyone's project files. This is a deceptively big problem area that will take time and careful planning to properly solve.
An error occurred while saving the comment A feature such as this is quite tricky to implement, as NCrunch would need to be able to find the built assembly for the referenced project.
The error you're receiving about a missing or ignored project reference seems strange to me. NCrunch would normally load the referenced projects outside the solution and treat it as a normal project (though not reported in the IDE). Did you switch the project off via the 'Ignore this project completely' setting?
Note that you can significantly reduce the overhead of loaded projects by setting their 'Instrument output assembly' setting to False. In this way, NCrunch will still build the projects as part of the dependency chain, but they won't take much attention from the engine as they won't have any coverage data.
-
1 vote
An error occurred while saving the comment It's possible to do this using configuration.
You can assign a category to your integration tests, then exclude them from your engine mode's 'Tests to execute automatically' using the 'Customise engine modes' option - https://www.ncrunch.net/documentation/concepts_engine-modes
-
7 votes
An error occurred while saving the comment At the moment no feature in NCrunch exists that can export coverage in partcover/opencover/sonarcube/dotcover format. This is because NCrunch's code coverage is flat over the source (i.e. files and directories) rather than IL-indexed (classes/methods).
We've been looking at ways to convert between these, though there are performance implications. At some stage this will likely happen.
-
1 vote
An error occurred while saving the comment This behaviour is controlled by the following configuration setting: http://www.ncrunch.net/documentation/reference_global-configuration_auto-adjust-clashing-marker-colours
-
18 votes
An error occurred while saving the comment Thanks for this. I've updated the website based on your feedback.
-
0 votes
An error occurred while saving the comment This feature does already exist, but it is controlled on the grid node itself (not the client).
To make use of it, you need to use the Grid node configuration tool (NCrunch.GridNode.Configuration.exe) and assign a Node Id (Name) to the grid node. If no value is specified for this field, the grid node will use its network name instead (which is what you're seeing right now).
-
14 votes
An error occurred while saving the comment Der Albert has also suggested stack traces should be cleaned up for async/await - http://forum.ncrunch.net/yaf_postsm11731_Make-async-await-Stacktrace-readable.aspx#post11731
-
16 votes
An error occurred while saving the comment It's hard for me to say never on this, because you never really know, but what I do know is:
1. It's proving very hard to keep up with eco-system changes, because NCrunch is so integrated with so many toolsets, and many of these are moving targets. This is a strong reason to avoid integration with additional toolsets if possible.
2. There doesn't seem to be huge demand for NSpec, relative to the other frameworks.
3. We hope to eventually implement an API or SDK allowing people to build atop of NCrunch and implement their own test frameworks or adapters. I can't promise yet when this will be available, but NSpec would be an ideal use case for it.
4. We don't expect to implement an NSpec adapter ourselves in the foreseeable future.
-
2 votes
An error occurred while saving the comment There are built-in shortcuts to do this via the menu system.
Alt+U, S, 1
Alt+U, S, 2
Alt+U, S, 3,
etcThere are no hard coded shortcuts to do this directly because the engine modes are entirely dynamic, and under Visual Studio shortcuts need to be statically configured.
-
1 vote
An error occurred while saving the comment This can be done with custom configuration.
Since v3, it's possible to override many of the UI setting of NCrunch inside the engine modes. A tidy way to know which engine mode is selected is by customising your engine modes so that they each override the 'Marker style' configuration setting. In this way, you can have a differently shaped marker for each engine mode.
-
37 votes
An error occurred while saving the comment Hi,
Sorry there are no plans to introduce this in the near future.
Support for C++ would require a whole new build, instrumentation and runtime system. It would require a significant effort to both introduce this support and maintain it. At present, there does not appear to be a strong enough business case to warrant the cost of this.
-
2 votes
An error occurred while saving the comment Use of the NCrunch Category attribute is only necessary if you're using a test framework that doesn't have its own category attribute equivalent (NCrunch will detect categories of tests via framework integration).
Once you've categorised the test, all your need to do and update the engine mode to set the 'Tests to execute automatically' filter inside this engine mode.
An error occurred while saving the comment It's possible to get the engine to do this without needing a new feature.
You can add a special category to tests that use Automapper, then modify the 'Run Impacted Tests Automatically' engine mode to also run tests that are marked with this category, in addition to the impacted tests.
-
60 votes
An error occurred while saving the comment So this feature is planned, at least, in the sense that I expect it will eventually be delivered. It's hard for me to make any commitments on the timing of its delivery because it is a very large feature and dependent on many other areas of the product to be ready for it.
The design of this SDK/API will depend very much on the sorts of things people want to do with NCrunch. I understand that being able to introduce support for additional test frameworks and access code coverage data would probably be top requests here.
Can anyone else share any ideas for plugins they'd like to write?
-
1 vote
An error occurred while saving the comment Thanks for the extra details. Have you noticed anything else in the NCrunch UI that seems a bit off? Or is it just the engine mode filter window?
An error occurred while saving the comment This is defective behaviour. Can you share any more information about your display settings and DPI? If I can reproduce it, I may be able to introduce a fix.
-
5 votes
An error occurred while saving the comment Have you tried using the 'Sliding build delay' global configuration setting?
I ask because this seems very similar to what you're requesting.
Having NCrunch trigger when a line is completed can be subjective - how would it know when a line is finished?
-
9 votes
An error occurred while saving the comment If I understand this request correctly, NCrunch should already be doing this.
In the Tests Window, you can choose to 'pin' the test fixture itself. When this is done, you should see the little pin appear next to the fixture and all its child tests. If any new tests are added to the fixture, they will automatically be pinned.
Note that this only works at fixture level - it isn't currently possible to pin a namespace or project.
-
2 votes
An error occurred while saving the comment By default, tests that are excluded from automatic execution using an engine mode will still have their code coverage showing inline. An exception to this is if you also have the engine mode deliberately set up with a filter to also exclude these tests from coverage.
Of course, this is provided the tests have already been previously run and they have coverage data available to the engine.
We do support Directory.build.props, but unfortunately we can't support it for certain properties (like TargetFrameworks). This is unfortunately due to technical limitations that we haven't yet found a way around.
If you're using Directory.build.props to set TargetFrameworks, a simple workaround is to use the NCrunch Custom Build Properties setting to force NCrunch to use a specific set of frameworks. https://www.ncrunch.net/documentation/reference_project-configuration_custom-build-properties
If you set the TargetFrameworks build property to the frameworks you intend to use NCrunch for, there will be no need to adjust the way you specify it in your projects, and you can keep using Directory.build.props as you are.