Unit Test fails on Run All but not on Run Selected Tests with all selected

my test fail when run together but pass individually c
jest test passes individually
unit test passes when in debug but fails when run java
xunit run tests in order
visual studio 2019 not running all tests
test is not running
test not run mstest
nunit not running tests

I'm getting some weird behavior. If I click "Run All" in Test Explorer then 1 of my unit tests fails, but if I select all tests and click "Run Selected Tests" then the unit test passes.

The test that is failing is throwing a reflection error: System.Reflection.TargetException: Non-static method requires a target. on a type which is defined in the dll code which I'm testing. There doesn't appear to be anything odd with the class - there are plenty of other classes defined in the dll which reflection is happy with. I've included the test stack trace below.

NB It is a complex test - it reads inputs & expected answers from an .xlsx file, populates a LocalDb with data from the xlsx, performs a calculation using the data in the LocalDB and then compares the calculated to expected values. However as I say it is working and works when I run all tests (using select all > Run Selected Test).

What is different about Run All? Any insights would be appreciated.

I have tried a clean and rebuild with no luck. Catching and logging reflection error indicates that the GetValue call is throwing for every property I attempt to access on the type - but only when run by "Run All" and only on this one type? (if I catch the error then all GetValues succeed on all other types).

Stack Trace

Test Name:  IT_CheckCashOnly1DepositOutputValues
Test FullName:  Lib.AE.Tests.Integration.CalculationTests.IT_CheckCashOnly1DepositOutputValues
Test Source:    c:\netreturn.co.za\Main\NetReturn\Lib.AE.Tests\IntegrationTests\CalculationTest.cs : line 23
Test Outcome:   Failed
Test Duration:  0:00:00.1661906

Result Message: 
Test method Lib.AE.Tests.Integration.CalculationTests.IT_CheckCashOnly1DepositOutputValues threw exception: 
System.Reflection.TargetException: Non-static method requires a target.
Result StackTrace:  
at System.Reflection.RuntimeMethodInfo.CheckConsistency(Object target)
at System.Reflection.RuntimeMethodInfo.InvokeArgumentsCheck(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimePropertyInfo.GetValue(Object obj, BindingFlags invokeAttr, Binder binder, Object[] index, CultureInfo culture)
at System.Reflection.RuntimePropertyInfo.GetValue(Object obj, Object[] index)
at Lib.AE.Xlsx.XlsxHelper.Compare[T](T expected, T calculated, ExcelWorksheet ws, Int32 r, Int32 colStart, Boolean& valid) in c:\netreturn.co.za\Main\NetReturn\Lib.AE\Xlsx\XlsxHelper.cs:line 101
at Lib.AE.Xlsx.XlsxWorkSheet_SharePNL.CompareXlsx(ExcelPackage pck, List`1 expectedXlsx, ValuationCalculation calc) in c:\netreturn.co.za\Main\NetReturn\Lib.AE\Xlsx\XlsxSharePNL.cs:line 143
at Lib.AE.Tests.Integration.CalculationTests.CheckCalculationResults(String xlsxDocToLoad, WorkSheets testingScenarios) in c:\netreturn.co.za\Main\NetReturn\Lib.AE.Tests\IntegrationTests\CalculationTest.cs:line 64
at Lib.AE.Tests.Integration.CalculationTests.IT_CheckCashOnly1DepositOutputValues() in c:\netreturn.co.za\Main\NetReturn\Lib.AE.Tests\IntegrationTests\CalculationTest.cs:line 23

Solution

So this turned out to be (a) my problem - my unit test shared state with another unit test and (b) an order problem. Be aware that it's not obvious which order TestExplorer will run your tests in. I created a new UnitTestProject with 2 x UnitTest .cs files and three TestMethods each i.e.:

UnitTest1.cs

    [TestMethod]
    public void ONE_AAA() {}

    [TestMethod]
    public void ONE_BBB() {}

    [TestMethod]
    public void ONE_CCC() {}

UnitTest2.cs

    [TestMethod]
    public void TWO_CCC() {}

    [TestMethod]
    public void TWO_BBB() {}

    [TestMethod]
    public void TWO_AAA() {}

Then ran these tests via two methods i.e. (1) Run All (2) Select all and Run Selected Tests, and logged the order in which the tests were started by TestExplorer. The results for Run Selected Tests are fairly unintuitive:

-- Run All
2013-01-16 11:53:47.4062 INFO TestInitialize: ONE_AAA
2013-01-16 11:53:47.4122 INFO TestCleanup: ONE_AAA
2013-01-16 11:53:47.4122 INFO TestInitialize: ONE_BBB
2013-01-16 11:53:47.4122 INFO TestCleanup: ONE_BBB
2013-01-16 11:53:47.4122 INFO TestInitialize: ONE_CCC
2013-01-16 11:53:47.4282 INFO TestCleanup: ONE_CCC
2013-01-16 11:53:47.4282 INFO TestInitialize: TWO_CCC
2013-01-16 11:53:47.4282 INFO TestCleanup: TWO_CCC
2013-01-16 11:53:47.4282 INFO TestInitialize: TWO_BBB
2013-01-16 11:53:47.4282 INFO TestCleanup: TWO_BBB
2013-01-16 11:53:47.4282 INFO TestInitialize: TWO_AAA
2013-01-16 11:53:47.4282 INFO TestCleanup: TWO_AAA

-- Select All > Run Selected
2013-01-16 11:55:26.0139 INFO TestInitialize: TWO_BBB
2013-01-16 11:55:26.0139 INFO TestCleanup: TWO_BBB
2013-01-16 11:55:26.0249 INFO TestInitialize: ONE_BBB
2013-01-16 11:55:26.0249 INFO TestCleanup: ONE_BBB
2013-01-16 11:55:26.0249 INFO TestInitialize: TWO_AAA
2013-01-16 11:55:26.0249 INFO TestCleanup: TWO_AAA
2013-01-16 11:55:26.0249 INFO TestInitialize: TWO_CCC
2013-01-16 11:55:26.0249 INFO TestCleanup: TWO_CCC
2013-01-16 11:55:26.0249 INFO TestInitialize: ONE_CCC
2013-01-16 11:55:26.0249 INFO TestCleanup: ONE_CCC
2013-01-16 11:55:26.0249 INFO TestInitialize: ONE_AAA
2013-01-16 11:55:26.0249 INFO TestCleanup: ONE_AAA

I have been caught out in the past with running multiple unit tests in that the order in which the tests run may not be in the order in which they are declared in the test class, and may in fact be in order of test method name. e.g. if I have

[Test]
public void PreviousTest()
{
}

[Test]
public void LaterTest()
{
}

Then LaterTest gets run first as its name appears before PreviousTest when ordered alphabetically.

This won't matter if all of your tests are entirely independent, but if they are modifying shared resources then you may get unusual behaviour if you were expecting that LaterTest's changes wouldn't have any effect on PreviousTest due to it being declared second.

Running all tests fails some tests that pass individually, but fails a few when I run them all (via Unit Test Runner in VS2003). When single tests run but a group doesn't it is a sign that the tests are Strict = true or method specific strict (see Arbitary Calls). My expectation is that all objects (​mocked or not) will be freed after each test and each test creates all  The test passes when it is run by itself, run in debug mode with or without other tests, run with all other tests by selecting them all and then right clicking and choosing "Run Selected Tests", run all tests by closing VS, reopening and then choosing "Run Not Run Tests.


Make sure your members are being initialized in the setup scope, otherwise they can be reused.

[TestFixture]
public class RegistrationInteractorTests
{
    private  IRegistrationService _registrationService;
    private  IRegistrationValidationService _registrationValidationService;
  ......

    [SetUp]
    public void Init()
    {
        _registrationService = A.Fake<IRegistrationService>();
        _registrationValidationService = A.Fake<IRegistrationValidationService>();
    }

....

}

// The wrong way would be like this:

 [TestFixture]
    public class RegistrationInteractorTests
    {
        protected readonly IRegistrationService _registrationService = A.Fake<IRegistrationService>();

        protected readonly IRegistrationValidationService _registrationValidationService =
            A.Fake<IRegistrationValidationService>();



        [SetUp]
        public void Init()
        {

        }

18 Unit Test will pass when selecting "Run Selected Tests", but 2-5 , 18 Unit Test will pass when selecting "Run Selected Tests", but 2-5 out of the 18 unit test will fail when I select "Analyze Code Coverage for Selected Test" set the test setting file to local.runsettings file which excludes all the *\.test\.dll files. Although, it still shouldn't run out of memory whether the runsettings file is set or not,  I am having this wierd problem with MOQ Unit tests. I have around 636 unit tests written using MOQ Framework in VS 2012. When I run them project-wise and/or individually, they pass successfully. but whenever I select Run-All they fail randomly showing different errors. Most of the methods show IO.FileNotFoundException.


We tried to investigate this issue for a week now. Make sure that all your TEST Projects are using the same version of DLLs, especially the Nuget Packages.

The thing is that, when you execute the "Run All", all the used libraries in all your TEST Projects are placed in one folder (something like this MyApp\TestResults\Deploy_user1 2018-11-20 15_52_15\Out). and because of these Dll libraries conflicted.

Tests fail in large runs but pass individually in JustMock General , I am having a problem with JustMock where tests pass when run individually but fail when all tests are run. I am not using any static or final  My tests have been working quite well up to now in VS2015. However, over my holiday period I installed VS2017 so I don't know if that affected anything. When I select an [Explicit] test (in VS2015) it is not being run. All other tests are being run; which isn't quite what I want. I updated to R# 2017.3.1 and I'm on NUnit 3.4.1.


Your [TestInitialize] method runs before every single test. Remove any single-test-dependent code from it and put it in the individual test.

Individual, successful unit tests fail when run with others, Also, since I'm not sure how to phrase the question with a lot of detail, I'm just hoping for some I have a few unit tests for a solution that, when run individually in Visual Studio, run fine, but when I selected multiple tests to run, only the first test executed succeeds, while others all subsequent tests fail. 1. When single tests run but a group doesn't it is a sign that the tests are not isolated and don't clean up after themselves. 2. When you have a -1 instance, it is a sign that an object was created before the mock was kicked in. Both of these together can point to the problem, which can be one of the following. 1.


Unit test fails only when I "Run all" - MSDN, I have a unit test which fails when I choose "Run All", but passes under all other choosing "Run Selected Tests", run all tests by closing VS, reopening and then I tried locking the dataContext and that made no difference. If I add <resueForks>false</reuseForks> to my surefire-plugin definition, all tests pass, but I am not happy about that solution/workaround. It slows down the build and it isn't respected in Eclipse - that is I simply run a JUnit test runner against an entire project in a single shot without it failing.


My test cases fail when run together but pass individually, at android , PostShiftResponse import org.junit. My test cases fail when run together but pass individually, at android https://stackoverflow.com/questions/26561511/​tests-pass-when-run-individually-but-not-when-the-whole-test-class-  All tests are running, all tests are showing the right test outcome. Now a coworker told me, that the tests are not running on his machine using the Visual Studio test explorer. They are not working either on my machine, so i can exclude some local missing files or something.


Automated tests pass every time when run individually, but when , The button becomes enabled once a role is selected. Sometimes the test will try to push the button before it has been enabled. I have not  Its the first test that pass's then all following tests generally fail and they allways fail for the same reason posted above. – Graeme Secondwave Jan 24 '12 at 13:10 Doesn't help your specific question, but check out ncrunch.net , it runs your tests in the background.