How to test same assertion for large amount of data

python ddt
python data driven test framework
pytest ddt
python unittest multiple inputs
unpack ddt
modulenotfounderror: no module named ddt
ddt example

I am using python unittest module to do a number of tests; however, it is very repetitive.

I have a lot of data that I want to run through the same test over and over, checking if correct. However, I have to define a test for every one.

For instance I want to do something similar to this. I know I could do it using a generator (found it in a previous thread here). But are there alternatives, maybe even using a different testing module?

Any suggestions would be great.

import unittest

class TestData(unittest.TestCase):
    def testNumbers(self):
        numbers = [0,11,222,33,44,555,6,77,8,9999]
        for i in numbers:
            self.assertEqual(i, 33)

Sample code for solution suggested by Bill Gribble could look like this:

import unittest

class DataTestCase(unittest.TestCase):
    def __init__(self, number):
        unittest.TestCase.__init__(self, methodName='testOneNumber')
        self.number = number

    def testOneNumber(self):
        self.assertEqual(self.number, 33)

    def shortDescription(self):
        # We need to distinguish between instances of this test case.
        return 'DataTestCase for number %d' % self.number


def get_test_data_suite():
    numbers = [0,11,222,33,44,555,6,77,8,9999]
    return unittest.TestSuite([DataTestCase(n) for n in numbers])

if __name__ == '__main__':
    testRunner = unittest.TextTestRunner()
    testRunner.run(get_test_data_suite())

Is it OK to have multiple asserts in a single unit test?, [Test] public void ValueIsInRange() { int value = GetValueToTest(); Assert. three (or more) asserts into one big boolean condition makes it harder to read, harder to as you can always rewrite them as multiple conditions in the same assertion. steps to verify the intermediate data between steps rather than risking that the  You test this assertion by examining title documents or deeds for proof of ownership. You also review lease agreements to make sure that any capitalized leases are included on the balance sheet and that any operating leases aren’t. Completeness: Completeness evaluates the management assertion opposite to occurrence. This means all the fixed and intangible assets your client owns show up on the balance sheet; none are missing.


You may want to consider using the unittest.TestSuite class, which will allow you to dynamically construct a set of unittest.TestCase instances which will get run separately. Your unittest.TestCase subclass should define just one test method, with the class accepting a construction parameter passing in the value to test against for that particular instance.

Big Data and Overcoming QA challenges in Big Data Testing, Big Data and Overcoming QA challenges in Big Data Testing Big Data, we all have heard this term, and Everyone is talking about big data in  Assertions enable you to check the state of data produced by other actions. An assertion query is written to find rows that violate one or more rules. If the query returns any rows, then the assertion will fail. Auto-generated assertions


The ddt library was built to solve exactly what you are asking for unittest[*].

For example:

import ddt
import unittest

@ddt.ddt
class EvalTests(unittest.TestCase):

    @ddt.data(
            ('1', 1),
            ('1 == 1',  True),
            ('1 == 2',  False),
            ('1 + 2',   4),  ## This will fail
    )
    def test_eval_expressions(self, case):
        expr, exp_value = case
        self.assertEqual(eval(expr), exp_value)

And when you run it, you get 4 TestCases instead of just one:

$ python -m unittest  -v  test_eval.py
test_eval_expressions_1___1___1_ (test_eval.EvalTests) ... ok
test_eval_expressions_2___1__1___True_ (test_eval.EvalTests) ... ok
test_eval_expressions_3___1__2___False_ (test_eval.EvalTests) ... ok
test_eval_expressions_4___1_2___4_ (test_eval.EvalTests) ... FAIL

======================================================================
FAIL: test_eval_expressions_4___1_2___4_ (test_eval.EvalTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python/lib/site-packages/ddt.py", line 129, in wrapper
    return func(self, *args, **kwargs)
  File "/Work/test_eval.py", line 15, in test_eval_expressions
    self.assertEqual(eval(expr), exp_value)
AssertionError: 3 != 4

----------------------------------------------------------------------
Ran 4 tests in 0.002s

FAILED (failures=1)

Notice that ddt tries to come up with names for the generated TCs.

Install it with pip:

pip install ddt

[*] The same solution for the pythonic pytest framework (pytest.mark.parametrize) is integrated into the core tool, and is worth switching to pytest just for this feature alone.

Data-Driven Testing using Python and the Parameterized Module, Data-driven tests run the same test logic and assertions, but they fetch Also, data-driven tests may still require a large amount of coding,  An introduction to effective test assertions In the software industry, before we deliver a product, we need to run many test assertions to make sure the product has met the customer's expectation. So what is a test assertion? What is a test assertion? In software industry, before we deliver a product, we need to run many test assertions to make sure the productmet customer's expectation.


The problem with running assertions in a loop is that, if one of the assertions fails, you don't know which value caused it (in your example, it would fail on 0, but you don't know that until you debug). On the other hand, repeating self.assertEqual(i, 33) is an even worse idea, because it introduces code duplication.

What I do in my test is to create a simple, shortly-named internal function inside the test and call it with different arguments. So your function would look like this:

import unittest

class TestData(unittest.TestCase):
    def testNumbers(self):
        def eq(i):
            self.assertEqual(i, 33)
        eq(0)
        eq(11)
        eq(222)
        eq(33)
        eq(44)
        eq(555)
        ... 

This way, when the assertion fails for 0, you immediately see it on the stack trace printed by the unittest module.

Data quality testing with SQL assertions, A short overview of how to to use SQL based data assertions to test data above into a single query to quickly find any customer_id value violating one of our rules default customer types are set, and duplicate rows for the same customer_id  we will use XML assertion in JMeter to check whether the provided URL consist of valid XML data. If the invalid XML code found, failure will be reported with an assertion failure message. To verify XML assertion, you just have to add XML Assertion same as the way of adding other assertions and run the test plan.


In another post I stumbled accross Nose Tests It is more suited to data driven testing.

class Test_data():
    def testNumbers():
        numbers = [0,11,222,33,44,555,6,77,8,9999]
        for i in numbers:
            yield checkNumber, num

def checkNumber(num):
    assert num == 33

The above code does the exact same thing as my first post. No imports are needed, just write a python class.

You execute the tests by typing:

nosetests filename

[PDF] An Algorithm for Efficient Assertions-Based Test Data , Email: dralyami@tvtc.gov.sa. Abstract1—Automated assertion-based test data generation removed previously, therefore, a large number of assertions will not be violated. violating more than one assertion based on the same input data x. My guideline is usually that you test one logical CONCEPT per test. you can have multiple asserts on the same object. they will usually be the same concept being tested. I think that, there are some cases where multiple assertions are needed (e.g. Guard Assertion ), but in general I try to avoid this.


Getting started with Assertions, Any number of assertions can be added to a sampler TestStep, each Same goes for enabling/disabling particular Assertion inside the category. A large number of assertions are available for validating received messages, some being JDBC Timeout - validates that the result is received from the target database within  The Assertion test step expands the idea of assertion handling and management. It provides the flexibility of creating simple and complex assertions that can assert any property from the project level to individual test steps in a test case, as well as request, response, JMS, JDBC or Security-related activities.


Best practices for writing unit tests, Unit tests are standalone, can be run in isolation, and have no dependencies on any outside factors such as a file system or database. If you find testing the code taking a large amount of time compared to Let's speak the same language Arrange, Act, Assert is a common pattern when unit testing. A large amount of audit evidence may, however, not compensate for its poor quality. Appropriateness is the measure of the quality of audit evidence. The quality of audit evidence depends on whether it is relevant and reliable in providing support to the conclusions on which the auditor’s opinion is based.


How to use Assertions in JMeter (Response Example), Assertion help verify that your server under test returns the expected results. Reviews · Salesforce · SEO · Software Engineering · VBA. Big Data the size be equal to, greater than, less than, or not equal to a given number of bytes. The XML Assertion tests that the response data consists of a formally  One way to reassure yourself about your own test would be to carry out (before the test, and indeed hopefully before you have data) a study of the power at some small-but-relevant-to-your-application effect size; if you have very good power then, and reasonably low type I error rate, then you would nearly always be making the right decision when the effect size is at least that large and nearly always be making the right decision when the effect size was 0.