How to mock the service layer in a python (flask) webapp for unit testing?
I am working on a webapp in flask and using a services layer to abstract database querying and manipulation away from the views and api routes. Its been suggested that this makes testing easier because you can mock out the services layer, but I am having trouble figuring out a good way to do this. As a simple example, imagine that I have three SQLAlchemy models:
class User(db.Model): id = db.Column(db.Integer, primary_key = True) email = db.Column(db.String) class Group(db.Model): id = db.Column(db.Integer, primary_key = True) name = db.Column class Transaction(db.Model): id = db.Column(db.Integer, primary_key = True) from_id = db.Column(db.Integer, db.ForeignKey('user.id')) to_id = db.Column(db.Integer, db.ForeignKey('user.id')) group_id = db.Column(db.Integer, db.ForeignKey('group.id')) amount = db.Column(db.Numeric(precision = 2))
There are users and groups, and transactions (which represent money changing hands) between users. Now I have a services.py that has a bunch of functions for things like checking if certain users or groups exist, checking if a user is a member of a particular group, etc. I use these services in an api route which is sent JSON in a request and uses it to add transactions to the db, something similar to this:
import services @app.route("/addtrans") def addtrans(): # get the values out of the json in the request args = request.get_json() group_id = args['group_id'] from_id = args['from'] to_id = args['to'] amount = args['amount'] # check that both users exist if not services.user_exists(to_id) or not services.user_exists(from_id): return "no such users" # check that the group exists if not services.group_exists(to_id): return "no such group" # add the transaction to the db services.add_transaction(from_id,to_id,group_id,amount) return "success"
The problem comes when I try to mock out these services for testing. I've been using the mock library, and I'm having to patch the functions from the services module in order to get them to be redirected to mocks, something like this:
mock = Mock() mock.user_exists.return_value = True mock.group_exists.return_value = True @patch("services.user_exists",mock.user_exists) @patch("services.group_exists",mock.group_exists) def test_addtrans_route(self): assert "success" in routes.addtrans()
This feels bad for any number of reasons. One, patching feels dirty; two, I don't like having to patch every service method I'm using individually (as far as I can tell there's no way to patch out a whole module).
I've thought of a few ways around this.
- Reassign routes.services so that it refers to my mock rather than the actual services module, something like:
routes.services = mymock
- Have the services be methods of a class which is passed as a keyword argument to each route and simply pass in my mock in the test.
- Same as (2), but with a singleton object.
I'm having trouble evaluating these options and thinking of others. How do people who do python web development usually mock services when testing routes that make use of them?
Using Mocks to Test External Dependencies or Reduce Duplication, This isn't even something that only works inside unit tests. You can do this kind of "monkeypatching" in any kind of Python code! That may take a The solution is simple - we can create a bare-bones local implementation of the service we are calling (which is called a mock in testing parlance) and run our unit tests using this mock. We can separate out our integration tests into a separate file and run those before we commit changes to our code.
You can patch out the entire services module at the class level of your tests. The mock will then be passed into every method for you to modify.
@patch('routes.services') class MyTestCase(unittest.TestCase): def test_my_code_when_services_returns_true(self, mock_services): mock_services.user_exists.return_value = True self.assertIn('success', routes.addtrans()) def test_my_code_when_services_returns_false(self, mock_services): mock_services.user_exists.return_value = False self.assertNotIn('success', routes.addtrans())
Any access of an attribute on a mock gives you a mock object. You can do things like assert that a function was called with the
mock_services.return_value.some_method.return_value. It can get kind of ugly so use with caution.
Guide to Python Flask Unit Testing, Even if you don't use Flask, the unit-testing concepts illustrated are generally applicable. We will use a pytest feature called “fixtures” to turn our web app into a Python is a layer of abstraction that handles all of the app's interactions with the server. from test.unit.webapp import client import mock from mock import call This package provides some classes to build a Service layer and expose an API that interacts with the model. The first idea is to remove all logic of the routes and model of the Flask application, and put it in the service layer. The second goal is to provide a common API that can be use to manipulate a model regardless of its storage backend.
I would also raise a hand for using dependency injection for such needs. You can use Dependency Injector to describe structure of your application using inversion of control container(s) to make it look like this:
"""Example of dependency injection in Python.""" import logging import sqlite3 import boto3 import example.main import example.services import dependency_injector.containers as containers import dependency_injector.providers as providers class Core(containers.DeclarativeContainer): """IoC container of core component providers.""" config = providers.Configuration('config') logger = providers.Singleton(logging.Logger, name='example') class Gateways(containers.DeclarativeContainer): """IoC container of gateway (API clients to remote services) providers.""" database = providers.Singleton(sqlite3.connect, Core.config.database.dsn) s3 = providers.Singleton( boto3.client, 's3', aws_access_key_id=Core.config.aws.access_key_id, aws_secret_access_key=Core.config.aws.secret_access_key) class Services(containers.DeclarativeContainer): """IoC container of business service providers.""" users = providers.Factory(example.services.UsersService, db=Gateways.database, logger=Core.logger) auth = providers.Factory(example.services.AuthService, db=Gateways.database, logger=Core.logger, token_ttl=Core.config.auth.token_ttl) photos = providers.Factory(example.services.PhotosService, db=Gateways.database, s3=Gateways.s3, logger=Core.logger) class Application(containers.DeclarativeContainer): """IoC container of application component providers.""" main = providers.Callable(example.main.main, users_service=Services.users, auth_service=Services.auth, photos_service=Services.photos)
Having this will give your a chance to override particular implementations later:
Hope it helps.
Python Web Applications With Flask – Part III – Real Python, a Web Application with Flask we'll explore unit and integration testing. Integration Tests; Mocking Free GeoIP; Set up the test data and mocks; Patch the mock This ensures that our unit tests will run quickly - otherwise, every time we the database and the user - it also interacts with the third-party service Free GeoIP. Such a layer is commonly called a REST layer, because usually the semantic of the addresses comes from the REST recommendations. Flask is a lightweight web server with a modular structure that provides just the parts that the user needs. In particular, we will not use any database/ORM, since we already implemented our own repository layer.
@patch("dao.qualcomm_transaction_service.QualcommTransactionService.get_max_qualcomm_id",20) def test_lambda_handler(): lambda_handler(event, None)
I used mocking seeing your example and my method expects to return 20 whenever in lambda function testing locally get_max_qualcomm_id us made .but on reaching the above method i get a exception int type object is not Callable. Please let me know what is the problem here .
This is actual method being call which i am trying to mock :
last_max_id = QualcommTransactionService().get_max_qualcomm_id(self.subscriber_id)
Mocking External APIs in Python – Real Python, NOTE: The mock library is part of unittest if you are using Python 3.3 or greater. You should pull the code out of your test and refactor it into a service function python webapp flask. Related course: Python Flask: Make Web Apps with Python. Passing Variables. Lets display random quotes instead of always the same quote. We will need to pass both the name variable and the quote variable. To pass multiple variables to the function, we simply do this:
Unit testing the web layer - Mockito for Spring, In this section, we'll build the controllers and unit test them in isolation from the web server. We have to mock out the service and data access logic. Perform the This series will show you how to create websites with python using the micro framework flask. Flask is designed for quick development of simple web applications and is much easier to learn and use
Flask API and Service Layer, Our First Use Case: Flask API and Service Layer Back to our allocations project! Figure 4-1 shows Unit testing with fakes at the service layer (test_services.py) Additionally, I rarely write unit tests for my Flask applications. I generally only write functional tests. In other words, I'm testing that all application endpoints work as expected with valid and invalid request data. Tools. In the Python world there are countless testing tools and libraries and its often difficult to decide which ones to use.
Large web apps in Python: A good architecture, If you are setting out to write a large application in Python using a to learn what unit tests really are: some devs write integration tests thinking they are unit tests enthusiasts may want everything in the system to be a Flask plugin). put it in a new, reusable layer, which most people call a “Service” layer, simple tables in a web app using flask and pandas with Python. Aug 9, 2015. Display pandas dataframes clearly and interactively in a web app using Flask. Web apps are a great way to show your data to a larger audience. Simple tables can be a good place to start. Imagine we want to list all the details of local surfers, split by gender.