Testing with Django REST Framework

One of the most overlooked things in software development, especially in fast-paced environments, is unit testing.
Today we’ll take a look at testing Django REST Framework applications. I’ll take you through creating the application, writing some tests and doing things the TDD way at the end.

DO I REALLY NEED TO WRITE UNIT TESTS?

The correct answer here would usually be “yes”, but I don’t think it’s that simple. Writing unit tests is as much of an art-form as it is to write code.
I’d say the necessity of writing tests is a contextual thing – there are times when you really should, times when it’s not necessary and times when you really should not.

WHEN NOT TO WRITE TESTS:

Don’t test that your framework does what it should do. That is the job of the framework developers.
Don’t write tests for hardcoded stuff. Tests for these things can be covered by their consumers. If you need to test that a method always returns Hello, world! with no changes, ever, then you should consider whether that method is even necessary to begin with.
Throw-away code. If you are planning to commit code to source control, it should be tested. If it’s just a quick bit of testing code, don’t bother testing it.

Another thing I have learned over time is to not chase high code coverage numbers. 80% is fine in my experience. Overly brittle tests can really make life difficult. Tests should ideally catch things that could break the functionality of your application.

WHEN SHOULD I WRITE TESTS?

Any code that needs to result in some expected outcome should be tested. You can test for things like:
- Making sure an external method was called with the correct parameters
- Making sure your method does what it should with values it received
- Checking changes on an object after some operation was performed on it.
- When you are fixing a bug. Write the test before you write the code to fix the bug. The test should replicate the bug. When you then write the code to make that test pass, your bug should be fixed.
As mentioned above, any code that you plan to commit to source control should be tested as much as is reasonable.

DEADLINES EXIST.

I’ve seen many projects where tests are being thrown out the door because of tight deadlines, lack of proper sprint planning, lack of know-how and so forth. There are lots of reasons why this happens. If we use test coverage as a metric to measure our code quality then testing should be an integral part of what we do on a daily basis. It’s worth noting that code coverage is not the same thing as code quality. Missing tests (or bad tests) result in technical debt that really bites you in the ass down the line, so it should be made clear to stake holders and project managers that a lack of proper testing or proper planning will have a financial impact at some point when that technical debt needs to be paid.

LET'S WRITE SOME CODE

For this post, I am not going to focus on any particular method of testing like TDD – we’ll just write an application and test it as we go. We’ll do a bit of TDD at the end, but I want the focus to be on actual unit testing. Let’s dive in.
To keep the example simple, we’re going to write a simple API that allows you to manage a list of people with their first and last names as well as their job titles.
We’ll then add some basic actions to those objects as well and make sure everything is well tested.

For brevity, I’ll leave out imports unless they’re really not obvious.

ENVIRONMENT SETUP

I personally use PyEnv for virtual environments because it’s easy to use and allows me to switch between Python versions easily.

				
					pyenv virtualenv venv-django-example
pyenv activate venv-django-example

Create a directory for your project, such as djangotesting-example with the following requirements.txt file in it:

				
					django
djangorestframework
coverage
django-filter

Run pip install -r requirements.txt.

Once done, run the following:

				
					django-admin startproject djangotesting .  ## the docs say this as well but seriously - take note of the '.' at the end!
cd djangotesting
django-admin startproject api
cd ..

Just a bit of setup before we get started: In your settings.py, add the following:

				
					INSTALLED_APPS = [
    ... # All the other boilerplate stuff
    'djangotesting.api'
]

... # Some more boilerplate stuff

REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': ['django_filters.rest_framework.DjangoFilterBackend']
}

In api/models.py create the following model:

				
					class Person(models.Model):
    JOB_TITLE_CHOICES = [
        ('developer', 'Developer'),
        ('business_analyst', 'BA'),
        ('tester', 'Tester'),
        ('magician', 'Magic person'),
    ]

    first_name = models.CharField(max_length=30, null=False, blank=False)
    last_name = models.CharField(max_length=30, null=False, blank=False)
    job_title = models.CharField(max_length=30, null=True, blank=True, choices=JOB_TITLE_CHOICES)

    def get_greeting(self):
        return f'Hello, {self.first_name} {self.last_name}.'

And create the accompanying serializer in api/serializers.py. You will probably need to create this file manually.

				
					class PersonSerializer(serializers.ModelSerializer):
    class Meta:
        model = Person
        fields = '__all__'

Almost done with our functional app, let’s create the ModelViewSet in api/views.py

				
					class PersonViewSet(viewsets.ModelViewSet):
    serializer_class = PersonSerializer
    queryset = Person.objects.all()

And in our djangotesting/urls.py:

				
					router = routers.DefaultRouter()
router.register(r'people', PersonViewSet)

urlpatterns = [
    path('', include(router.urls)),
    path('admin/', admin.site.urls),
]

In your CLI, run migrations:

				
					python manage.py makemigrations
python manage.py migrate

You should now be able to start the project with python manage.py runserver and manage people via the API using Postman, Insomnia or any other interestingly named REST client.

Done, right? NOPE. We’ve done quite a bit that won’t really need much testing – it’s mostly boilerplate DRF code, right? Except for that method in the model; the one to get a greeting. That will need some testing.

First, delete the file at api/tests.py and create a new folder djangotesting/tests with the following files in it:

__init__.py
test_models.py
test_views.py

In test_models.py, add the following code:

				
					from django.test import TestCase
from djangotesting.api.models import Person  

class PersonTest(TestCase):
    def test_get_greeting_when_called_should_return_greeting(self):
        person = Person.objects.create(first_name="John", last_name="Smith")
        greeting = person.get_greeting()

        self.assertEqual(greeting, "Hello, John Smith.")

Note the name of the class and method. The class is named after the class being tested, with the word Test following it. The name of the test method is named something like this: test_when_CONDITION_sould_DO_SOMETHING. These naming conventions are not set in stone, it’s just a convention I follow.

All we are doing here is creating a person, then calling the get_greeting method of that person. We then assert that there will be some outcome. In other frameworks, assertions might be called expectations. The point is, we are following three stages:

Arrange: This is the first line in the test method, where we create a person. This is where we do some setup for our test to be able to run.
Act: Also often referred to as test instead of act, this is the part where we run our code that is being tested. In this case, we run the greet method.
Assert: This is the part where we make assumptions about the outcome of our test. These can take the form of comparing one value to another, checking if a method was called or checking that some data is present in some form. There are lots of different assertions you can make to validate the outcome of the test.

To run tests, enter the following in your console:

				
					coverage run --source='djangotesting.api' manage.py test djangotesting

If all tests pass, you can run coverage report to get a coverage report on the command line or coverage html to get a coverage report as HTML.

This test represents the way we want our users to be greeted in this application. If someone changes the get_greeting method on the Person class, then this test will fail. That will then either spark a conversation as to whether the greeting should be changed or get the developer to update the test. Either way, the test caught the change in the outcome of that method and that is what is important. Let’s move on and test the view.

In your test_view.py file, add the following:

				
					from django.test import TestCase, Client

from djangotesting.api.models import Person


class PersonViewTest(TestCase):
    def setUp(self) -> None:
        self.client = Client()
        Person.objects.create(first_name="John", last_name="Smith", job_title="developer")
        Person.objects.create(first_name="Jane", last_name="Smith")
        Person.objects.create(first_name="Rowan", last_name="Atkinson")

For viewset testing you would normally do a bit of setup. Add some objects to the database, setup an API client and whatever else you need to do. Generally, I add stuff that all my tests might make use of here so that I don’t need to repeat it. Here we have just set up an API client and created some Person objects.

Furthermore, add the following:

				
					    def test_get_all_people_should_return_all_people(self):
        response = self.client.get("/people/")
        people = Person.objects.all()
        serializer = PersonSerializer(people, many=True)

        self.assertEqual(response.data, serializer.data)

    def test_get_individual_person_should_return_single_person(self):
        special_person = Person.objects.create(first_name="Penn", last_name="Teller", job_title="magician")
        response = self.client.get(f"/people/{special_person.id}/")
        serializer = PersonSerializer(special_person, many=False)

        self.assertEqual(response.data, serializer.data)

These tests, in my mind, are not strictly necessary. They simply test that DRF does what it will do. The best reason I can think of to add these tests is to make sure your URLs and serializers work as expected.

Now let’s add a bit of functionality to our viewset. I like DRF’s @action decorator – it’s not strictly speaking RESTful and all that, but the way it works makes sense to me. Let’s add an action to our PersonViewSet:

				
					@action(methods=['GET'], detail=True)
def greet(self, request, pk):
    person = Person.objects.get(pk=pk)
    return Response(data=person.get_greeting())

There are some obvious problems with this method – what would happen if the person does not exist? Let’s not worry about that right now. Let’s pretend we didn’t think of that. Let’s write a test for this in our test_views.py:

				
					@patch('djangotesting.api.models.Person.get_greeting')
def test_get_person_greeting_should_call_person_greeting(self, get_greeting):
    special_person = Person.objects.create(first_name="Penn", last_name="Teller", job_title="magician")
    get_greeting.return_value = f"Hello!"

    response = self.client.get(f"/people/{special_person.id}/greet/")

    self.assertEqual(response.data, "Hello!")
    self.assertEqual(response.status_code, 200)

And the test passes. Great! Now imagine you get an email from your angry client who paid a lot of money for this application, saying that the greetings are sometimes not working and they even give you an example. The greeting endpoint being called is this: /people/99999/greet/

First thing you might do is check for the existence of a person with that ID in the database and find that there is not such record. Before fixing the bug, though, let’s write a test to reproduce the bug.

				
					def test_get_non_existent_person_greeting_should_return_not_found(self):
    response = self.client.get(f"/people/99999/greet/")

    self.assertEqual(response.status_code, 404)

And the test will fail, because your action endpoint will produce a 500, not a 404. Now let’s fix the endpoint:

				
					@action(methods=['GET'], detail=True)
def greet(self, request, pk):
    try:
        person = Person.objects.get(pk=pk)
        return Response(data=person.get_greeting())
    except Person.DoesNotExist:
        return Response(data=None, status=status.HTTP_404_NOT_FOUND)

And the test will pass again.

We also need to be able to filter people by their job titles, so add the following to the PersonViewSet:

filterset_fields = ('job_title',)

And the following test to make sure our filters do what we want:

				
					def test_filter_by_job_title_should_return_filtered_results(self):
    response = self.client.get("/people/?job_title=developer")

    self.assertEqual(len(response.data), 1)
    self.assertEqual(response.data[0].get("job_title", None), "developer")

Remember that, in our setup method, we created one Person as a developer. This test is not strictly speaking necessary either – it’s testing something that Django does for us already. This is also a brittle test – it could easily be broken if someone changes the setup method. At least we learned something, though.

Let’s do one more action for good measure. We’ve received a new requirement from the client: We need to be able to change any person’s job title to developer. Now to be fair, we could just use a PATCH request for this, but for the purpose of this post and learning about tests, we’ll add another action. In this case, we’ll write the test first.

In our tests file:

				
					def test_make_developer(self):
    non_developer = Person.objects.create(first_name="Penn", last_name="Teller", job_title="magician")
    response = self.client.put(f"/people/{non_developer.id}/make_developer/")
    
    developer = Person.objects.get(id=non_developer.id)
    
    self.assertEqual(response.status_code, 200)
    self.assertEqual(developer.job_title, "developer")

The test will instantly fail, because the status code is 404. Let’s fix that in our view:

				
					@action(methods=['PUT'], detail=True)
def make_developer(self, request, pk):
    try:
        person = Person.objects.get(pk=pk)
        person.job_title = "developer"
        person.save()
        serializer = PersonSerializer(person, many=False)

        return Response(data=serializer.data, status=status.HTTP_200_OK)
    except Person.DoesNotExist:
        return Response(data=None, status=status.HTTP_404_NOT_FOUND)

Since we’re already aware of the previous bug we encountered where a person might not be found if they don’t exist, we’ll code defensively for that to begin with – there is no real need to test that again.

ALL DONE. FOR NOW.

Not all the tests we wrote here are really necessary, but the point is that we need to know how to write tests. In such a small application, it’s not strictly speaking productive to write so many tests either, but most well-tested applications will end up having more tests than functional code, which is fine.

You can find the completed repo here.

TO TDD OR NOT TO TDD?

Test driven development has been a hot topic for years. I don’t care much for it, but I do see the value in it. Once you get good at TDD, your code does become cleaner, more compact and less error-prone. However, TDD is not the only way to do things. I personally prefer a healthy mix of writing tests before and after I write code. It all depends on what you are coding, and it takes time to find the right balance.