Advanced Argument Expectations

Most of our examples of working with Scenario expectations are of exact matches, e.g.

s.classroom.set('A', index=9, name='Alpha')

Means we expect the .set() method to be called on the fake object Fake('classroom') with these exact arguments: a positional argument with the value 'A', and two keyword arguments index=9, name='Alpha'.

In most cases this is exactly what we want. However, sometimes we want something else. Let’s see what Testix supports.

Ignoring A Specific Argument

Sometimes we don’t care about the value of a specific argument. For example, we might want to verify that our code calls time.sleep() but we don’t want to test for a specific number of seconds.

Here’s how to do this in Testix:

with Scenario() as s:
    s.time.sleep(IgnoreArgument())
    # must call time.sleep() with one argument, but any value for this argument will be OK

You can also use this in a kwarg expectation

with Scenario() as s:
    s.greeter.greet('hello!', person=IgnoreArgument())
    # must use 'hello!', but any value for person will be OK

Ignoring All Call Details

What if we want to make sure a method is called, but we don’t care about the arguments at all? This is what IgnoreCallDetails() is for, e.g. this test expects the .connect() method to be called on the fake object Fake('database') three times, but specifies that it doesn’t care about the exact arguments:

from testix import *

import server

def test_person_connects_somehow():
    with Scenario() as s:
        s.database.connect(IgnoreCallDetails())
        s.database.connect(IgnoreCallDetails())
        s.database.connect(IgnoreCallDetails())

        tested = server.Server(Fake('database'))
        tested.connect1()
        tested.connect2()
        tested.connect3()

Here is an example of code that passes this test

class Server:
    def __init__(self, database):
        self._database = database

    def connect1(self):
        self._database.connect()

    def connect2(self):
        self._database.connect(1, 2, 3, x='y')

    def connect3(self):
        self._database.connect(a='1', b='2', c='3')

As you can see, the .connect() method is called three times, but with different arguments each time, and this satisfies the IgnoreCallDetails() expectation.

Testing for Object Identity

Sometimes we want to ensure a method is called with a specific object. We are not satisfied with it being called with an equal object, we want the same actual object. That is, we are interested in testing actual is expected and not actual == expected.

We can do this with ArgumentIs. Here is an example:

import pytest
from testix import *

import classroom

def test_this_will_pass():
    joe = classroom.Person('Joe')
    with Scenario() as s:
        s.mylist.append(ArgumentIs(joe))

        tested = classroom.Classroom(Fake('mylist'))
        tested.enter_original(joe)

def test_this_will_fail():
    joe = classroom.Person('Joe')
    with Scenario() as s:
        s.mylist.append(ArgumentIs(joe))

        tested = classroom.Classroom(Fake('mylist'))
        tested.enter_copy(joe)

We list here two tests, both demand that the object passed to mylist.append() will be the actual object joe created at the start of the test. The test test_this_will_fail() is made to fail on purpose by using the wrong method on the tested object, this is the code that passed (and fails) these tests:

class Person:
    def __init__(self, name):
        self.name = name

    def __eq__(self, other):
        return self.name == other.name

class Classroom:
    def __init__(self, people: list):
        self._people = people

    def enter_original(self, student):
        self._people.append(student)

    def enter_copy(self, student):
        self._people.append(Person(student.name))

If we didn’t use ArgumentIs and just used

s.mylist.append(joe)

The test_this_will_fail() test would have passed, because joe is equal to the object passed to .append() as defined by the __eq__ method. With ArgumentIs, however, you will get something like this failure message:

E       testix: ExpectationException
E       testix details:
E       === Scenario (no title) ===
E        expected: mylist.append(|IS <classroom.Person object at 0x7f5162c2c5e0>|)
E        actual  : mylist.append(<classroom.Person object at 0x7f5162c2c250>)

Capturing Arguments

Sometimes we don’t want to demand anything about a method’s arguments, but we do want to capture them. This is useful for when we want to simulate the triggering of an internal callback.

For example, suppose we have a class which implements some logic when the process ends via an atexit <https://docs.python.org/3/library/atexit.html>_ handler. Testing this might seem hard, since we don’t want to actually make the process (which is running our test) exit.

Here’s how to do it using Testix’s SaveArgument feature.

from testix import *
import pytest
import robot

@pytest.fixture
def mock_imports(patch_module):
    patch_module(robot, 'atexit') # mock atexit module

def test_atexit_handler(mock_imports):
    with Scenario() as s:
        s.atexit.register(saveargument.SaveArgument('the_handler'))

        tested = robot.Robot(cleanup_func=Fake('cleanup_logic'))

        handler = saveargument.saved()['the_handler']
        s.cleanup_logic(1, 2, 3)

        handler()

We use saveargument.SaveArgument() to capture the argument passed to atexit.register(), and name this captured argument the_handler.

We later retrieve the captured callback via the saveargument.saved() dictionary.

This enables us to trigger the callback ourselves by calling handler() - which satisfies our demand s.cleanup_logic(1, 2, 3).

NOTE a simpler way might have been to make the _cleanup() function public, and then we could just call it: tested.cleanup(). However, if this is not called in our code, we should not make it public, and since we are forbidden by good ethics from accessing a private function from outside the class, we need to capture it.

Implementing Arbitrary Argument Matching

Sometimes you need some complicated logic that Testix doesn’t support out of the box.

You can define your own argument expectation classes with some arbitrary logic, and use them in your tests, by implementing classes derived from the ArgumentExpectation base class, which is essentially an interface:

class ArgumentExpectation:
    def __init__(self, value):
        self.expectedValue = value

    def ok(self, value):
        # returns true if value meets the expectation, false otherwise
        raise Exception("must override this")

The following tests implements a new StartsWith expectation, which expects a string that starts with a given prefix.

import pytest
from testix import *

import temporary_storage

@pytest.fixture(autouse=True)
def mock_builtin(patch_module):
    patch_module(temporary_storage, 'open')

class StartsWith(ArgumentExpectation):
    def ok(self, value):
        return value.startswith(self.expectedValue)

def test_person_connects_somehow():
    with Scenario() as s:
        s.open(StartsWith('/tmp/'), 'w') >> Fake('the_file')
        s.the_file.write('file_name: some_file\n\n')

        tested = temporary_storage.TemporaryStorage()
        the_file = tested.create_file('some_file')
        assert the_file is Fake('the_file')

We demand that open() be called with a filename that starts with /tmp/, and with exactly 'w' as the second argument.

This code passes this test:

class TemporaryStorage:
    def create_file(self, filename):
        f = open('/tmp/' + filename, 'w')
        f.write('file_name: ' + filename + '\n\n')
        return f

The ArgumentExpectation base class implements a default __repr__ function, but you can implement one yourself e.g.

class StartsWith(ArgumentExpectation):
    def ok(self, value):
        return value.startswith(self.expectedValue)

    def __repr__(self):
        return 'StartsWith({})'.format(repr(self.expectedValue))

Using this ArgumentExpectation interface, you can make Testix support any arbitrary and complicated argument verification you need.