Welcome to requests-mock’s documentation!

Contents:

Overview

The requests library has the concept of pluggable transport adapters. These adapters allow you to register your own handlers for different URIs or protocols.

The requests-mock library at its core is simply a transport adapter that can be preloaded with responses that are returned if certain URIs are requested. This is particularly useful in unit tests where you want to return known responses from HTTP requests without making actual calls.

As the requests library has very limited options for how to load and use adapters requests-mock also provides a number of ways to make sure the mock adapter is used. These are only loading mechanisms, they do not contain any logic and can be used as a reference to load the adapter in whatever ways works best for your project.

Using the Mocker

The mocker is a loading mechanism to ensure the adapter is correctly in place to intercept calls from requests. It’s goal is to provide an interface that is as close to the real requests library interface as possible.

Activation

Loading of the Adapter is handled by the requests_mock.Mocker class, which provides two ways to load an adapter:

Context Manager

The Mocker object can work as a context manager.

>>> import requests
>>> import requests_mock

>>> with requests_mock.Mocker() as m:
...     m.get('http://test.com', text='resp')
...     requests.get('http://test.com').text
...
'resp'

Decorator

Mocker can also be used as a decorator. The created object will then be passed as the last positional argument.

>>> @requests_mock.Mocker()
... def test_function(m):
...     m.get('http://test.com', text='resp')
...     return requests.get('http://test.com').text
...
>>> test_function()
'resp'

If the position of the mock is likely to conflict with other arguments you can pass the kw argument to the Mocker to have the mocker object passed as that keyword argument instead.

>>> @requests_mock.Mocker(kw='mock')
... def test_kw_function(**kwargs):
...     kwargs['mock'].get('http://test.com', text='resp')
...     return requests.get('http://test.com').text
...
>>> test_kw_function()
'resp'

Class Decorator

Mocker can also be used to decorate a whole class. It works exactly like in case of decorating a normal function. When used in this way they wrap every test method on the class. The mocker recognise methods that start with test as being test methods. This is the same way that the unittest.TestLoader finds test methods by default. It is possible that you want to use a different prefix for your tests. You can inform the mocker of the different prefix by setting requests_mock.Mocker.TEST_PREFIX:

>>> requests_mock.Mocker.TEST_PREFIX = 'foo'
>>>
>>> @requests_mock.Mocker()
... class Thing(object):
...     def foo_one(self, m):
...        m.register_uri('GET', 'http://test.com', text='resp')
...        return requests.get('http://test.com').text
...     def foo_two(self, m):
...         m.register_uri('GET', 'http://test.com', text='resp')
...         return requests.get('http://test.com').text
...
>>>
>>> Thing().foo_one()
'resp'
>>> Thing().foo_two()
'resp'

This behavior mimics how patchers from mock library works.

Methods

The mocker object can be used with a similar interface to requests itself.

>>> with requests_mock.Mocker() as mock:
...     mock.get('http://test.com', text='resp')
...     requests.get('http://test.com').text
...
'resp'

The functions exist for the common HTTP method:

  • delete()
  • get()
  • head()
  • options()
  • patch()
  • post()
  • put()

As well as the basic:

  • request()
  • register_uri()

These methods correspond to the HTTP method of your request, so to mock POST requests you would use the post() function. Futher information about what can be matched from a request can be found at Request Matching

Real HTTP Requests

The Mocker object takes the following parameters:

real_http (bool):
 If True then any requests that are not handled by the mocking adapter will be forwarded to the real server. Defaults to False.
>>> with requests_mock.Mocker(real_http=True) as m:
...     m.register_uri('GET', 'http://test.com', text='resp')
...     print(requests.get('http://test.com').text)
...     print(requests.get('http://www.google.com').status_code)  
...
'resp'
200

New in 1.1

Similarly when using a mocker you can register an individual URI to bypass the mocking infrastructure and make a real request. Note this only works when using the mocker and not when directly mounting an adapter.

>>> with requests_mock.Mocker() as m:
...     m.register_uri('GET', 'http://test.com', text='resp')
...     m.register_uri('GET', 'http://www.google.com', real_http=True)
...     print(requests.get('http://test.com').text)
...     print(requests.get('http://www.google.com').status_code)  
...
'resp'
200

Request Matching

Whilst it is preferable to provide the whole URI to requests_mock.Adapter.register_uri() it is possible to just specify components.

The examples in this file are loaded with:

>>> import requests
>>> import requests_mock
>>> adapter = requests_mock.Adapter()
>>> session = requests.Session()
>>> session.mount('mock', adapter)

Note

The examples within use this syntax because request matching is a function of the adapter and not the mocker. All the same arguments can be provided to the mocker if that is how you use requests_mock within your project, and use the

mock.get(url, ...)

form in place of the given:

adapter.register_uri('GET', url, ...)

Note

By default all matching is case insensitive. This can be adjusted by passing case_sensitive=True when creating a mocker or adapter or globally by doing:

requests_mock.mock.case_sensitive = True

for more see: Case Insensitivity

Simple

The most simple way to match a request is to register the URL and method that will be requested with a textual response. When a request is made that goes through the mocker this response will be retrieved.

.. >>> adapter.register_uri('GET', 'mock://test.com/path', text='resp')
.. >>> session.get('mock://test.com/path').text
.. 'resp'

Path Matching

You can specify a protocol-less path:

.. >>> adapter.register_uri('GET', '//test.com/', text='resp')
.. >>> session.get('mock://test.com/').text
.. 'resp'

or you can specify just a path:

.. >>> adapter.register_uri('GET', '/path', text='resp')
.. >>> session.get('mock://test.com/path').text
.. 'resp'
.. >>> session.get('mock://another.com/path').text
.. 'resp'

Query Strings

Query strings provided to a register will match so long as at least those provided form part of the request.

>>> adapter.register_uri('GET', '/7?a=1', text='resp')
>>> session.get('mock://test.com/7?a=1&b=2').text
'resp'

If any part of the query string is wrong then it will not match.

>>> session.get('mock://test.com/7?a=3')
Traceback (most recent call last):
   ...
requests_mock.exceptions.NoMockAddress: No mock address: GET mock://test.com/7?a=3

This can be a problem in certain situations, so if you wish to match only the complete query string there is a flag complete_qs:

>>> adapter.register_uri('GET', '/8?a=1', complete_qs=True, text='resp')
>>> session.get('mock://test.com/8?a=1&b=2')
Traceback (most recent call last):
   ...
requests_mock.exceptions.NoMockAddress: No mock address: GET mock://test.com/8?a=1&b=2

Matching ANY

There is a special symbol at requests_mock.ANY which acts as the wildcard to match anything. It can be used as a replace for the method and/or the URL.

>>> adapter.register_uri(requests_mock.ANY, 'mock://test.com/8', text='resp')
>>> session.get('mock://test.com/8').text
'resp'
>>> session.post('mock://test.com/8').text
'resp'
>>> adapter.register_uri(requests_mock.ANY, requests_mock.ANY, text='resp')
>>> session.get('mock://whatever/you/like').text
'resp'
>>> session.post('mock://whatever/you/like').text
'resp'

Regular Expressions

URLs can be specified with a regular expression using the python re module. To use this you should pass an object created by re.compile().

The URL is then matched using re.regex.search() which means that it will match any component of the url, so if you want to match the start of a URL you will have to anchor it.

.. >>> import re
.. >>> matcher = re.compile('tester.com/a')
.. >>> adapter.register_uri('GET', matcher, text='resp')
.. >>> session.get('mock://www.tester.com/a/b').text
.. 'resp'

If you use regular expression matching then requests-mock can’t do it’s normal query string or path only matching, that will need to be part of the expression.

Request Headers

A dictionary of headers can be supplied such that the request will only match if the available headers also match. Only the headers that are provided need match, any additional headers will be ignored.

>>> adapter.register_uri('POST', 'mock://test.com/headers', request_headers={'key': 'val'}, text='resp')
>>> session.post('mock://test.com/headers', headers={'key': 'val', 'another': 'header'}).text
'resp'
>>> resp = session.post('mock://test.com/headers')
Traceback (most recent call last):
   ...
requests_mock.exceptions.NoMockAddress: No mock address: POST mock://test.com/headers

Additional Matchers

As distinct from Custom Matching below we can add an additional matcher callback that lets us do more dynamic matching in addition to the standard options. This is handled by a callback function that takes the request as a parameter:

>>> def match_request_text(request):
...     # request.text may be None, or '' prevents a TypeError.
...     return 'hello' in (request.text or '')
...
>>> adapter.register_uri('POST', 'mock://test.com/additional', additional_matcher=match_request_text, text='resp')
>>> session.post('mock://test.com/headers', data='hello world').text
'resp'
>>> resp = session.post('mock://test.com/additional', data='goodbye world')
Traceback (most recent call last):
   ...
requests_mock.exceptions.NoMockAddress: No mock address: POST mock://test.com/additional

Using this mechanism lets you do custom handling such as parsing yaml or XML structures and matching on features of that data or anything else that is not directly handled via the provided matchers rather than build in every possible option to requests_mock.

Custom Matching

Internally calling register_uri() creates a matcher object for you and adds it to the list of matchers to check against.

A matcher is any callable that takes a requests.Request and returns a requests.Response on a successful match or None if it does not handle the request.

If you need more flexibility than provided by register_uri() then you can add your own matcher to the Adapter. Custom matchers can be used in conjunction with the inbuilt matchers. If a matcher returns None then the request will be passed to the next matcher as with using register_uri().

>>> def custom_matcher(request):
...     if request.path_url == '/test':
...         resp = requests.Response()
...         resp.status_code = 200
...         return resp
...     return None
...
>>> adapter.add_matcher(custom_matcher)
>>> session.get('mock://test.com/test').status_code
200
>>> session.get('mock://test.com/other')
Traceback (most recent call last):
   ...
requests_mock.exceptions.NoMockAddress: No mock address: POST mock://test.com/other

Creating Responses

Note

The examples within use this syntax because response creation is a function of the adapter and not the mocker. All the same arguments can be provided to the mocker if that is how you use requests_mock within your project, and use the

mock.get(url, ...)

form in place of the given:

adapter.register_uri('GET', url, ...)

Registering Responses

Responses are registered with the requests_mock.Adapter.register_uri() function on the adapter.

>>> adapter.register_uri('GET', 'mock://test.com', text='Success')
>>> resp = session.get('mock://test.com')
>>> resp.text
'Success'

register_uri() takes the HTTP method, the URI and then information that is used to build the response. This information includes:

status_code:The HTTP status response to return. Defaults to 200.
reason:The reason text that accompanies the Status (e.g. ‘OK’ in ‘200 OK’)
headers:A dictionary of headers to be included in the response.
cookies:A CookieJar containing all the cookies to add to the response.

To specify the body of the response there are a number of options that depend on the format that you wish to return.

json:A python object that will be converted to a JSON string.
text:A unicode string. This is typically what you will want to use for regular textual content.
content:A byte string. This should be used for including binary data in responses.
body:A file like object that contains a .read() function.
raw:A prepopulated urllib3.response.HTTPResponse to be returned.
exc:An exception that will be raised instead of returning a response.

These options are named to coincide with the parameters on a requests.Response object. For example:

>>> adapter.register_uri('GET', 'mock://test.com/1', json={'a': 'b'}, status_code=200)
>>> resp = session.get('mock://test.com/1')
>>> resp.json()
{'a': 'b'}

>>> adapter.register_uri('GET', 'mock://test.com/2', text='Not Found', status_code=404)
>>> resp = session.get('mock://test.com/2')
>>> resp.text
'Not Found'
>>> resp.status_code
404

It only makes sense to provide at most one body element per response.

Dynamic Response

A callback can be provided in place of any of the body elements. Callbacks must be a function in the form of

def callback(request, context):

and return a value suitable to the body element that was specified. The elements provided are:

request:The requests.Request object that was provided.
context:An object containing the collected known data about this response.

The available properties on the context are:

headers:The dictionary of headers that are to be returned in the response.
status_code:The status code that is to be returned in the response.
reason:The string HTTP status code reason that is to be returned in the response.
cookies:A requests_mock.CookieJar of cookies that will be merged into the response.

These parameters are populated initially from the variables provided to the register_uri() function and if they are modified on the context object then those changes will be reflected in the response.

>>> def text_callback(request, context):
...     context.status_code = 200
...     context.headers['Test1'] = 'value1'
...     return 'response'
...
>>> adapter.register_uri('GET',
...                      'mock://test.com/3',
...                      text=text_callback,
...                      headers={'Test2': 'value2'},
...                      status_code=400)
>>> resp = session.get('mock://test.com/3')
>>> resp.status_code, resp.headers, resp.text
(200, {'Test1': 'value1', 'Test2': 'value2'}, 'response')

Response Lists

Multiple responses can be provided to be returned in order by specifying the keyword parameters in a list. If the list is exhausted then the last response will continue to be returned.

>>> adapter.register_uri('GET', 'mock://test.com/4', [{'text': 'resp1', 'status_code': 300},
...                                                   {'text': 'resp2', 'status_code': 200}])
>>> resp = session.get('mock://test.com/4')
>>> (resp.status_code, resp.text)
(300, 'resp1')
>>> resp = session.get('mock://test.com/4')
>>> (resp.status_code, resp.text)
(200, 'resp2')
>>> resp = session.get('mock://test.com/4')
>>> (resp.status_code, resp.text)
(200, 'resp2')

Callbacks work within response lists in exactly the same way they do normally;

>>> adapter.register_uri('GET', 'mock://test.com/5', [{'text': text_callback}]),
>>> resp = session.get('mock://test.com/5')
>>> resp.status_code, resp.headers, resp.text
(200, {'Test1': 'value1', 'Test2': 'value2'}, 'response')

Raising Exceptions

When trying to emulate a connection timeout or SSLError you need to be able to throw an exception when a mock is hit. This can be achieved by passing the exc parameter instead of a body parameter.

>>> adapter.register_uri('GET', 'mock://test.com/6', exc=requests.exceptions.ConnectTimeout),
>>> session.get('mock://test.com/6')
Traceback (most recent call last):
   ...
ConnectTimeout:

Handling Cookies

Whilst cookies are just headers they are treated in a different way, both in HTTP and the requests library. To work as closely to the requests library as possible there are two ways to provide cookies to requests_mock responses.

The most simple method is to use a dictionary interface. The Key and value of the dictionary are turned directly into the name and value of the cookie. This method does not allow you to set any of the more advanced cookie parameters like expiry or domain.

>>> adapter.register_uri('GET', 'mock://test.com/7', cookies={'foo': 'bar'}),
>>> resp = session.get('mock://test.com/7')
>>> resp.cookies['foo']
'bar'

The more advanced way is to construct and populate a cookie jar that you can add cookies to and pass that to the mocker.

>>> jar = requests_mock.CookieJar()
>>> jar.set('foo', 'bar', domain='.test.com', path='/baz')
>>> adapter.register_uri('GET', 'mock://test.com/8', cookies=jar),
>>> resp = session.get('mock://test.com/8')
>>> resp.cookies['foo']
'bar'
>>> resp.cookies.list_paths()
['/baz']

Known Issues

Case Insensitivity

By default matching is done in a completely case insensitive way. This makes sense for the protocol and host components which are defined as insensitive by RFCs however it does not make sense for path.

A byproduct of this is that when using request history the values for path, qs etc are all lowercased as this was what was used to do the matching.

To work around this when building an Adapter or Mocker you do

with requests_mock.mock(case_sensitive=True) as m:
   ...

or you can override the default globally by

requests_mock.mock.case_sensitive = True

It is recommended to run the global fix as it is intended that case sensitivity will become the default in future releases.

Note that even with case_sensitive enabled the protocol and netloc of a mock are still matched in a case insensitive way.

Request History

The object returned from creating a mock or registering a URI in an adapter is capable of tracking and querying the history of requests that this mock responded to.

Called

The easiest way to test if a request hit the adapter is to simply check the called property or the call_count property.

>>> import requests
>>> import requests_mock

>>> with requests_mock.mock() as m:
...     m.get('http://test.com, text='resp')
...     resp = requests.get('http://test.com')
...
>>> m.called
True
>>> m.call_count
1

Request Objects

The history of objects that passed through the mocker/adapter can also be retrieved

>>> history = m.request_history
>>> len(history)
1
>>> history[0].method
'GET'
>>> history[0].url
'http://test.com/'

The alias last_request is also available for the last request to go through the mocker.

This request object is a wrapper around a standard requests.Request object with some additional information that make the interface more workable (as the Request object is generally not dealt with by users.

These additions include:

text:The data of the request converted into a unicode string.
json:The data of the request loaded from json into python objects.
qs:The query string of the request. See urllib.parse.parse_qs() for information on the return format.
hostname:The host name that the request was sent to.
port:The port the request was sent to.
>>> m.last_request.qs.scheme
'http'
>>> m.last_request.qs.netloc
'test.com'

The following parameters of the requests.request() call are also exposed via the request object:

timeout:How long to wait for the server to send data before giving up.
allow_redirects:
 Set to True if POST/PUT/DELETE redirect following is allowed.
proxies:Dictionary mapping protocol to the URL of the proxy.
verify:whether the SSL cert will be verified.
cert:The client certificate or cert/key tuple for this request.

Note: That the default value of these attributes are the values that are passed to the adapter and not what is passed to the request method. This means that the default for allow_redirects is None (even though that is interpretted as True) if unset, whereas the defautl for verify is True, and the default for proxies the empty dict.

Adapter Usage

Creating an Adapter

The standard requests means of using an adapter is to mount() it on a created session. This is not the only way to load the adapter, however the same interactions will be used.

>>> import requests
>>> import requests_mock

>>> session = requests.Session()
>>> adapter = requests_mock.Adapter()
>>> session.mount('mock', adapter)

At this point any requests made by the session to a URI starting with mock:// will be sent to our adapter.

Additional Loading

Common additional loading mechanism are supported in the requests_mock.contrib module.

These modules may require dependencies outside of what is provided by requests_mock and so must be provided by the including application.

Fixtures

Fixtures provide a way to create reusable state and helper methods in test cases.

To use the requests-mock fixture your tests need to have a dependency on the fixtures library. This can be optionally installed when you install requests-mock by doing:

pip install requests-mock[fixture]

The fixture mocks the requests.Session.get_adapter() method so that all requests will be served by the mock adapter.

The fixture provides the same interfaces as the adapter.

>>> import requests
>>> from requests_mock.contrib import fixture
>>> import testtools

>>> class MyTestCase(testtools.TestCase):
...
...     TEST_URL = 'http://www.google.com'
...
...     def setUp(self):
...         super(MyTestCase, self).setUp()
...         self.requests_mock = self.useFixture(fixture.Fixture())
...         self.requests_mock.register_uri('GET', self.TEST_URL, text='respA')
...
...     def test_method(self):
...         self.requests_mock.register_uri('POST', self.TEST_URL, text='respB')
...         resp = requests.get(self.TEST_URL)
...         self.assertEqual('respA', resp.text)
...         self.assertEqual(self.TEST_URL, self.requests_mock.last_request.url)
...

Indices and tables