puppeteer-request-spy

With puppeteer-request-spy you can easily watch, fake, modify or block requests from puppeteer matching patterns.

• allows you to write tests verifying specific resources are loaded as expected
• allows you to exclude unneeded requests from tests, speeding them up significantly
• allows you to alter requests and responses with custom content and http status
• avoids conflicts resulting from already aborted / continued or responded requests

Install

npm install puppeteer-request-spy --save-dev

Table Of Content

• Spying On Requests
• Altering Requests
  • Modifying Requests
  • Redirecting Requests
  • Blocking Requests
• Altering Responses
  • Faking Responses
  • Modifying Responses
• Asynchronous Options
• Interception Order
• Advanced Usage
• Full API

Usage

Spying On Requests With A KeywordMatcher

First create a new RequestInterceptor with a matcher function and an optional logger.

function KeywordMatcher(testee, keyword) {
    return testee.indexOf(keyword) > -1; 
}

let requestInterceptor = new RequestInterceptor(KeywordMatcher, console);

Next create a new RequestSpy with a pattern to be matched against all requests.

let imageSpy = new RequestSpy('/pictures');

The RequestSpy needs to be registered with the RequestInterceptor.

requestInterceptor.addSpy(imageSpy);

To use the puppeteer's request event the RequestInterception flag on the page object has to be set to true.

await page.setRequestInterception(true);

The RequestInterceptor must be registered with puppeteer.

page.on('request', requestInterceptor.intercept.bind(requestInterceptor));

After puppeteer's page object finished navigating to any page, you can query the RequestSpy.

await page.goto('https://www.example.com');

assert.ok(!imageSpy.getMatchedRequests()[0].failure());
assert.ok(imageSpy.hasMatch() && imageSpy.getMatchCount() > 0);

When all responses have been loaded you can also query the response of any matched Request. You can ensure all responses have been loaded by using the networkidle0 option. For further information check the official puppeteer API.

for (let match of imagesSpy.getMatchedRequests()) {
    assert.ok(match.response().ok());
}

Note

Since unhandled Promise rejections causes the node process to keep running after test failure, the RequestInterceptor will catch and log puppeteer's exception, if the requestInterception flag is not set.

Altering Requests

Modifying Requests

Intercepted requests can be modified by passing an overrides object to the RequestModifier. The response overrides have to match the Overrides object as specified in the official puppeteer API.

let requestModifier = new RequestModifier('/ajax/some-post-request', {
  url: '/ajax/some-get-request',
  method: 'GET',
  postData: '',
  headers: {}
});                  

requestInterceptor.addRequestModifier(requestModifier);

Redirecting Requests

If you just want to replace the url of an intercepted request, you can use the RequestRedirector.

let requestRedirector = new RequestRedirector('/ajax/some-request', 'some/new/url');                  

requestInterceptor.addRequestModifier(requestRedirector);

The RequestRedirector uses the IRequestModifier interface.

Blocking Requests

Optionally you can add patterns to block requests. Blocking requests speeds up page load since no data is loaded. Blocking requests takes precedence over overriding requests or faking responses, so any request blocked will not be replaced even when matching a ResponseFaker. Blocked or faked requests will still be counted by a RequestSpy with a matching pattern.

requestInterceptor.block(['scripts', 'track', '.png']);      

Altering Responses

Faking Responses

The response of intercepted requests can be replaced by adding a ResponseFaker to the RequestInterceptor. The fake response has to match the Response object as specified in the official puppeteer API.

let responseFaker = new ResponseFaker('/ajax/some-request', {
    status: 200,
    contentType: 'application/json',
    body: JSON.stringify({successful: false, payload: []})
});

requestInterceptor.addFaker(responseFaker);

For further details on how to replace different formats of data like images, text or html, please refer to the examples provided in the github repository.

Modifying Responses

It's also possible to replace the content of a response instead of replacing it:

let responseModifier = new ResponseModifier('/ajax/some-request', (response, request) => {
    return response.replace('</head>', '<script></script></head>');
});

requestInterceptor.addFaker(responseModifier);

Note:

The request is currently loaded in the node environment, not the browser environment.

The ResponseModifier uses the IResponseFaker interface.

Asynchronous Options

All ResponseFakers and ResponseModifiers now accept a callback for resolving the passed options. This callback can also be async or return a promise.

// static options
let requestRedirector = new RequestRedirector(
    '/ajax/some-request',
    'some/other/url'
);                  

// callback options
let requestModifier = new RequestModifier(
    '/ajax/some-request',
    (matchedRequest) => ({url: '/ajax/some-different-request'})
);  
  
// async callback options
let requestRedirector = new RequestRedirector(
    '/ajax/some-request',
     async (matchedRequest) => 'some/new/url'
);   
               
// promise callback options
let responseFaker = new ResponseFaker(
    '/ajax/some-request',
    (matchedRequest) => Promise.resolve(({
        status: 200, 
        contentType: 'application/json',
        body: JSON.stringify({successful: false, payload: []})
    }))
);                                                        

RequestInterceptor Request Interception Order:

Advanced Usage

As long as you follow the interfaces provided in the github repository you can create your own Spies, Fakers, Modifiers or Blocker.

let interceptor = new RequestInterceptor(
    (testee, pattern) => testee.indexOf(pattern) > -1
);

let count = 0;
interceptor.addSpy({
    isMatchingRequest: (_request, _matcher) => true,
    addMatch: (_request) => count++
});

interceptor.addFaker({
    isMatchingRequest: (_request, _matcher) => true,
    getResponseFake: (request) => ({body: ''})
});

interceptor.addRequestModifier({
    isMatchingRequest: (_request, _matcher) => true,
    getOverride: (interceptedRequest) => ({url: ''})
});

interceptor.setRequestBlocker({
    shouldBlockRequest: (_request, _matcher) => true,
    clearUrlsToBlock: () => undefined,
    addUrlsToBlock: (urlsToBlock) => undefined
});

Minimatch

puppeteer-request-spy works great with minimatch, it can be passed as the matcher function.

const minimatch = require('minimatch');

let cssSpy = new RequestSpy('**/*.css');
let responseFaker = new ResponseFaker('**/*.jpg', someFakeResponse);
                                                        
let requestInterceptor = new RequestInterceptor(minimatch);  
requestInterceptor.addFaker(responseFaker);
requestInterceptor.addSpy(cssSpy);   
requestInterceptor.block('**/*.js');

await page.setRequestInterception(true);
page.on('request', requestInterceptor.intercept.bind(requestInterceptor));  
await page.goto('https://www.example.com');
                                                 
assert.ok(cssSpy.hasMatch() && cssSpy.getMatchCount() > 0);
for (let matchedRequest of cssSpy.getMatchedRequests()) {
    assert.ok(matchedRequest.response().status() === 200);
}

API

Full API can be found here.

Examples

There are some usage examples included in the github repository. Check them out to get started with writing a simple test with puppeteer and puppeteer-request-spy.

Related

• minimatch - For easily matching path-like strings to patterns.
• puppeteer - Control chrome in headless mode with puppeteer for automated testing.

License

MIT