Skip to main content

HTTP testing in R

13 CRAN- (and Bioconductor) preparedness for your tests

There is no one right answer to how to manage your tests for CRAN, except that you do want a clean check result on CRAN at all times. This probably applies to Bioconductor too. The following is a discussion of the various considerations - which should give you enough information to make an educated decision.

13.1 Running tests on CRAN?

You can run vcr/httptest/httptest2/webfakes enabled tests on CRAN. CRAN is okay with files associated with tests, and so in general on CRAN you can run your tests that use cassettes, mock files or recorded responses on CRAN. Another aspect is the presence of dependencies: make sure the HTTP testing package you use is listed as Suggests dependency in DESCRIPTION! With webfakes this might mean also listing optional dependencies in DESCRIPTION. With webfakes, your tests if run on CRAN should not assume the availability of a given port.

When running HTTP tests on CRAN, be aware of a few things:

  • If your tests require any secret environment variables or R options (apart from the “foobar” ones used to fool your package when using a saved response), they won’t be available on CRAN. In these cases you likely want to skip these tests with testthat::skip_on_cran().
  • If your tests have cassettes, mock files or recorded responses with sensitive information in them, you probably do not want to have those cassettes on the internet, in which case you won’t be running vcr enabled tests on CRAN either. In the case of sensitive information, you might want to Rbuildignore the cassettes, mock files or recorded responses (and to gitignore them or make your package development repository private).
  • There is a maximal size for package sources so you will want your cassettes, mock files or recorded responses to not be too big. There are three ways to limit their size
    • Make requests that do not generate a huge response (e.g. tweak the time range);
    • Edit the recorded responses (why not even copy-paste responses from the API docs as those are often short) — see vcr docs about editing cassettes for pros and cons;
    • Share cassettes / mock files / recorded responses between tests.

Do not compress cassettes, mock files or recorded responses: CRAN submissions are already compressed; compressed files will make git diffs hard to use.

13.2 Skipping a few tests on CRAN?

If you are worried at all about problems with HTTP tests on CRAN you can use testthat::skip_on_cran() to skip specific tests. Make sure your tests run somewhere else (on continuous integration) regularly!

We’d recommend not running tests making real requests on CRAN, even when interacting with an API without authentication.

13.3 Skipping all tests on CRAN?

If you have a good continuous integration setup (several operating systems, scheduled runs, etc.) why not skip all tests on CRAN?

13.4 Stress-test your package

To stress-test your package before a CRAN submission, use rhub::check_for_cran() without passing any environment variable to the function, and use WinBuilder.