skip to content
Jerrie Pelser's Blog

End-to-end API testing with Flurl HTTP testing library

/ 8 min read

flurl , testing

Introduction

I am a big fan of Flurl. I wrote previously how I use Flurl to create API clients in Cloudpress. Today, I want to talk about how I use the Flurl HTTP testing library to test all of those API clients.

Specifically, I want to focus on the tests I run that do end-to-end testing - i.e. where I get an input document from one API endpoint, do some processing, and send to result to a second API endpoint.

Background

Before I talk about how I do these end-to-end tests, I want to talk about why it is important in the context of Cloudpress.

Cloudpress allow users to export content from Google Docs and Notion to Content Management Systems (CMSs) such as WordPress, Webflow, Contentful, Sanity, and others. To simplify things, let’s take an example of exporting a Google Doc to WordPress.

When a user exports a document from Google Docs to WordPress, Cloudpress downloads the document using the Google Docs API and converts the document to an intermediate Cloudpress document format.

It then passes the document to the WordPress service which converts the document from the Cloudpress format to the WordPress Gutenberg format and call the Create Post API endpoint to create a new post on WordPress.

Since exporting content is the core function of Cloudpress, it is important for me to ensure that Cloudpress exports content correctly every time. I have created a suite of test documents in Google Docs and Notion and want to ensure that Cloudpress exports the correct content to each of the supported CMSs.

Obviously, I cannot do real exports every time I run the test suite. So, I have to mock the API requests on the Google Docs/Notion side, as well as the CMS side.

For this, I use Flurl’s HTTP testing library.

Why the Cloudpress document format?

Before I carry on, I want to talk about the need for an intermediate Cloudpress document format.

I mentioned previously that Cloudpress can export from Google Docs and Notion to various CMSs. The means that we have to write a converter from Google Docs to each of the CMS formats, and from Notion to each of the CMS formats.

To make things worse, many of the CMSs support multiple formats. For example, in Contentful you can use a Long Text field or a Rich Text field for your content. For Long Text fields we use Markdown, and for Rich Text fields we use Contentful’s proprietary Rich Text format.

Likewise, when using WordPress, a user can use the Gutenberg format or the “classic” format.

If we did direct conversions between all the formats, things would become messy.

Conversion between all the different formats

The diagram above does not even take into consideration the multiple formats for Contentful and WordPress, and you can already see it is a mess. For every target format we add it gets worse. And if we were to add a new source format, it would get much worse.

By introducing an intermediate format, we can greatly simplify things.

Converting to Cloudpress format

The Cloudpress document format is a superset of HTML. I chose this format for the following reasons:

  • Many of the CMSs already use a subset of HTML. Since our document format is so close, all I really need to do is remove elements that are invalid for the specific CMS or substitute them for different elements.
  • There are many HTML parsers available, so we would not have to write our own document parser. In the case of Markdown (used by Contentful Long Text fields), there are libraries we can use to convert from HTML to markdown
  • We can easily extend HTML with our own extensions by using data-* attributes or custom elements and the aforementioned HTML parsers will have no problem with them.

How I structured my test suites

Let’s look at how I structured my test suites. To keep things simple, I will look at a single test suite which tests the export from Google Docs to WordPress (Gutenberg).

Google input documents

In side Google Drive, I have created a test suite that contains documents for all the various scenarios I would like to test. This folder contains 30+ documents that represent the various styles and elements that Cloudpress must handle.

The Cloudpress test suite of documents inside Google Drive

This is an example of one of those documents:

The Google Doc for testing anchor text

Google Docs API responses

Inside my test project, I have a folder that contains the JSON responses from the Google Docs API endpoint. There is a JSON response representing each of the documents inside the Google Drive test folder. These JSON files are added as embedded resources to the test project.

The Google Docs test JSON responses

This is the JSON returned by the Google Docs API for the Anchor text document from before.

When the tests run, I load all of these JSON responses and use them as input for an xUnit data driven test.

WordPress API requests

On the WordPress side, I want to test that a given Google Doc will result in the correct content being sent to WordPress. Once again, I have a folder with embedded resources that represent the body of the API request I expect Cloudpress to make.

Folder containing expected WordPress API requests

This is the JSON representing the API request made to the WordPress API endpoint for the Anchor text document.

{
  "status": "draft",
  "title": "Anchor text",
  "content": "<!-- wp:paragraph -->\r\n<p>This is linking to an <a href=\"https://www.usecloudpress.com?query=1\">external website</a>.</p>\r\n<!-- /wp:paragraph -->\r\n<!-- wp:paragraph -->\r\n<p>This link (<a href=\"https://www.usecloudpress.com/\">https://www.usecloudpress.com/</a>) was automatically linked by Google Docs</p>\r\n<!-- /wp:paragraph -->\r\n<!-- wp:paragraph -->\r\n<p>A single space with a hyperlink, like this<a href=\"https://www.usecloudpress.com/\"> </a>one should be ignored when converted to markdown</p>\r\n<!-- /wp:paragraph -->\r\n"
}

Putting it all together

To tie this all together, this is what the code for these tests look like at a high level.

    [Theory]


    [MemberData(nameof(GetTestCases))]
    public async Task GoogleDocs_Exports_Correctly_To_WordPress(string testCaseName)
    {
        // Arrange


        using var httpTest = new HttpTest();


        httpTest.ForCallsTo("https://docs.googleapis.com/v1/documents/*")
            .WithVerb(HttpMethod.Get)
            .RespondWith(LoadGoogleApiHttpResponse(testCaseName));


        var googleDocsApi = new GoogleDocsApi();
        var googleDocsIntegration = new GoogleDocsIntegration(googleDocsApi);


        var wordPressApi = new WordPressApi();
        var sut = new WordPressIntegration(wordPressApi);


        var cloudpressDocument = await googleDocsIntegration.ReadAndConvertDocument(testCaseName);


        // Act
        await sut.ExportDocument(cloudpressDocument);


        // Assert


        var callToCreateContentItem = httpTest.CallLog.FirstOrDefault(call => call.Request.Url.ToString() == "https://www.my-wordpress-site.com/wp-json/wp/v2/posts" && call.Request.Verb == HttpMethod.Post);
        var expectedJson = LoadWordPressApiHttpRequest(testCaseName);
        var actualJson = callToCreateContentItem.RequestBody;
        Assert.Equal(expectedJson, actualJson);
    }
  1. The GetTestCases method scans the embedded resources in the Google Docs test folder and returns the names for each of the input documents as test case names.
  2. Create an instance of the Flurl HttpTest class which puts Flurl into test mode and allow me to fake API responses and record API requests.
  3. For calls to the Google Docs API, I respond with the content of the embedded resource containing the JSON representation of the document.
  4. I find the request to the WordPress API, extract the request body and compare it with the expected request body that is also loaded from an embedded resource.

A simplified demo

I understand that this may all still be a little bit abstract and hard to grasp. To demonstrate all of this in practice, I have put together a small demo application you can find at https://github.com/jerriepelser-blog/end-to-end-api-tests.

Why end-to-end tests?

One of the questions you may ask is why I test the flow end-to-end. Why not test the conversion from Google Doc to the Cloudpress document format and then from the Cloudpress document format to the various CMSs?

There are a few reasons:

  1. It more accurately represents the user workflow. Quite simply, a Cloudpress user does not export from Google Docs to the Cloudpress format and then from the Cloudpress format to their CMS. They export from Google Docs to their CMS. So testing that complete flow more accurately represents the user’s actual workflow.
  2. Cloudpress exports additional fields. Cloudpress does not export only the document content - it can also export additional fields to the CMS such as the author, categories, tags, etc. I want to make sure that the API call to the CMS is passing along all the correct fields.

Conclusion

In this blog posts I described the strategy I use to perform end-to-end testing between various APIs in Cloudpress. For .NET developers, I have included a small demo application that demonstrates how to put this together in practice in your own applications.

A simplified demo application can be found at https://github.com/jerriepelser-blog/end-to-end-api-tests.