Load testing a RESTful API with Ruby JMeter

In this post we're going to show you how to load test a RESTful API using our popular Ruby-JMeter gem. If you only use JMeter, don't worry, the gem will also output the JMX format so you can use the examples here without the gem as well.

The Test Application

We're using an example application for this demonstration, based on code taken from a popular Railscast about API versioning.

Our application lets users create, read, update or delete products. The application can be accessed using a browser or via a RESTful API. The following product list is just a simple index view of all the products. It looks like this in HTML.

We can also make the same call using a HTTP GET to the products API endpoint with a response in JSON like this.

Using Ruby-JMeter

This gem lets you write test plans for JMeter in your favorite text editor, and optionally run them on Flood IO. It's great for users who want to skip using the JMeter GUI and express their test plans in a succinct, easy to read and shared format.

So instead of looking at something like this in JMeter:

We can write the test plan like this in Ruby-JMeter.

require 'ruby-jmeter'

test do  
  with_json
  threads 1, loops: 5 do
    get name: 'get_products_index',
        url: 'http://example-rest-api.herokuapp.com/api/products'
  end
end.run  

HTTP Verbs / Methods

Most RESTful APIs will respond to HTTP verbs (or methods) such as POST, GET, PUT, and DELETE. These normally relate to Create, Read, Update, and Delete (CRUD) operations.

Our test application supports the following routes.

GET         /api/products              api/v2/products#index  
POST     /api/products              api/v2/products#create  
GET         /api/products/:id          api/v2/products#show  
PUT         /api/products/:id          api/v2/products#update  
DELETE     /api/products/:id          api/v2/products#destroy  

So tying that all together, we can extend our Ruby-JMeter test plan to cover some of these other methods as follows.

Show all Products using GET /api/products

The HTTP GET verb is often used to retrieve (or read) a representation of a resource. We've already demonstrated the index view of our products which lists all the products in our catalog using a GET as follows.

get name: 'get_products_index', url: "#{base_url}/products"  

Create a Product using POST /api/products

The POST verb is often used for creation of new resources. In order for us to create a new product we need to provide some additional parameters for the product itself via the fill_in parameter as follows.

post  name: 'create_new_product',  
      url: "#{base_url}/products",
      fill_in: {
        "product[name]"        => 'Thomas the Tank Engine',
        "product[price]"       => 9.99,
        "product[released_on]" => Time.now
      }

We should also validate the response and extract the newly created product ID so we can use it in subsequent requests. If the request is processed without errors we should expect a HTTP/1.1 201 Created response code along with a response body in JSON that looks like this.

{
  "category_id":null,
  "created_at":"2014-05-16T02:41:46Z",
  "id":30,
  "name":"Thomas the Tank Engine",
  "price":"9.99",
  "released_on":"2014-05-16T12:41:01Z",
  "updated_at":"2014-05-16T02:41:46Z"
}

We can check for the same in our test plan using the assert and extract methods on the response body like this.

post  name: 'create_new_product',  
      ...
      } do
        assert equals: '201', test_field: 'Assertion.response_code'
        extract name: 'product_id', regex: '"id":(\d+)'
end  

We've already published a more complete guide to using JMeter Regular Expressions which might be of help. JMeter-Plugins also provide a useful JSON path extractor if you don't want to deal with regex.

Show a Product using GET /api/products/:id

Now that we've created a product, we can use its product ID extracted during the creation and use the GET verb to show the matching product in the database. We'll also assert that the product name is the same as the product we created as follows.

get name: 'get_products_show', url: "#{base_url}/products/${product_id}" do  
  assert substring: 'Thomas the Tank Engine'
end  

Update a Product using PUT /api/products/:id

Now that we've created a product, we can use its product ID extracted during the creation and use the PUT verb to update its attributes. If the request is processed without errors we should expect a HTTP/1.1 204 No Content response code along like this.

put name: 'put_products_edit',  
    raw_path: true,
    url: "#{base_url}/products/${product_id}?product[name]=Salty the Steam Engine&product[released_on]=#{Time.now}" do
      assert equals: '204', test_field: 'Assertion.response_code'
end  

Notice in this case we used the raw_path parameter in order to modify attributes via query parameters instead.

Delete a Product using DELETE /api/products/:id

Finally we can delete a product using the DELETE verb. It's as straightforward as this.

delete name: 'delete_product', url: "#{base_url}/products/${product_id}" do  
  assert equals: '204', test_field: 'Assertion.response_code'
end  

Ready for Load Testing

Once you've completed your test plan, you can scale out and run the test on distributed infrastructure in the AWS cloud using Flood IO.

As promised, if you don't want to use Ruby-JMeter you can use the JMX formatted test plan available here.

Upload your test plan using our GUI, or use our own API to start your own load test.

ruby test/performance/flood_load_test.rb  
I, [2014-05-16T13:48:50.572687 #54778]  INFO -- : Flood results at: https://flood.io/1YTVqUGoN1fH9hcRtCIIjg