Setting up a load-test with JMeter for performing SP initiated SSO's with SAML 2.0

Tuesday, November 12, 2013

In my current job we where in the need for a load-test script which performs Service Provider (SP) initiated Web Browser Single Sign-On’s (SSO’s) based on SAML 2.0 (Security Assertion Markup Language). A mouthful, but it sounds harder than it actually is. We need this to determine how much load our Identity Provider (IdP) can handle, but we are also planning to use it as an instrument to test if our identity provider can meet our performance requirements.

Now off course you could also load-test the login process without the use of SAML. In this scenario you go directly to the login URL of the identity provider which doesn’t involve exchanging SAML messages between the service provider and the identity provider. While this is easier to setup it does not produce the same pressure as an SP initiated SSO especially if you want to sign and encrypt the messages.

As a prerequisite we need a working service and identity provider in a trust. In our scenario the service provider uses the SAML HTTP-POST binding for transporting SAML messages between each other. With little effort you could make the script work for the SAML HTTP Redirect binding as well. We’ve used OpenAM, a all-in-one access management system for setting up an identity provider. Even if you have a different setup you could still benefit from the provided steps and change the details to your configuration.

For creating the script I’ve used Apache JMeter 2.10. It’s a free open source, cross-platform performance testing tool that has been around for a long time but was new to me. It’s really powerful and supports many server/protocol types like: HTTP(S), FTP, Databases via JDBC, LDAP, etc.

Before we continue we need to explain the responsibility of the three involved parties: the service provider, the identity provider and JMeter in the context of the test plan.

We need the service provider for creating the SAML request, more specifically the AuthNRequest. It’s not easy to compose them with JMeter, this would take a serious investment which isn’t worth the time and money because we already have working service providers in place which can do the job perfectly for use. Because we have decided to involve the service provider we also expand the test a tiny bit by sending the SAML response to the service provider and do some behavioral assertions.

The identity provider should off course authenticate the user with backed user data stores and return SAML assertions about the user like roles and name identifier.

Last but not least JMeter is responsible for sending and measuring all the request to the different parties acting more or less as a man in the middle and orchestrates the test.

Now that we know some context and background let’s start with going through the script step by step.

image

The first node is a Test Plan. This node is the root of our script and houses all the components. Here we give the plan a name and define some variables we can access in the child components. You should define the hostname and port of your service and identity provider in this screen.

image

In order to simulate the login of many users we created 100.000 users in the user data store of the identity provider, each with his own username and password. With the CSV Data Set Config Element you can read lines from a file, and split them into variables. We use a CSV file with the following content (username,password).

user1,abcd1234

user2,abcd1234

user100000,abcd1234

Each JMeter thread simulates a user that will read one line and has access to the variables login and password (which are basically the columns in your CSV). For the file location you can use an absolute path or a path which is relative to your script location.

image

A HTTP Cookie Manager stores and send cookies just like a web browser does. In our scenario there are a lot cookies involved so we can’t live without it. Each JMeter thread has its own "cookie storage area" for storing session related cookies. If needed you could manually add some cookies as well. This cookies will be shared by all JMeter threads.

We instructed the Cookie Manager to clear all server defined cookies after the thread group loop (see below) is executed.

image

A thread group is the core of a test plan and is responsible for controlling the number of threads used to execute the test plan. Under Thread Properties you can enter the following values:

Property

Description

Number of Threads (users)

This value will set the amount of threads to be used for this test plan. In our screenshot we only have one thread. Always start with one thread and see if your test plan works for one thread. Increase the number when you are ready for it.

 

Always keep in mind that the amount of threads that can be run on a single JMeter client machine depends on the hardware of the client machine. JMeter support distributed testing (on multiple machines) but this post is not about that.

Ramp-Up Period (in seconds)

This value is very important when you have a high number of threads. It means how long JMeter takes to "ramp-up" to the full number of threads chosen.

For example when you have 10 threads with a ramp-up period of 100 seconds JMeter will start a thread 10 seconds after the previous. This will spread the load on the JMeter client machine.

Ramp-up needs to be long enough to avoid a large work-load at the beginning of a test, and short enough that the last threads start running before the first ones finish (unless you want that to happen).
Loop Count The amount of times you want to run the test script for one thread. In our scenario this will be one.

 

The thread group node contains HTTP Request Samplers and one IF Controller. When we expand the node we see the following.

 

image

The first HTTP Request Sampler does a HTTP GET on a protected resource on the service provider. In our case that’s the top level domain serviceprovider.com. You can also enter a path to the resource. The thread (user) accessing the protected resource is not authenticated and therefor the service provider will redirect the user via an HTTP-POST to the identity provider. This means that the result of the GET will be an html page containing a form that submits itself. Because JMeter is no browser and doesn’t run JavaScript this will obviously not work. To get this to work we need to extract the values from the form and create the post our self.

In order to do this we create two Regular Expression Extractors which are Post Processors and place them under the HTTP Request Sampler. We need one for extracting the SAML Request and one for the Relay State.

image

image

The reference name will be the name of the variable to store the value found by the regular expression extractor. The template $1$ refers to group one of the regular expression. Match No 1. indicates which math to use since it can match multiple times. With the extracted values we can compose the HTTP POST to the identity provider.

image

Again we created an HTTP Request Sampler but this time the HTTP method is POST. We specified the endpoint path of the identity provider. In our case this is the default value for OpenAM in combination with the HTTP-POST binding. As you can see the SAML Request is encoded and the RelayState is not. I don’t know why but this was needed to make it work (it may be different for yours!).

The result of this post is an HTML page with a login form where the user needs to enter his credentials. The form also contains some hidden elements with information needed by OpenAM. We are going to extract all of them in the follow steps by creating Regular Expression Extractors and place under the HTTP Request Sampler.

image

image

image

image

image

We also added a Duration Assertion on the HTTP Request Sampler. We expect that the response of the HTTP POST is returned within 1000ms. If not the response of the HTTP Request Sampler must be marked as failed.

image

We applied the duration on the main sample and sub-samples. This means that the measuring is done over the HTTP Request and all the embedded resources like CSS, images, webfonts and JavaScript.

Now we are going to compose the HTTP Request to the form action of the previous login form.

image

Again we set the HTTP method to POST and fill in all the needed parameters. Take a good look at the parameters IDToken1 and IDToken2. They refer to the ${login} and ${password} which come from the current line in the CSV file. Our request will go to the default login URL provided by OpenAM. We could also extract the action value from the login form and use that but in our scenario the value is static.

This HTTP Request does the actual login of the user and if everything goes right a SAML response is created that must be send to service provider. This uses the same trick as we’ve seen before. The result will again be a form that does the post the assertion consumer service URL of the identity provider. So again we need to extract the values.

image

For extracting the SAML response we can’t use the Regular Expression Extractor. This is because it doesn’t handle XML line feeds properly, so we used the XPath Extractor instead.

image

We create two important assertions. One, a Response Assertion for checking that the authentication of the user didn’t fail. If it failed the result of the POST will contain a certain message (i.e. authentication failed). An authentication failure also ensures that the extractors don’t matches anything.

image

Also add a duration assertion like we did before.

image

The last step for this HTTP Request Sampler is adding a BeanShell Listener. This makes it possible to create a BeanShell script which can access the results of the assertions programmatically. You can’t use a BeanShell Post Processor because assertions aren’t executed yet in this phase. The reason why we need this script is to create a flag which can be accessed by an IF Controller. We can’t send the SAML Response to the service provider if the authentication failed.

image

(you can copy the code by downloading the JMX file at the end of the post)

As you can see this code will create a variable called do_post_assertion_consumer_service_url with a string value true or false. True when the response assertion didn’t fail.

In the next step we create an IF Controller around a HTTP Request Sampler, which will check the value of the do_post_assertion_consumer_service_url variable.

image

Inside the controller we create an HTTP Request Sampler for sending the SAML Response to the service provider.

image

Other important things besides the parameters are the Implementation and Follow Redirect property. I changed the default implementation because it was not following redirects properly. We want to follow the redirects because the post will be targeted to the assertion consumer service URL of the service provider. If the processing of the SAML Response succeeds the user will be redirected to the relay state value. We want to follow this redirect so we can do a response assertion. If this succeeds as well we know the user has successfully logged in on the service provider.

image

To see the results of the executed test plan we need to add some listeners to the Test Plan node. In our scenario I used the View Results in Table and View Result Tree listener. There are multiple listeners you can chose from, pick the one you like.

This are all the steps needed to do an SP initiated SSO. Run the test by clicking and watch the result listeners.

image

You could extend the test by adding extra steps for performing an SP initiated Single Log-Out (SLO). It should now be easy to add those extra steps. In our example we didn’t use HTTPS but this should be possible as well.

The JMeter Test Plan file can be downloaded at http://www.martijnburgers.net/downloads/SAML-SP-initiated-SSO.jmx.txt (rename extension to .jmx).

Comments are closed