Using ChatGPT to learn how to use JMeter

2023-08-06 2116 words 10 minutes

Contents

A little while ago now, my team at work was looking at improving the performance of a FHIR Server API. Thanks to Azure Workbooks, we’ve already been able to identify a bottleneck with the underlying data storage resource - CosmosDB. Now, as we explore options to enhance performance, we need a way to reliably benchmark the server. These benchmarks will help us measure the impact of any changes we make to the configuration and infrastructure. It’s in this context that load testing tools can be really helpful as they offer up insights into an applications performance under stress.

Starting off with NBomber

We started off using NBomber, which is a really nifty Dotnet library that can be used to write load tests. We were easily able to put together a simple load test, however very quickly hit a limitation in how many simultaneous requests we were able to make to the API - which kind of defeated the point of a load test. When running the load tests from my laptop I could reliably get to around 340 simultaneous requests before my the load test gave up.

Our journey began by utilizing NBomber, a useful .NET library that is aimed at developer, as it allows them to easily write load tests in c#. I was able to really easily write a basic load test, only to quickly encounter a limitation regarding the number of simultaneous requests we could send to the API. When running the load tests on my laptop, I consistently reached a point of approximately 340 simultaneous requests before the load test process came to a halt.

Introducing JMeter

Needing an alternative I turned my mind to JMeter, I’ve never used it before but other teams at DrDoctor have, so it was on my radar as a technology option. However as I had never used it before I needed to quickly get up to speed. That’s where ChatGPT comes into play.

A conversation with ChatGPT

Instead of reaching for Google, which I would have done in the past I thought I would ask ChatGPT instead if it could offer some advice on how to use JMeter to load test our API. Here’s a rehash of our conversation:

Can you write a jmeter load test for a FHIR server that creates a large dataset made up for Patient and Appointment resources

ChatGPT

Certainly! Here’s an example of a JMeter load test script for a FHIR server that creates a large dataset consisting of Patient and Appointment resources:

First, make sure you have JMeter installed and running. You can download it from the Apache JMeter website.
Create a new JMeter test plan:

Right-click on “Test Plan” in the left panel.
Select “Add” -> “Threads (Users)” -> “Thread Group”.
Set the desired number of threads (representing concurrent users) and ramp-up period (time to start all threads).
Set the loop count to the desired number of iterations (e.g., -1 for infinite).

Add an HTTP Header Manager:

Right-click on the Thread Group.
Select “Add” -> “Config Element” -> “HTTP Header Manager”.
Configure the manager to set the “Content-Type” header to “application/fhir+json”.

Add an HTTP Request sampler to create a Patient resource:

Right-click on the Thread Group.
Select “Add” -> “Sampler” -> “HTTP Request”.
Set the HTTP Method to “POST”.
Set the Server Name or IP to the FHIR server’s address.
Set the Path to the appropriate endpoint (e.g., “/Patient”).
In the “Body Data” section, enter the JSON payload for creating a Patient resource. You can generate the payload using a tool like Postman or construct it manually.

Add an HTTP Request sampler to create an Appointment resource:

Repeat the previous step but configure the sampler for creating an Appointment resource.
Set the Path to the appropriate Appointment endpoint (e.g., “/Appointment”).

Configure the desired number of patients and appointments:

Right-click on the Thread Group.
Select “Add” -> “Config Element” -> “CSV Data Set Config”.
In the “Filename” field, provide the path to a CSV file containing the data for Patient and Appointment resources.
Define variables (e.g., ${patient_id}, ${appointment_id}) to represent the values in the CSV file.

Replace the hardcoded values in the JSON payload:

Open the HTTP Request sampler for creating a Patient resource.
Replace the static values with the JMeter variables (e.g., “${patient_id}”).

Add a View Results Tree listener:

Right-click on the Thread Group.
Select “Add” -> “Listener” -> “View Results Tree”.
This listener will display the results of the load test, including response times and any error messages.

Save the test plan and run the test:

Click on the “Play” button to start the load test.
Monitor the results in the “View Results Tree” listener.

Note: Make sure to adjust the test plan according to your specific FHIR server configuration and the structure of your Patient and Appointment resources. Additionally, consider adding timers, assertions, and other JMeter elements to further enhance the load test.

This was a really good starting point, and using the steps above I was able to get a very simple JMeter test plan working. However soon after I hit up against the age old problem of testing….needing test data. I was just about to look up some example FHIR resources but then decided to see if ChatGPT could keep being helpful.

Can you generate a csv file that can be used to generate appointments and another one for patients

ChatGPT

Certainly! Here’s an example of a CSV file that you can use to generate appointments:

appointments.csv:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


appointment_id,patient_id,appointment_date,appointment_time
1,1001,2023-05-15,09:00 AM
2,1002,2023-05-15,10:30 AM
3,1003,2023-05-15,01:00 PM
4,1004,2023-05-16,11:30 AM
5,1005,2023-05-16,03:00 PM
6,1006,2023-05-17,02:30 PM
7,1007,2023-05-18,09:30 AM
8,1008,2023-05-18,11:00 AM
9,1009,2023-05-18,04:30 PM
10,1010,2023-05-19,10:00 AM

And here’s an example of a CSV file that you can use to generate patients:

patients.csv:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


patient_id,first_name,last_name,date_of_birth,email,phone
1001,John,Doe,1985-01-15,[email protected],1234567890
1002,Jane,Smith,1990-06-22,[email protected],9876543210
1003,Michael,Johnson,1978-11-05,[email protected],4567890123
1004,Emily,Williams,1995-03-12,[email protected],7890123456
1005,David,Brown,1982-09-08,[email protected],2109876543
1006,Sarah,Davis,1989-12-29,[email protected],5432109876
1007,Robert,Miller,1976-08-17,[email protected],8901234567
1008,Amy,Wilson,1993-04-02,[email protected],6789012345
1009,Christopher,Taylor,1980-07-25,[email protected],3210987654
1010,Olivia,Anderson,1998-02-18,[email protected],0123456789

Please note that these are just sample data files, and you can modify them according to your specific needs.

Right, that was super helpful to generate some test data, but I also needed the tokenized JSON for the FHIR resources that could be used in JMeter

Can you generate the appointment and patient fhir json that can be used with the above csv files and jmeter

ChatGPT

Certainly! Here’s an example of the FHIR JSON payloads for creating an Appointment and a Patient resource based on the CSV files provided:

Appointment FHIR JSON Payload:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


{
  "resourceType": "Appointment",
  "id": "${appointment_id}",
  "status": "booked",
  "description": "Appointment for ${patient_id}",
  "start": "${appointment_date}T${appointment_time}:00",
  "end": "${appointment_date}T${appointment_time}:30",
  "participant": [
    {
      "actor": {
        "reference": "Patient/${patient_id}"
      }
    }
  ]
}

Patient FHIR JSON Payload:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


{
  "resourceType": "Patient",
  "id": "${patient_id}",
  "name": [
    {
      "family": "${last_name}",
      "given": [
        "${first_name}"
      ]
    }
  ],
  "birthDate": "${date_of_birth}",
  "telecom": [
    {
      "system": "email",
      "value": "${email}"
    },
    {
      "system": "phone",
      "value": "${phone}"
    }
  ]
}

Please note that the placeholders (e.g., ${appointment_id}, ${patient_id}, ${appointment_date}, etc.) should be replaced with the corresponding values from the CSV files during runtime.

Ensure that the JSON payloads are properly formatted, and use these templates to construct the actual payloads in your JMeter test script.

This was really helpful, however I wasn’t actually sure how to use the above, so I asked ChatGPT

How should variables be entered into the csv data set config in jmeter

ChatGPT

In JMeter’s CSV Data Set Config, variables should be entered in a specific format to ensure proper usage. Here’s how you should enter variables in the CSV Data Set Config in JMeter:

Open the CSV Data Set Config element in your JMeter test plan.
In the “Filename” field, provide the path to the CSV file containing the data.
In the “Variable Names” field, enter the names of the variables that correspond to the columns in the CSV file. The variables should be separated by commas.

For example, using the Patient resource variables, you would enter:

1

patient_id,first_name,last_name,date_of_birth,email,phone,nhs_number

Make sure that the variable names match the column headers in the CSV file exactly.

Configure other settings in the CSV Data Set Config, such as file encoding, delimiter, and other options as per your requirements.

With this configuration, JMeter will read each row of the CSV file, assign the corresponding values to the variables, and use them in your test plan during runtime.

Excellent, now I was really make quick progress - I had a load test which read from a CSV file and created a bunch of Patient and Appointment resources. However, as you can see from the CSV output above this was only 10 rows of sample data, not really enough when I wanted to run 1000s of requests. So back to ChatGPT, initially I asked if it could generate a larger set of data, but this was taking a really long time so I came up with a different idea…

Instead of generating 10,000 records for me, could you write some c# code that would generate the csv file for me instead

ChatGPT

Certainly! Here’s an example of C# code that generates a CSV file with 10,000 patient records:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95


using System;
using System.IO;

public class Patient
{
    public int PatientId { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime DateOfBirth { get; set; }
    public string Email { get; set; }
    public string Phone { get; set; }
    public string NhsNumber { get; set; }
}

public class CsvGenerator
{
    private static Random random = new Random();

    public static void GeneratePatientsCsv(string filePath, int count)
    {
        using (StreamWriter writer = new StreamWriter(filePath))
        {
            // Write the header
            writer.WriteLine("patient_id,first_name,last_name,date_of_birth,email,phone,nhs_number");

            // Generate patient records
            for (int i = 1; i <= count; i++)
            {
                Patient patient = GenerateRandomPatient(i);
                writer.WriteLine($"{patient.PatientId},{patient.FirstName},{patient.LastName},{patient.DateOfBirth:yyyy-MM-dd},{patient.Email},{patient.Phone},{patient.NhsNumber}");
            }
        }

        Console.WriteLine($"CSV file generated successfully: {filePath}");
    }

    private static Patient GenerateRandomPatient(int id)
    {
        string[] firstNames = { "John", "Jane", "Michael", "Emma", "David", "Olivia", "Daniel", "Sophia", "Andrew", "Isabella" };
        string[] lastNames = { "Smith", "Johnson", "Williams", "Jones", "Brown", "Davis", "Miller", "Wilson", "Taylor", "Clark" };
        string[] domains = { "example.com", "test.com", "dummy.com", "sample.com" };

        Patient patient = new Patient
        {
            PatientId = id,
            FirstName = firstNames[random.Next(firstNames.Length)],
            LastName = lastNames[random.Next(lastNames.Length)],
            DateOfBirth = GenerateRandomDateOfBirth(),
            Email = GenerateRandomEmail(domains[random.Next(domains.Length)]),
            Phone = GenerateRandomPhoneNumber(),
            NhsNumber = GenerateRandomNhsNumber()
        };

        return patient;
    }

    private static DateTime GenerateRandomDateOfBirth()
    {
        DateTime startDate = new DateTime(1950, 1, 1);
        DateTime endDate = new DateTime(2005, 12, 31);
        int range = (endDate - startDate).Days;
        return startDate.AddDays(random.Next(range));
    }

    private static string GenerateRandomEmail(string domain)
    {
        string[] providers = { "gmail", "yahoo", "outlook", "hotmail" };
        string[] extensions = { "com", "net", "org" };
        string username = Path.GetRandomFileName().Replace(".", "").Substring(0, 8);
        string provider = providers[random.Next(providers.Length)];
        string extension = extensions[random.Next(extensions.Length)];
        return $"{username}@{provider}.{extension}";
    }

    private static string GenerateRandomPhoneNumber()
    {
        return $"{random.Next(100, 1000)}-{random.Next(100, 1000)}-{random.Next(1000, 10000)}";
    }

    private static string GenerateRandomNhsNumber()
    {
        return $"NHS{random.Next(1000000000, int.MaxValue)}";
    }
}

public class Program
{
    public static void Main(string[] args)
    {
        string filePath = "patients.csv";
        int recordCount = 10000;

        CsvGenerator.GeneratePatientsCsv(filePath, recordCount);
    }
}

This code will generate a CSV file

I also asked ChatGPT to write some code that woudl generate appointments which I’ve ommitted from this blog post.

This was the first time I realised how useful ChatGPT could be while doing really mundane dev work. As you can see it performs really well as a side kick. There were a few places where it wasn’t very helpful.

The FHIR Server was protected with OAuth authentication and so I asked ChatGPT how I should go about generating an access token before the main load test started and then including it on subsequent requests. It gave me an answer and (as you can imagine) was very confident about it. I tried what was suggested but that didn’t work, I asked a few variations on how I should do it but in the end I turned back to Google and read through a couple of posts that others have written and the JMeter docs. So ChatGPT isn’t quite there yet for everything but it’s definitely useful!

🍪 I use Disqus for comments

Because Disqus requires cookies this site doesn't automatically load comments.

I don't mind about cookies - Show me the comments from now on (and set a cookie to remember my preference)