5 tips for using JMeter Fragments for enterprise API testing

Article by: Scott Nelson, Director at Logic20/20

Google is the greatest boon to software development since disks replaced those paper bunch tapes … but solutions for individual topics found online often skip the implications when applied at scale. This blog post will not add new information to the collective knowledge of using fragments in JMeter to test APIs; it will put some important concepts in the perspective of applying them to real-world enterprise solutions.

First things are not always first

Ideally, this article would start with the first tip. I know I generally prefer my information that way, i.e., direct and to the point. If you are reading this, it is because you understand that isn’t always the best way to get to where you need to be. Specifically, a good tip to start with for using Test Fragments in large-scale JMeter API Test Plans is how to plan your taxonomy. The problem with that approach is that you may not understand why that is important until you get to the point where significant refactoring will be required. So, we’ll start with the “why,” then get to the “how.”

Tip 1: Not all fragments are the same

The main purpose of a Test Fragment is to apply the same type of reuse that is encouraged in enterprise design. The other value it provides is the ability to cleanly separate different aspects of a single JMeter Test Plan in support of multiple contributors and source control.

There are two types of Test Fragments in JMeter:

• A node with a Test Plan as the parent and referenced by a Module Controller that is a child to a Thread Group. For convenience, these will be referred to as Test Fragment nodes.

• A JMX file that is created by applying the “Save As Test Fragment” menu option on a Controller and then referenced by an Include Controller that is a child of a Thread Group. These will be referred to as Test Fragment files.

Test Fragment nodes cannot be included in a Test Fragment file, which is why they either need to be saved off as a Test Fragment file or duplicated within the controller that will be exported as a Test Fragment file. This is the first thing to know before creating your source taxonomy.

Tip 2: Refactoring before committing

A key concept in using Test Fragments for large Test Plans that is rarely pointed out is that it is best that Test Fragment nodes are only used during development to avoid redundant refactoring. When specific steps are required multiple times, placing them in a Test Fragment node while working them out is a huge time-saver.

At the completion of development, the contents should be saved as one or more Test Fragment files that will be included in the main JMX file containing the Test Plan and potentially (though rarely) other Test Fragment files. The value of moving the node to a Test Fragment file before committing to source control is that it minimizes the overlap in Test Fragment files between efforts. Working alone, this doesn’t present an issue. But as soon as there as multiple contributors, it quickly becomes very time consuming to manage merge conflicts when too much is in the main JMX file. The use of Test Fragment files greatly simplifies this.

Tip 3: Rules are made to be broken, except when they shouldn’t be

The decision to save a Test Fragment node as a Test Fragment file is based on how many other Test Fragment files may require the Test Fragment. For example, if a Test Fragment node is used a few times in a specific Test Fragment file but not in any other Test Fragment file, then saving as a separate Test Fragment file is of little value. Likewise, if the Test Fragment node is only referenced once or twice and in only two Test Fragment files, it is still not worth creating a separate Test Fragment file, because they should rarely need to be refactored once the API they test is released to production. This is when you must break the rule of re-use and simply duplicate the nodes where necessary once they are stable.

While this type of duplication would not be appropriate in an application developed in an object-oriented language, JMeter is a platform that runs off configuration files where re-use is possible only in a limited way without becoming more work than it is worth.

There may be cases where a particular process is repeated several times. First, review the use of this process to determine if there is a more elegant solution. If not, continue to develop with the process in a Test Fragment node and later save it as a separate Test Fragment file with the naming convention of [parentFragment.childFragment.jmx] to make the usage and dependency clear.

Finally, it is important to understand that APIs are meant to abstract the underlying code. Even if you know the origin of the code, test fragments should be grouped primarily by functionality and (where appropriate) by common test data and not by the namespaces of the source code.

Tip 4: Plan fragments like they are stand-alone test plans

Remember that part earlier about occasionally needing to include other Test Fragment files in Test Fragment files? One of those occasions is when a fragment depends on a variable that is the result of another fragment that is included in the thread. For one reason or another, that first fragment may later be excluded from the thread. In such a case, the dependent fragment should test for the variable with an IF Controller and then include the Test Fragment file to generate the necessary variable.

One additional note about Test Fragment files including Test Fragment files: When a Test Fragment file refers to another Test Fragment file, the path is relative to the main JMX file.

Tip 5: The simplest approach to refactoring is to use two files

This last tip raises a lot of eyebrows when first presented. In the likely event that a Test Fragment file has to be updated, either as a result of debugging or an update to the application behavior, the way to work on the Test Fragment file is to copy the nodes into the main JMX and do the work there, then export the results back to the Test Fragment file. The temptation to make the changes in the Test Fragment file and then test from the main JMX should be avoided for all cases but the extremely trivial, like a variable with a name change and only referenced in a single node. Trying to do all of the work in the Test Fragment file will result in moving back and forth between the files and take much longer than anticipated. Just remember to run a final test with the new Test Fragment file and the original main JMX.

Bonus Tip: Taxonomy

Since I clearly pointed out the need for this in the introduction, I feel obligated to include this last tip. At a high level, the recommended taxonomy looks like this (replace [BRACKETED_TEXT] with your own labels and the example text as necessary):

More tips to come

This post is based on common guidelines I help enterprises define. It has been my experience that using fragments as described here is the best way to support testing of a large, complex, API-driven application that is continuously evolving. It has also been my experience that using them counter to the recommendation of always moving fragments to files early results in productivity loss by requiring more time in maintenance, which detracts from expanding the testing footprint. But it is not the only important lesson is scaling JMeter API testing. If you benefited from reading it, please let me know what other areas are of most interest to you and I will base the next article in the series on that.