During some recent work with a client, I had the task of pulling up about 100 automated regression tests for a critical business application. These scripts were frequently executed, and took hours to complete. They had also been in service for years and seen many updates, but we knew they worked… or did they?
Building the Project
The first thing I did was create a new project in Test Design Studio to include these various tests. In addition to the tests, I added all the relevant function libraries and object repositories. Each of these assets were stored in Quality Center/ALM, and Test Design Studio handled them perfectly. That’s when the fun begin.
Hello “Option Explicit”
Test Design Studio features a code analysis feature that helps you focus on improving the quality of your code. Now that my project was established, the code analysis was lighting up all the areas I needed to address. If there is one practice I have always preached for writing VBScript-based code, it is this… turn on “Option Explicit”. When set, this requires the VBScript run-time engine to throw an error if you attempt to use an undeclared variable. Anyone who has ever mistyped the name of a variable and tried to debug why their script failed knows exactly what I’m talking about (hint: VBScript just uses the mistyped name as a new variable with an empty value). When over 50 of the tests were displayed in my Error List as not having Option Explicit set (which is one of the code analysis rules you can turn on), I knew it was my first order of business.
Updating the Files
Remember, all of these tests are in Quality Center/ALM and are scattered between folders. What would normally be a daunting task of searching out the noncompliant tests, opening in QuickTest (switching to Expert View), and saving back was made remarkably fast by Test Design Studio. The process was simple…
- Double-click the error in the Error List that indicated the lack of Option Explicit. It did the heavy lifting of finding the right file.
- The file was opened directly in Test Design Studio with the code readily available.
- A pasted in my line for “Option Explicit”.
- Saved and closed the editor.
- On to the next error; repeat.
What a relief! I had those 50-or-so tests updated quickly.
What About Those Undimmed Variables?
Now comes the tricky part. Turning on Option Explicit may sound like a simple task, but the consequences can be devastating for an existing script. If I run the tests “as-is”, they will throw an error and halt execution any time an undeclared variable is used. These tests are critical, and silly failures like this cannot be allowed. Executing, fixing, executing, and fixing would take hours for each test. Many of these scripts were 1000’s of lines long and could take over 30 minutes to complete. Variable declaration was never required, so the assumption was that many of the variables in use had never been declared. That’s when I saw the new Error List from Test Design Studio. It contained over 1,200 errors, most related to the use of an undeclared variable. Thankfully, that’s another code analysis rule you can turn on in Test Design Studio!
Without code analysis, I never would have dared to scour all those lines of code looking for every possible variable reference and making sure it was declared. When adding all those declarations, it’s also easy to accidentally declare a variable in one section of code that was already declared somewhere else. That, too, can cause a run-time error. You guessed it… Test Design Studio has a rule for that too!
My process was simple… I would find one of the errors in the list and open that test. I’d then filter the error list to only show me the errors for the currently opened file. That let me know exactly how much work was before me. When I’d find the usage of an undeclared variable, I’d do a quick “Find in File” to see all the places in the test where the variable was used. Once the file was clean, I’d save/close, restore my error list to the unfiltered view, then pick the next error from another file.
Most of the time, the variables were legitimate and needed to be declared. Sometimes I found that variables were assigned values but never actually used in the test! That’s confusing to new authors trying to understand a test, so I got rid of those (they were left over from previous versions of the test that were later modified). The worst is when I found variables that were used but never assigned! The author of the test clearly expected a value to be in the variable, but it was always going to be blank. That could lead to the test not validating the functionality it was designed to test. Was this test even valid?
How Did It Turn Out?
After spending a couple hours going over all the tests, adding Option Explicit, and fixing all those variable references, I found several logic errors in the script that never would have happened if Option Explicit had been turned on to begin with (confirming my theory that you should always use Option Explicit). When the tests were executed, not a single run-time error was encountered from all those changes.
Tens of thousands of lines of code, and I didn’t miss a single variable… THANK YOU TEST DESIGN STUDIO!
Use Test Design Studio For Your Own Projects
You can find out more about how the Code Quality features in Test Design Studio on this special feature page. Of course, you can also download a fully functioning trial version of the software and try it out for yourself.
Most areas of computer programming have the privilege of access to many reference books. QuickTest automation, which I have always stated is more of a programming activity than testing activity, is not one of those areas. This is why a new book on QuickTest programming is always a gem! The latest work by Anshoo Arora and Tarun Lalwani, titled “QTP Descriptive Programming Unplugged” does not disappoint.
This is not the first book for Tarun. Those who are still learning the in’s and out’s of QuickTest should check out his first book titled “QuickTest Professional Unplugged”. I also reviewed his second book “And I thought I knew QTP!” which utilizes a narrative technique to introduce technical concepts. This latest book, with the help of Anshoo Arora, is a return to the typical style of a technical reference manual.
I have been working with GUI test automation for 13 years, and there is one key aspect of interacting with a GUI that has never changed; you must be able to recognize the UI objects! Anyone who has ever been given a new application to test has had the realization that comes when you fire up your automation tool only to see that few, if any, of the screen objects are recognized by the tool. This is the “knife to the heart” of any automation effort because object recognition is so vital to successful automation projects. We face enough challenges with GUI automation, that object recognition should not be one of them.
When you find that QuickTest is able to recognize your objects, be thankful! The journey typically does not end there, and that is where this book is a valuable resource. Even when working with supported technologies, getting QuickTest to properly and consistently recognize your objects is a must.
This book is a valuable resource because it takes many years of experience in QuickTest object recognition and presents it in a clear, well-organized fashion. You will learn the differences between Local and Shared object repositories, and which strategy to use. Those who prefer to avoid object repositories will receive a healthy discussion on descriptive programming, the art of defining your objects at the time you use them instead of within a repository. Even advanced topics such as using the Document Object Model (DOM) for web applications or XPATH to identify elements are covered in detail.
The authors do more than just introduce topics. Topics are what you can expect from the QuickTest-provided documentation. This book takes those topics and discusses the risks and benefits of each. This was evident when I read about the topic of “Smart Identification” (I’ve always believed it’s one of the biggest misnomers since Little John in the Robin Hood stories), how it works, and why disabling it is the first thing you should do. For example, you can quickly learn about the concept of descriptive programming, but this book explains why you might want to use it, suggestions for improving success on large projects, and the potential pitfalls like escaping regular expression characters.
While primarily a discussion on object recognition techniques, the book does go off-topic on a few extras such as how you could write test code in a .NET language.
Those who are new to QuickTest should still start out with “QuickTest Professional Unplugged”, but they should quickly follow it up with this latest title. Even automators with several years and projects behind them have something to gain through the reference.
The most notable additions to this release are related to the build process. For those who may not be familiar with this feature, the build process allows Test Design Studio to perform routine tasks associated with the collection of files in your project.
A Little Background on Build
Perhaps the most important of the build tasks is the ability to update your QuickTest tests. QuickTest does not have a centralized approach to test configurations unless you use Business Process Testing. Each test is a self-contained entity requiring its own configuration. This is great if you write tests in isolation, but most users typically create a suite of many tests for a given application. Shared resources such as function libraries and object repositories are used by most (if not all) of those tests. When you add a new library or object repository to your shared resource pool, you must update each test to add this new resource. That process is time-consuming and easy to overlook.
The Test Design Studio build process changes that! Test Design Studio already has an excellent project-centric approach to test management, and keeping your tests updated is as easy as running the build process. Once initiated, Test Design Studio looks at all the tests in your project as well as all the shared resources. Each test is then opened in QuickTest through their API, object repositories are associated with each action, and function libraries are associated with each test. Test Design Studio even makes sure you have the proper add-ins set.
When I develop automated tests, I place most of my functionality in reusable function libraries. I keep libraries small and each one focuses on specific aspects of the application. This keeps my code organized and easy to manage, but can easily create 10 to 20 function libraries. Managing that many individual files would be cumbersome without Test Design Studio. When I need a new test, my actions are simple:
- Add the test to my project using the ‘Add New Item’ command and selecting my QuickTest file template
- Right-click the file in Solution Explorer and select ‘Build’
That’s it! My new test is immediately configured with all the function libraries and object repositories that are defined for my project. When I need to add a new function library to my projects, the process is equally simple:
- Add the library to my project using the ‘Add New Item’ command and selecting library file template
- Right-click the project in Solution Explorer and select ‘Build’
Just like that, Test Design Studio will process all of my tests to make sure they reference the new library.
What Changed in New Release
The build process is key to increasing your productivity and working in conjunction with QuickTest. One of our newer customers recently posted a series of suggestions for improving the build process, and we were pleased to make those enhancements (you know how you are, and thank you!).
Hint: If you have an idea on how to improve Test Design Studio, it’s a great idea to share it!
Test Left Open after Build
If you initiate the build process for a single test, Test Design Studio will now leave that test open in QuickTest after the build is complete. This is a great time-saver when actively writing and debugging a test since you can essentially use the ‘Build’ command to open the test in QuickTest (something always available off the context menu) but also ensure that all resources are properly associated to the test.
Version Control Improved
We made many adjustments to how version controlled tests are managed. A test must be checked out in order to modify it, so the build process will check out any test that was not already checked out. If no changes were necessary from the build, that check-out will be cancelled. When the test is modified, it will be checked back in only if the test was not already checked out. Those tests that were already checked out before initiating the build will remain checked out afterwards.
Keeping a test checked out proved to be a complex task due to an issue with the QuickTest API. If you save a version-controlled test and try to close it without checking it in, QuickTest displays a prompt asking if you want to check it in. It even does this if you are using the API, and that dialog was blocking the build process while waiting for user input. Even worse, the dialog displayed by QuickTest was often not visible on the desktop giving the illusion that Test Design Studio had stopped responding. To keep the build process operating smoothly, we actually had to use GUI automation to detect and dismiss that dialog when displayed.
Build Selected Command Updated
The ‘Build Selected’ command was originally designed to build the selected item in the Solution Explorer tool window. For those that do not have the Solution Explorer track the currently selected document, this caused some confusion. If you had one test selected in Solution Explorer but were actively editing a different test in an editor, the ‘Build Selected’ command would build the test in Solution Explorer instead of your current document. That has now changed. The command will only process the selection from Solution Explorer if that tool window is selected when activating the command. Otherwise, it will attempt to process the actively edited document.
For those not familiar with the build process, we hope you integrate it into your project management process. For those already using it, we hope you enjoy these enhancements. Thank you again to the customer who brought these suggestions to us, and keep that feedback coming! We all benefit from the ideas of others.
How safe do you feel knowing your code passes a syntax check?
Keeping reading and you might change your mind.
We have talked at length about having correct syntax does not mean your code is correct. Sure, we start with getting the syntax correct, but we must also follow the rules of the language.
The ability to create functions and return values is one of the greatest tools available to a programmer! VBScript, unfortunately, does not use a simple statement such as “return” to signify the return value (as is common in “C”-style languages). Instead, it makes you repeat the name of the function in a syntax similar to assigning a value to a variable. This works great, of course… provided you type the name correctly!
While working on support for the Web Services add-in for the next release of Test Design Studio, we opened the “Web_Services.txt” file that ships with QuickTest to ensure that the user-defined functions were being properly registered to the “WebService” object. That is when we found the issue below:
Do you see what’s wrong? Test Design Studio makes it pretty obvious that the author of this file mistyped the name of the function when trying to set the return value. What should have been “GetHeaders” was typed as “GetHeader”. Test Design Studio saw two potential problems here:
- “GetHeader” (since it does not match the name of the function) is the same as a reference to a variable. This file has Option Explicit turned on, and a variable named “GetHeader” does not exist. Test Design Studio will not let you use an undeclared variable when Option Explicit is enabled.
- Functions, by definition, are meant to return values. This function declaration does not (due to the mistyped name). Test Design Studio warns you that you might have forgotten to return a value.
This error means that anyone attempting to use the “GetHeaders” function attached to the “WebServices” object is either going to get a run-time error or, even worse, never get the object they were trying to retrieve.
Sadly, that’s not the only error Test Design Studio caught. Elsewhere in the same file was this code:
Again, Test Design Studio makes it obvious that there is a problem here! The code author misspelled the parameter name in the function declaration, which made the reference to the parameter (using the proper spelling) inaccurate. Test Design Studio gave the following reports:
- The code attempted to use an undeclared variable “XMLSchemaFile” (Option Explicit turned on). Since the name did not match the parameter, it was expecting the name to be defined elsewhere.
- A warning was given that the user declared the parameter “XMLScheamFile” and never actually used it.
This function, as defined, would never properly validate the XML Schema file.
Had the author of this code been using Test Design Studio and the powerful code analysis features, neither of these mistakes would have been distributed to thousands of HP customers! Keep in mind that the syntax of this file was 100% accurate even though the file had multiple errors.
So I ask again… how safe do you feel that syntax checking is enough to verify your code?
The two major flaws in this code were caused by 3 simple characters. What might you have missed in your code?
Note: This file was analyzed from the general release of QuickTest v11.0. I have not verified if the issues have been corrected in any patched release. What to check your file? The default location is “C:\Program Files\HP\QuickTest Professional\data\BPT_Resources”. You may also find the file automatically added to your Quality Center/ALM projects in the Resources module.
We are happy to announce that the next release of Test Design Studio will add support for Delphi test objects! Support for QuickTest test objects like those in the Delphi add-in are just one of the features that give Test Design Studio an advantage over standard VBScript editors like Notepad++. Many VBScript-based editors can provide support for standard VBScript language elements, but only Test Design Studio adds support specifically for objects that are unique to QuickTest!
The added capability will bring IntelliSense to the collection of objects associated with this add-in, including: DelphiButton, DelphiCheckBox, DelphiComboBox, DelphiEdit, DelphiEditor, DelphiList, DelphiListView, DelphiNavigator, DelphiObject, DelphiRadioButton, DelphiScrollBar, DelphiSpin, DelphiStatic, DelphiStatusBar, DelphiTable, DelphiTabStrip, DelphiTreeView, and DelphiWindow.
This addition means that Test Design Studio now has full IntelliSense for all of the following add-ins:
- Standard Windows (i.e. Win32)
- Visual Basic
- Terminal Emulator
Test Design Studio has long offered the ability to modify the fonts and colors used by the editing control. This is done by selecting ‘Tools –> Options’ from the main menu, expanding the ‘Environment’ node and selecting ‘Fonts and Colors’.
Until now, however, there was never a way to change the background of the text editor from the system default settings provided by the Windows theme in use. Many users have requested the ability to create a “dark” editing environment, and now you can! The screen shot below shows what is now possible with the new customization options:
This was achieved by the addition of the following new entries in the “Fonts and Colors” list:
- Text Area Background – Normal background color of the editor
- Text Area Background (Disabled) – Background color of the editor when editing is disabled (i.e. read-only)
- User Margin Background – Background color of the user margin (i.e. the area where icons are displayed for language elements)
- Selection Margin Background – Background color of the selection margin (i.e. the area where outlining indicators are displayed)
- Outlining Indicator – The Foreground color controls the border and lines of the outlining node, while the background color determines the background of the expand/collapse indicator.
- Line Numbers – While not a new entry, the “Background Color” is now supported to control the background of the line number margin.
We hope you enjoy this new level of customization that is coming in the next release (expected by mid-February).
Once again, another great feature request from one our users is being integrated into the next release of Test Design Studio!
Test Design Studio features extensive support for outlining the various code elements of your file so that you can collapse areas that you do not want to focus on at the time. The ‘Toggle Outlining Expansion’ command is what you use to expand/collapse the node closest to the caret position. But what about when you want to toggle a lot of nodes at once?
Previously, the only other option was ‘Toggle All Outlining’ that worked on the whole document. We have now expanded the functionality of the ‘Toggle Outlining Expansion’ command to support a selected range.
If you have not selected any text in the editor, the command functions just like it always did before. If you have selected a text range, all the nodes that are fully encompassed in the text range will either be expanded or collapsed. Since you are dealing with multiple outlined regions that could be in different expand/collapse states, the first encountered node in the region will determine if the other nodes are expanded or collapsed. So if the first node in the range is currently expanded, all nodes will be toggled to the collapsed state.
This subtle enhancement should make it that much easier to use outlining in Test Design Studio and further improve your productivity.
We are happy to provide another feature request in our next release of Test Design Studio. This feature, while simple, is very useful. When you press the ‘SPACE’ key after typing the ‘Call’ keyword, a member list will be displayed showing just the available methods that are accessible for you to call. This can make it easier to find what you’re looking for without displaying a full member list and ignoring all the irrelevant items like variables and keywords that are not valid in that context.
Note: This special feature is only available when you press the ‘SPACE’ key after the ‘Call’ method and will not be displayed if you use the standard options for showing a member list.
This feature is primarily targeted for use with Enumerations (a.k.a. Enum), but everything you see below will also work with Boolean values since it, too, has a pre-defined list of values (i.e. True or False). Support for Enum’s is something unique to Test Design Studio that you cannot find in QuickTest® Professional!
NOTE: This feature will be available in the next build of Test Design Studio
We have always supported the ability to setup the parameters of a method with an Enum list so that, when typing the arguments for your method call, you would be presented with a member list of available values for that parameter based on the associated Enum. We are happy to report that this option is now available when performing an assignment operation or equality comparison.
In the screen shot below, you can see an example of this feature. At the top of our script, we have defined an Enum for a series of strings called “Color”. Later, we declare a variable and use the XML comments to designate that this variable is expected to hold the values of the “Color” Enum instead of being treated as a normal string.
With this configuration in place, the Member List of available values is immediately displayed when you press "Space" after an "Equals" sign. You can also manually display the Member List (i.e. “CTRL+SPACE” for the Complete Word operation) of available values to assign to this variable. This feature can help ensure you are assigning the appropriate values.
The same feature is available when using the equals sign to do an equality comparison instead of assignment as shown below:
If you do not want to see the Enum-specific Member List, simply repeat the “CTRL+SPACE” shortcut command to toggle between the default Member List and the Enum-specific Member List.
Defining and using Enums is a feature unique to Test Design Studio that can easily help you ensure you are using data correctly. If you are not already using them, now is a great time to take a second look at the feature.
You can find more details about how to define an Enum in our on-line help.
This feature comes to you courtesy of a request from a user! We always strive to incorporate the features that are important to the users of our product, so we encourage you to keep those feature requests coming. All ideas are welcome! This feature was incorporated and made available to the user in less than 24 hours from receiving the request. We cannot promise that all features can be added or that you will have the same turnaround, but we hope you take this example as an indicator that we are serious about keeping our users happy. When is the last time you remember getting support like that from a software company!?
I am happy to announce that I will be presenting a session at Software Test Professionals Conference 2011 Fall in Dallas, TX!
Session Title: Improve Automation Code Quality, Clarity, and Comprehension
Session Track: Test Automation
Session #: 602
Date/Time: Wednesday, October 26, 11:30am – 12:45pm
How many times have you looked at someone else’ automation code and
wondered what they were doing? How about your own code months after you
wrote it? Much of the effort to perform test automation comes months or even
years after the initial scripts have been written. Before you can begin to debug
broken code or write new functionality for a test, you must first comprehend
what was previously written.
This session will teach you about many of the common pitfalls in authored code
and how you can refactor your code to dramatically improve comprehension.
Topics of discussion will include good naming conventions for variables and
methods, when to use callable routines, effective use of parameters in routines,
variable usage, avoiding “magic values”, code formatting, and objective
measurements of code complexity. Examples from the presentation will be
provided based on HP QuickTest(R) Professional, but the concepts can be
universally applied to any programming language.
This will be my first time presenting at a conference, so I am very excited about the opportunity and look forward to bringing the spotlight to code quality. Perhaps equally exciting is the chance to meet many of the Test Design Studio users out there. Are you planning to attend the conference? If so, drop me an e-mail. And of course… be sure to sign up for my session!