During some recent work with a client, I had the task of pulling up about 100 automated regression tests for a critical business application. These scripts were frequently executed, and took hours to complete. They had also been in service for years and seen many updates, but we knew they worked… or did they?
Building the Project
The first thing I did was create a new project in Test Design Studio to include these various tests. In addition to the tests, I added all the relevant function libraries and object repositories. Each of these assets were stored in Quality Center/ALM, and Test Design Studio handled them perfectly. That’s when the fun begin.
Hello “Option Explicit”
Test Design Studio features a code analysis feature that helps you focus on improving the quality of your code. Now that my project was established, the code analysis was lighting up all the areas I needed to address. If there is one practice I have always preached for writing VBScript-based code, it is this… turn on “Option Explicit”. When set, this requires the VBScript run-time engine to throw an error if you attempt to use an undeclared variable. Anyone who has ever mistyped the name of a variable and tried to debug why their script failed knows exactly what I’m talking about (hint: VBScript just uses the mistyped name as a new variable with an empty value). When over 50 of the tests were displayed in my Error List as not having Option Explicit set (which is one of the code analysis rules you can turn on), I knew it was my first order of business.
Updating the Files
Remember, all of these tests are in Quality Center/ALM and are scattered between folders. What would normally be a daunting task of searching out the noncompliant tests, opening in QuickTest (switching to Expert View), and saving back was made remarkably fast by Test Design Studio. The process was simple…
- Double-click the error in the Error List that indicated the lack of Option Explicit. It did the heavy lifting of finding the right file.
- The file was opened directly in Test Design Studio with the code readily available.
- A pasted in my line for “Option Explicit”.
- Saved and closed the editor.
- On to the next error; repeat.
What a relief! I had those 50-or-so tests updated quickly.
What About Those Undimmed Variables?
Now comes the tricky part. Turning on Option Explicit may sound like a simple task, but the consequences can be devastating for an existing script. If I run the tests “as-is”, they will throw an error and halt execution any time an undeclared variable is used. These tests are critical, and silly failures like this cannot be allowed. Executing, fixing, executing, and fixing would take hours for each test. Many of these scripts were 1000’s of lines long and could take over 30 minutes to complete. Variable declaration was never required, so the assumption was that many of the variables in use had never been declared. That’s when I saw the new Error List from Test Design Studio. It contained over 1,200 errors, most related to the use of an undeclared variable. Thankfully, that’s another code analysis rule you can turn on in Test Design Studio!
Without code analysis, I never would have dared to scour all those lines of code looking for every possible variable reference and making sure it was declared. When adding all those declarations, it’s also easy to accidentally declare a variable in one section of code that was already declared somewhere else. That, too, can cause a run-time error. You guessed it… Test Design Studio has a rule for that too!
My process was simple… I would find one of the errors in the list and open that test. I’d then filter the error list to only show me the errors for the currently opened file. That let me know exactly how much work was before me. When I’d find the usage of an undeclared variable, I’d do a quick “Find in File” to see all the places in the test where the variable was used. Once the file was clean, I’d save/close, restore my error list to the unfiltered view, then pick the next error from another file.
Most of the time, the variables were legitimate and needed to be declared. Sometimes I found that variables were assigned values but never actually used in the test! That’s confusing to new authors trying to understand a test, so I got rid of those (they were left over from previous versions of the test that were later modified). The worst is when I found variables that were used but never assigned! The author of the test clearly expected a value to be in the variable, but it was always going to be blank. That could lead to the test not validating the functionality it was designed to test. Was this test even valid?
How Did It Turn Out?
After spending a couple hours going over all the tests, adding Option Explicit, and fixing all those variable references, I found several logic errors in the script that never would have happened if Option Explicit had been turned on to begin with (confirming my theory that you should always use Option Explicit). When the tests were executed, not a single run-time error was encountered from all those changes.
Tens of thousands of lines of code, and I didn’t miss a single variable… THANK YOU TEST DESIGN STUDIO!
Use Test Design Studio For Your Own Projects
You can find out more about how the Code Quality features in Test Design Studio on this special feature page. Of course, you can also download a fully functioning trial version of the software and try it out for yourself.
The most notable additions to this release are related to the build process. For those who may not be familiar with this feature, the build process allows Test Design Studio to perform routine tasks associated with the collection of files in your project.
A Little Background on Build
Perhaps the most important of the build tasks is the ability to update your QuickTest tests. QuickTest does not have a centralized approach to test configurations unless you use Business Process Testing. Each test is a self-contained entity requiring its own configuration. This is great if you write tests in isolation, but most users typically create a suite of many tests for a given application. Shared resources such as function libraries and object repositories are used by most (if not all) of those tests. When you add a new library or object repository to your shared resource pool, you must update each test to add this new resource. That process is time-consuming and easy to overlook.
The Test Design Studio build process changes that! Test Design Studio already has an excellent project-centric approach to test management, and keeping your tests updated is as easy as running the build process. Once initiated, Test Design Studio looks at all the tests in your project as well as all the shared resources. Each test is then opened in QuickTest through their API, object repositories are associated with each action, and function libraries are associated with each test. Test Design Studio even makes sure you have the proper add-ins set.
When I develop automated tests, I place most of my functionality in reusable function libraries. I keep libraries small and each one focuses on specific aspects of the application. This keeps my code organized and easy to manage, but can easily create 10 to 20 function libraries. Managing that many individual files would be cumbersome without Test Design Studio. When I need a new test, my actions are simple:
- Add the test to my project using the ‘Add New Item’ command and selecting my QuickTest file template
- Right-click the file in Solution Explorer and select ‘Build’
That’s it! My new test is immediately configured with all the function libraries and object repositories that are defined for my project. When I need to add a new function library to my projects, the process is equally simple:
- Add the library to my project using the ‘Add New Item’ command and selecting library file template
- Right-click the project in Solution Explorer and select ‘Build’
Just like that, Test Design Studio will process all of my tests to make sure they reference the new library.
What Changed in New Release
The build process is key to increasing your productivity and working in conjunction with QuickTest. One of our newer customers recently posted a series of suggestions for improving the build process, and we were pleased to make those enhancements (you know how you are, and thank you!).
Hint: If you have an idea on how to improve Test Design Studio, it’s a great idea to share it!
Test Left Open after Build
If you initiate the build process for a single test, Test Design Studio will now leave that test open in QuickTest after the build is complete. This is a great time-saver when actively writing and debugging a test since you can essentially use the ‘Build’ command to open the test in QuickTest (something always available off the context menu) but also ensure that all resources are properly associated to the test.
Version Control Improved
We made many adjustments to how version controlled tests are managed. A test must be checked out in order to modify it, so the build process will check out any test that was not already checked out. If no changes were necessary from the build, that check-out will be cancelled. When the test is modified, it will be checked back in only if the test was not already checked out. Those tests that were already checked out before initiating the build will remain checked out afterwards.
Keeping a test checked out proved to be a complex task due to an issue with the QuickTest API. If you save a version-controlled test and try to close it without checking it in, QuickTest displays a prompt asking if you want to check it in. It even does this if you are using the API, and that dialog was blocking the build process while waiting for user input. Even worse, the dialog displayed by QuickTest was often not visible on the desktop giving the illusion that Test Design Studio had stopped responding. To keep the build process operating smoothly, we actually had to use GUI automation to detect and dismiss that dialog when displayed.
Build Selected Command Updated
The ‘Build Selected’ command was originally designed to build the selected item in the Solution Explorer tool window. For those that do not have the Solution Explorer track the currently selected document, this caused some confusion. If you had one test selected in Solution Explorer but were actively editing a different test in an editor, the ‘Build Selected’ command would build the test in Solution Explorer instead of your current document. That has now changed. The command will only process the selection from Solution Explorer if that tool window is selected when activating the command. Otherwise, it will attempt to process the actively edited document.
For those not familiar with the build process, we hope you integrate it into your project management process. For those already using it, we hope you enjoy these enhancements. Thank you again to the customer who brought these suggestions to us, and keep that feedback coming! We all benefit from the ideas of others.
How safe do you feel knowing your code passes a syntax check?
Keeping reading and you might change your mind.
We have talked at length about having correct syntax does not mean your code is correct. Sure, we start with getting the syntax correct, but we must also follow the rules of the language.
The ability to create functions and return values is one of the greatest tools available to a programmer! VBScript, unfortunately, does not use a simple statement such as “return” to signify the return value (as is common in “C”-style languages). Instead, it makes you repeat the name of the function in a syntax similar to assigning a value to a variable. This works great, of course… provided you type the name correctly!
While working on support for the Web Services add-in for the next release of Test Design Studio, we opened the “Web_Services.txt” file that ships with QuickTest to ensure that the user-defined functions were being properly registered to the “WebService” object. That is when we found the issue below:
Do you see what’s wrong? Test Design Studio makes it pretty obvious that the author of this file mistyped the name of the function when trying to set the return value. What should have been “GetHeaders” was typed as “GetHeader”. Test Design Studio saw two potential problems here:
- “GetHeader” (since it does not match the name of the function) is the same as a reference to a variable. This file has Option Explicit turned on, and a variable named “GetHeader” does not exist. Test Design Studio will not let you use an undeclared variable when Option Explicit is enabled.
- Functions, by definition, are meant to return values. This function declaration does not (due to the mistyped name). Test Design Studio warns you that you might have forgotten to return a value.
This error means that anyone attempting to use the “GetHeaders” function attached to the “WebServices” object is either going to get a run-time error or, even worse, never get the object they were trying to retrieve.
Sadly, that’s not the only error Test Design Studio caught. Elsewhere in the same file was this code:
Again, Test Design Studio makes it obvious that there is a problem here! The code author misspelled the parameter name in the function declaration, which made the reference to the parameter (using the proper spelling) inaccurate. Test Design Studio gave the following reports:
- The code attempted to use an undeclared variable “XMLSchemaFile” (Option Explicit turned on). Since the name did not match the parameter, it was expecting the name to be defined elsewhere.
- A warning was given that the user declared the parameter “XMLScheamFile” and never actually used it.
This function, as defined, would never properly validate the XML Schema file.
Had the author of this code been using Test Design Studio and the powerful code analysis features, neither of these mistakes would have been distributed to thousands of HP customers! Keep in mind that the syntax of this file was 100% accurate even though the file had multiple errors.
So I ask again… how safe do you feel that syntax checking is enough to verify your code?
The two major flaws in this code were caused by 3 simple characters. What might you have missed in your code?
Note: This file was analyzed from the general release of QuickTest v11.0. I have not verified if the issues have been corrected in any patched release. What to check your file? The default location is “C:\Program Files\HP\QuickTest Professional\data\BPT_Resources”. You may also find the file automatically added to your Quality Center/ALM projects in the Resources module.
We are happy to announce that the next release of Test Design Studio will add support for Delphi test objects! Support for QuickTest test objects like those in the Delphi add-in are just one of the features that give Test Design Studio an advantage over standard VBScript editors like Notepad++. Many VBScript-based editors can provide support for standard VBScript language elements, but only Test Design Studio adds support specifically for objects that are unique to QuickTest!
The added capability will bring IntelliSense to the collection of objects associated with this add-in, including: DelphiButton, DelphiCheckBox, DelphiComboBox, DelphiEdit, DelphiEditor, DelphiList, DelphiListView, DelphiNavigator, DelphiObject, DelphiRadioButton, DelphiScrollBar, DelphiSpin, DelphiStatic, DelphiStatusBar, DelphiTable, DelphiTabStrip, DelphiTreeView, and DelphiWindow.
This addition means that Test Design Studio now has full IntelliSense for all of the following add-ins:
- Standard Windows (i.e. Win32)
- Visual Basic
- Terminal Emulator
Test Design Studio has long offered the ability to modify the fonts and colors used by the editing control. This is done by selecting ‘Tools –> Options’ from the main menu, expanding the ‘Environment’ node and selecting ‘Fonts and Colors’.
Until now, however, there was never a way to change the background of the text editor from the system default settings provided by the Windows theme in use. Many users have requested the ability to create a “dark” editing environment, and now you can! The screen shot below shows what is now possible with the new customization options:
This was achieved by the addition of the following new entries in the “Fonts and Colors” list:
- Text Area Background – Normal background color of the editor
- Text Area Background (Disabled) – Background color of the editor when editing is disabled (i.e. read-only)
- User Margin Background – Background color of the user margin (i.e. the area where icons are displayed for language elements)
- Selection Margin Background – Background color of the selection margin (i.e. the area where outlining indicators are displayed)
- Outlining Indicator – The Foreground color controls the border and lines of the outlining node, while the background color determines the background of the expand/collapse indicator.
- Line Numbers – While not a new entry, the “Background Color” is now supported to control the background of the line number margin.
We hope you enjoy this new level of customization that is coming in the next release (expected by mid-February).
Once again, another great feature request from one our users is being integrated into the next release of Test Design Studio!
Test Design Studio features extensive support for outlining the various code elements of your file so that you can collapse areas that you do not want to focus on at the time. The ‘Toggle Outlining Expansion’ command is what you use to expand/collapse the node closest to the caret position. But what about when you want to toggle a lot of nodes at once?
Previously, the only other option was ‘Toggle All Outlining’ that worked on the whole document. We have now expanded the functionality of the ‘Toggle Outlining Expansion’ command to support a selected range.
If you have not selected any text in the editor, the command functions just like it always did before. If you have selected a text range, all the nodes that are fully encompassed in the text range will either be expanded or collapsed. Since you are dealing with multiple outlined regions that could be in different expand/collapse states, the first encountered node in the region will determine if the other nodes are expanded or collapsed. So if the first node in the range is currently expanded, all nodes will be toggled to the collapsed state.
This subtle enhancement should make it that much easier to use outlining in Test Design Studio and further improve your productivity.
We are happy to provide another feature request in our next release of Test Design Studio. This feature, while simple, is very useful. When you press the ‘SPACE’ key after typing the ‘Call’ keyword, a member list will be displayed showing just the available methods that are accessible for you to call. This can make it easier to find what you’re looking for without displaying a full member list and ignoring all the irrelevant items like variables and keywords that are not valid in that context.
Note: This special feature is only available when you press the ‘SPACE’ key after the ‘Call’ method and will not be displayed if you use the standard options for showing a member list.
This feature is primarily targeted for use with Enumerations (a.k.a. Enum), but everything you see below will also work with Boolean values since it, too, has a pre-defined list of values (i.e. True or False). Support for Enum’s is something unique to Test Design Studio that you cannot find in QuickTest® Professional!
NOTE: This feature will be available in the next build of Test Design Studio
We have always supported the ability to setup the parameters of a method with an Enum list so that, when typing the arguments for your method call, you would be presented with a member list of available values for that parameter based on the associated Enum. We are happy to report that this option is now available when performing an assignment operation or equality comparison.
In the screen shot below, you can see an example of this feature. At the top of our script, we have defined an Enum for a series of strings called “Color”. Later, we declare a variable and use the XML comments to designate that this variable is expected to hold the values of the “Color” Enum instead of being treated as a normal string.
With this configuration in place, the Member List of available values is immediately displayed when you press "Space" after an "Equals" sign. You can also manually display the Member List (i.e. “CTRL+SPACE” for the Complete Word operation) of available values to assign to this variable. This feature can help ensure you are assigning the appropriate values.
The same feature is available when using the equals sign to do an equality comparison instead of assignment as shown below:
If you do not want to see the Enum-specific Member List, simply repeat the “CTRL+SPACE” shortcut command to toggle between the default Member List and the Enum-specific Member List.
Defining and using Enums is a feature unique to Test Design Studio that can easily help you ensure you are using data correctly. If you are not already using them, now is a great time to take a second look at the feature.
You can find more details about how to define an Enum in our on-line help.
This feature comes to you courtesy of a request from a user! We always strive to incorporate the features that are important to the users of our product, so we encourage you to keep those feature requests coming. All ideas are welcome! This feature was incorporated and made available to the user in less than 24 hours from receiving the request. We cannot promise that all features can be added or that you will have the same turnaround, but we hope you take this example as an indicator that we are serious about keeping our users happy. When is the last time you remember getting support like that from a software company!?
How you manage your QuickTest® Pro test assets will have a significant impact on the success of your project. We put together a white paper that discusses project management using either QC/ALM or the standard Windows® file system. We also go on to show you how Test Design Studio builds on these techniques to provide a superior experience.
These techniques are valuable to anyone working with QuickTest® Pro, and not just Test Design Studio customers. We hope you enjoy the white paper and will share with your colleagues.
The white paper can be downloaded here.
Test Design Studio does a lot that QuickTest® Professional does not! There are, however, a few elusive items that Test Design Studio does not (or cannot) do. You can view a feature comparison here. One of the biggest reasons for the lack of support has to do with how the QuickTest® Professional API is constructed.
Problems with the QuickTest® Professional API
A QTP test contains a lot of different data. Some of this data is easily viewable through standard text-based files, but a lot of it is locked up in binary files. These files, unfortunately, cannot be read without a proper interface, and that is where the QuickTest® Professional API comes in. The API allows you to read test properties, modify test resources, and other similar actions. The problem is that QTP was designed to only allow one test to be opened at a time.
So why is this a problem? Consider this scenario.
You have a test open in QTP that you are actively editing. At the same time, you have a different test open in Test Design Studio. If you want to do anything with your test in Test Design Studio that requires the API, the following must happen:
- Test Design Studio invokes a new instance of the QTP automation object. This does not create a new instance of QTP, however, since you cannot launch multiple sessions of QTP. Instead, it allows you to access and control the currently running instance.
- The test that you are currently working on in QTP must be closed so that the test you need from Test Design Studio can be opened. You cannot access the details of a QTP test without having it opened in the application.
- If the active test has been modified, you must save your changes before closing. If you were not done performing your edits, this places the test in a potentially corrupt state (e.g. syntax errors, incomplete code, incomplete logic).
- Once the test is open in QTP, Test Design Studio can finally read the details of the test.
As you can see, this process is very intrusive to the end user and makes it impossible for Test Design Studio to silently load data “behind the scenes” for a test.
Problems with QuickTest® Professional Actions
Test Design Studio includes basic support for existing Actions in a QuickTest® Professional test. This includes the ability to view the action code and make changes as necessary. What Test Design Studio cannot do is modify an Action or allow you to insert calls to new Actions that are not already part of the test.
Test Design Studio is essentially the equivalent of the ‘Expert View’ in QTP (although much more powerful and feature-rich). The following statement from the QTP v11 Help File regarding the “RunAction” statement explains the problem (emphasis added):
The RunAction statement can run only actions that are already associated with your test as part of the test flow. Therefore, in order to enter a RunAction statement in the Expert View for an external action, you must first insert a call to the action (Insert > Call to Action) or copy the external action (Insert > Copy of Action) into your test. Using this option associates the action with the testand also inserts a RunAction statement for you. After the external action is added to the test flow, you can add additional calls to that external action in the Expert View.
If you insert a RunAction statement in the Expert View for an external action that is not already associated with your test as a part of the test flow, the RunAction statement fails. For more information on copying or calling external actions, see the HP QuickTest Professional User Guide.
The key information to gain from that excerpt is that an Action must be associated with a QTP testin order for it to be called properly. The only way for Test Design Studio to associate an Action with a QTP test is using the QTP API, and those problems have already been noted.
NOTE: The limitations surrounding actions are why I (and many others in the industry) prefer using function libraries instead of Actions. Function libraries solve many of the same problems that Actions address, and are much easier to manipulate outside of QTP. In essence, Actions are largely just a “keyword view” equivalent of a function call.
The Tandem Solution
The solution to this issue is that you must use QuickTest® Professional in tandem with Test Design Studio for certain functionality. Many aspects of Test Design Studio were designed to work without the need for QTP to even be installed (thus a potential for saving money on QTP licenses), but some actions will simply require that QTP be available due to the need for the API.
Since we cannot live without QTP, we decided to make it as easy as possible to work with it. Test Design Studio offers two commands that makes this tandem solution work; “Open in QuickTest” and “Open Active QuickTest File”.
‘Open in QuickTest’ Command
This command is simple, yet very useful. It takes the file you are currently working with in Test Design Studio (QTP Test or Scripted BPT Component) and opens the same file in QTP. While opening, it also ensures that QTP is launched with the proper add-ins for your file. If it is not, Test Design Studio will re-launch QTP and select the corresponding add-ins for you. Test Design Studio will even make sure you are properly connected to the corresponding Quality Center or Application Lifecycle Management server. In other words, Test Design Studio does all the heavy lifting to make the process as quick and painless as possible.
You can access this command in two ways… either right-click on the document tab for your file and select ‘Open in QuickTest’, or you can right-click the corresponding file in the ‘Solution Explorer’ tool window and select ‘Open in QuickTest’.
This command is especially useful when you need to perform an action on your file that is only available from QTP. For example:
- Modify test/action parameters
- Insert a new action
- Call an existing action
- Modify test flow
When you are done, you simply save your changes and return to Test Design Studio where you will be prompted to re-load the file.
‘Open Active QuickTest File’ Command
This command is accessed from the ‘File –> Open’ menu in Test Design Studio.
The purpose for this command is the opposite of what is done with the ‘Open in QuickTest’ command. It will use the QTP API to determine the test you currently have open, and the same file will then be opened within Test Design Studio.
This command is useful for when you want to use the powerful editing capabilities of Test Design Studio instead of the basic editing experience you get from the ‘Export View’ in QTP. When you have finished making changes in Test Design Studio, you simply need to save your file in Test Design Studio and then re-open the file in QTP.
NOTE: QTP does not notify you when a file has been modified outside of the QTP IDE, so you have to remember re-load your test. Test Design Studio, of course, does notify you when a file has been modified.
Use Case Scenario
Let’s describe a scenario that would require tandem use of Test Design Studio and QuickTest® Professional. Imagine you currently have a QTP test open, and you need to add a call to a new action. Since Test Design Studio does not provide the feature, you need to use QTP. The steps would be as follows:
- Save your test in Test Design Studio so that all changes are committed.
- Right-click the document tab for your test (or corresponding file in ‘Solution Explorer’) and select the ‘Open in QuickTest’ command.
- Switch to the QTP application
- Perform the necessary steps in QTP to add your new action
- Save the test in QTP
- Switch back to Test Design Studio
- You will be prompted to load the modified test, so select ‘Yes’.
With those basic steps, you can easily use Test Design Studio to gain the powerful features it offers while still working with QuickTest® Professional when necessary. It may look like a lot of steps, but the process is pretty straight-forward and will become familiar as you use it.
Why Not Add Direct API Support in Test Design Studio?
The answer is pretty simple. For starters, you must have the QTP API available to make changes like this, so that means you already have QTP installed on your system. We would then have to duplicate all the GUI aspects of QTP within Test Design Studio while still supporting multiple versions of QTP. Even if we did use the API, we would still have to open the test in QTP (and interrupt the current session) just to make those changes.
Since QTP is available and the test will have to be opened in QTP anyway, it is much easier to use the familiar (and version-specific) GUI of QTP to perform those actions instead of duplicating the logic in Test Design Studio. If the QTP API ever allows us to interact with files without having to take control of the running QTP instance, then we will consider providing the built-in functionality.