I am happy to announce that I will be presenting a session at Software Test Professionals Conference 2011 Fall in Dallas, TX!
Session Title: Improve Automation Code Quality, Clarity, and Comprehension
Session Track: Test Automation
Session #: 602
Date/Time: Wednesday, October 26, 11:30am – 12:45pm
How many times have you looked at someone else’ automation code and
wondered what they were doing? How about your own code months after you
wrote it? Much of the effort to perform test automation comes months or even
years after the initial scripts have been written. Before you can begin to debug
broken code or write new functionality for a test, you must first comprehend
what was previously written.
This session will teach you about many of the common pitfalls in authored code
and how you can refactor your code to dramatically improve comprehension.
Topics of discussion will include good naming conventions for variables and
methods, when to use callable routines, effective use of parameters in routines,
variable usage, avoiding “magic values”, code formatting, and objective
measurements of code complexity. Examples from the presentation will be
provided based on HP QuickTest(R) Professional, but the concepts can be
universally applied to any programming language.
This will be my first time presenting at a conference, so I am very excited about the opportunity and look forward to bringing the spotlight to code quality. Perhaps equally exciting is the chance to meet many of the Test Design Studio users out there. Are you planning to attend the conference? If so, drop me an e-mail. And of course… be sure to sign up for my session!
How you manage your QuickTest® Pro test assets will have a significant impact on the success of your project. We put together a white paper that discusses project management using either QC/ALM or the standard Windows® file system. We also go on to show you how Test Design Studio builds on these techniques to provide a superior experience.
These techniques are valuable to anyone working with QuickTest® Pro, and not just Test Design Studio customers. We hope you enjoy the white paper and will share with your colleagues.
The white paper can be downloaded here.
Test Design Studio does a lot that QuickTest® Professional does not! There are, however, a few elusive items that Test Design Studio does not (or cannot) do. You can view a feature comparison here. One of the biggest reasons for the lack of support has to do with how the QuickTest® Professional API is constructed.
Problems with the QuickTest® Professional API
A QTP test contains a lot of different data. Some of this data is easily viewable through standard text-based files, but a lot of it is locked up in binary files. These files, unfortunately, cannot be read without a proper interface, and that is where the QuickTest® Professional API comes in. The API allows you to read test properties, modify test resources, and other similar actions. The problem is that QTP was designed to only allow one test to be opened at a time.
So why is this a problem? Consider this scenario.
You have a test open in QTP that you are actively editing. At the same time, you have a different test open in Test Design Studio. If you want to do anything with your test in Test Design Studio that requires the API, the following must happen:
- Test Design Studio invokes a new instance of the QTP automation object. This does not create a new instance of QTP, however, since you cannot launch multiple sessions of QTP. Instead, it allows you to access and control the currently running instance.
- The test that you are currently working on in QTP must be closed so that the test you need from Test Design Studio can be opened. You cannot access the details of a QTP test without having it opened in the application.
- If the active test has been modified, you must save your changes before closing. If you were not done performing your edits, this places the test in a potentially corrupt state (e.g. syntax errors, incomplete code, incomplete logic).
- Once the test is open in QTP, Test Design Studio can finally read the details of the test.
As you can see, this process is very intrusive to the end user and makes it impossible for Test Design Studio to silently load data “behind the scenes” for a test.
Problems with QuickTest® Professional Actions
Test Design Studio includes basic support for existing Actions in a QuickTest® Professional test. This includes the ability to view the action code and make changes as necessary. What Test Design Studio cannot do is modify an Action or allow you to insert calls to new Actions that are not already part of the test.
Test Design Studio is essentially the equivalent of the ‘Expert View’ in QTP (although much more powerful and feature-rich). The following statement from the QTP v11 Help File regarding the “RunAction” statement explains the problem (emphasis added):
The RunAction statement can run only actions that are already associated with your test as part of the test flow. Therefore, in order to enter a RunAction statement in the Expert View for an external action, you must first insert a call to the action (Insert > Call to Action) or copy the external action (Insert > Copy of Action) into your test. Using this option associates the action with the testand also inserts a RunAction statement for you. After the external action is added to the test flow, you can add additional calls to that external action in the Expert View.
If you insert a RunAction statement in the Expert View for an external action that is not already associated with your test as a part of the test flow, the RunAction statement fails. For more information on copying or calling external actions, see the HP QuickTest Professional User Guide.
The key information to gain from that excerpt is that an Action must be associated with a QTP testin order for it to be called properly. The only way for Test Design Studio to associate an Action with a QTP test is using the QTP API, and those problems have already been noted.
NOTE: The limitations surrounding actions are why I (and many others in the industry) prefer using function libraries instead of Actions. Function libraries solve many of the same problems that Actions address, and are much easier to manipulate outside of QTP. In essence, Actions are largely just a “keyword view” equivalent of a function call.
The Tandem Solution
The solution to this issue is that you must use QuickTest® Professional in tandem with Test Design Studio for certain functionality. Many aspects of Test Design Studio were designed to work without the need for QTP to even be installed (thus a potential for saving money on QTP licenses), but some actions will simply require that QTP be available due to the need for the API.
Since we cannot live without QTP, we decided to make it as easy as possible to work with it. Test Design Studio offers two commands that makes this tandem solution work; “Open in QuickTest” and “Open Active QuickTest File”.
‘Open in QuickTest’ Command
This command is simple, yet very useful. It takes the file you are currently working with in Test Design Studio (QTP Test or Scripted BPT Component) and opens the same file in QTP. While opening, it also ensures that QTP is launched with the proper add-ins for your file. If it is not, Test Design Studio will re-launch QTP and select the corresponding add-ins for you. Test Design Studio will even make sure you are properly connected to the corresponding Quality Center or Application Lifecycle Management server. In other words, Test Design Studio does all the heavy lifting to make the process as quick and painless as possible.
You can access this command in two ways… either right-click on the document tab for your file and select ‘Open in QuickTest’, or you can right-click the corresponding file in the ‘Solution Explorer’ tool window and select ‘Open in QuickTest’.
This command is especially useful when you need to perform an action on your file that is only available from QTP. For example:
- Modify test/action parameters
- Insert a new action
- Call an existing action
- Modify test flow
When you are done, you simply save your changes and return to Test Design Studio where you will be prompted to re-load the file.
‘Open Active QuickTest File’ Command
This command is accessed from the ‘File –> Open’ menu in Test Design Studio.
The purpose for this command is the opposite of what is done with the ‘Open in QuickTest’ command. It will use the QTP API to determine the test you currently have open, and the same file will then be opened within Test Design Studio.
This command is useful for when you want to use the powerful editing capabilities of Test Design Studio instead of the basic editing experience you get from the ‘Export View’ in QTP. When you have finished making changes in Test Design Studio, you simply need to save your file in Test Design Studio and then re-open the file in QTP.
NOTE: QTP does not notify you when a file has been modified outside of the QTP IDE, so you have to remember re-load your test. Test Design Studio, of course, does notify you when a file has been modified.
Use Case Scenario
Let’s describe a scenario that would require tandem use of Test Design Studio and QuickTest® Professional. Imagine you currently have a QTP test open, and you need to add a call to a new action. Since Test Design Studio does not provide the feature, you need to use QTP. The steps would be as follows:
- Save your test in Test Design Studio so that all changes are committed.
- Right-click the document tab for your test (or corresponding file in ‘Solution Explorer’) and select the ‘Open in QuickTest’ command.
- Switch to the QTP application
- Perform the necessary steps in QTP to add your new action
- Save the test in QTP
- Switch back to Test Design Studio
- You will be prompted to load the modified test, so select ‘Yes’.
With those basic steps, you can easily use Test Design Studio to gain the powerful features it offers while still working with QuickTest® Professional when necessary. It may look like a lot of steps, but the process is pretty straight-forward and will become familiar as you use it.
Why Not Add Direct API Support in Test Design Studio?
The answer is pretty simple. For starters, you must have the QTP API available to make changes like this, so that means you already have QTP installed on your system. We would then have to duplicate all the GUI aspects of QTP within Test Design Studio while still supporting multiple versions of QTP. Even if we did use the API, we would still have to open the test in QTP (and interrupt the current session) just to make those changes.
Since QTP is available and the test will have to be opened in QTP anyway, it is much easier to use the familiar (and version-specific) GUI of QTP to perform those actions instead of duplicating the logic in Test Design Studio. If the QTP API ever allows us to interact with files without having to take control of the running QTP instance, then we will consider providing the built-in functionality.
HP announced end of life for WinRunner® a couple of years ago. By this time, most people have migrated over to QuickTest® Professional instead. Since TDS 3 is a total rewrite of the underlying application code, we have decided that WinRunner® support will not be part of that effort. It simply does not make sense to spend resources on a product that HP no longer supports.
We hate to see WinRunner® go. I personally began my automation career with WinRunner® 6.0, so it closing a chapter in my working life. Even so, I fully understand and support HP’s decision to push a single functional testing tool, and QTP was the clear choice between the two.
Is lack of WinRunner® support going to impact you? If so, sound off in the comments. It’s possible we could still add basic support for opening WinRunner® scripts and TSL syntax highlighting, but advanced language support (like IntelliSense®, syntax checking, etc.), would not be available.
Do you think it is time for QTP to die? Paul Hammant posted on his blog that he thinks so. He makes some good points, but I disagree.
His first point discusses developers not using the tool. In my experience, most developers have rejected the tool when offered or failed to use it once it was delivered. Not because it was too expensive (concurrent licensing can ease the installation burden), but because they did not want to take the time to automate their tests. Even in this modern day, many development shops still are not writing unit tests! In the ideal world, all applications would have undergone a battery of automated tests before turning over the project to a testing team, but that is not the world most of us live in.
His second point discusses new lightweight testing tools that are either free or come at a low-cost. The terms “free” and “low-cost” should not be taken lightly, as the cost to actually implement and use a tool is never free! You have to be trained on the tool, gain experience on how it is used, and troubleshoot software issues with the tool itself. Some of these tools are not actively supported or lack a lot of the training material to get testers up to speed quickly. One of the biggest benefits to QTP is that you have “one tool to rule them all”. A single tool to handle testing a variety of technologies should not be underestimated. Sure, it may take a little time to adjust to testing a WPF-based application in QTP if you’ve only ever tested web-based applications, but at least you maintain the familiarity with the base tool and underlying automation concepts.
The remainder of his article primarily focuses on the growing support of Selenium. I have not personally used Selenium and have only recently looked at the product, but my initial thoughts were far from positive. The first webinar I attended about the technology did more to talk about what Selenium cannot do instead of what it is capable of. I have since attended additional webinars on the tool, and have yet to see a good presentation on how you implement this technology. Perhaps I have been spoiled by the IDE that QTP provides or the tight integration with Quality Center / ALM. To gain anything close to the QTP/QC/ALM combo, you have to start purchasing additional, 3rd-party tools. In the end, remember that investing in Selenium is also an investment in a single technology for web-based testing. If you ever need to automate .NET or Win32 applications, you will have to seek out, learn, and implement an entirely new technology.
Does QTP have it’s faults? Of course it does! The whole reason we initially developed Test Design Studio was to cover some of those faults. The fact remains, however, that QTP (even with it’s flaws) is a far more robust and mature product than any of the “free” or “low-cost” tools we see on the market today. Combine that with Test Design Studio, and you have a winning combination!
What do you think? Feel free to post your thoughts in the comments.
The ‘Start Page’ for Test Design Studio includes the ability to display an RSS feed of relevant information. By default, this feed is based on our general Test Design Studio news feed located here:
You have always had the ability to change this to any RSS feed you like. If you want to follow new blog entries, you could change your RSS feed to that of our blog, which is:
To do so, select ‘Tools –> Options’ from the main menu. On the ‘Options’ dialog, navigate to ‘Environment –> Startup’ to see the options for the startup page. Enter the above URL for the RSS feed and Test Design Studio will change to the new feed.
Click ‘OK’ and the start page will be updated.
NOTE: The content available from the Blog RSS feed is more feature rich than the standard news feed, and the extra formatting will not all be processed for display on the Start Page. Even so, you can easily grab the headlines and view the basic content before clicking the URL to jump to the full page.
The next release of Test Design Studio 2.0 will include a new feature to perform file locking in Quality Center / ALM. When you open a test, attachment, resource, or BPT component for editing, Test Design Studio will attempt to acquire a lock on that file. If the file is already locked by another user (either through Test Design Studio or tools like QuickTest), you will be notified that the file is locked. At that time, the file will open in read-only mode.
This new feature should help teams collaborate without impacting the work of someone else. If you prefer to disable the locking feature, you may do so using Test Design Studio options. Select ‘Tools –> Options’ from the main menu, then expand the path for ‘HP –> Quality Center’. Here you will find the option to enable/disable file locking.
We hope you enjoy this new feature coming in build 2.0.4571.
[Update] The new release of Test Design Studio 2.0.4572 is now available and includes this new feature.