| |
How To Use VNCRobot
This document is intended to give you an overview of how VNCRobot can
be used for various automated activities. Most of the use examples were
created to fit the originally designed purpose of the tool. A few ones
were submitted by community users who keep surpising me by inventing
new and new ways to use VNCRobot.
My kudos to you all!
Should you have any questions or suggestions, contact me through the contacts page.
Contents:
Example #1: Automated
Tasks
VNCRobot can automate virtually any action that user does on the
desktop. It's usage is not limited to software testing and it can
easily automate simple tasks which can make your life
and everyday job a
lot easier. Let's for example consider a guy called John who likes to
come to the
office every morning and read his E-mails. As he owns some IBM stock,
he'd love to get the trading results daily by E-mail.
VNCRobot offers a simple method to do it. The trick is to create an
automated script which connects to a VNC server, opens a URL in a
browser, takes a screenshot of the trade data and sends it to John. You
may use the Record&Replay
feature to generate the script in a matter of minutes. A sample
code for VNC server on Linux Gnome
desktop follows:
# Open the Run
Application window on
Gnome
Press Alt+F2 wait=4000
# Open the IBM
stock report on Google
Typeline "firefox
http://finance.google.com/finance?q=IBM&hl=en" wait=20000
# Take a
screenshot and send it to John
Screenshot
/tmp/ibm.jpg
Sendmail subject="IBM stock on {_DATE}" from="root@yourmachine" to="John.Doe@yourdomain"
server="smtpserver.yourdomain" attach="/tmp/ibm.jpg"
# Close the
browser so that we can rerun the script safely
Press Alt+F4 wait=500
Save the script to a file called example1.txt. You may
then execute
this task automatically any time by one of the following commands:
GUI mode
execution:
vncrobot.sh -c
localhost:1 -p <password> -r ~/example1.txt
CLI mode
execution:
vncrobot.sh -c
localhost:1 -p <password> -r ~/example1.txt -n
If you execute the task in CLI mode,
no GUI will be shown and all log messages will be printed out to the
console (command line, DOS prompt window). Running in CLI mode is
faster and requires less memory and system resources. On Unix/Linux
systems it also allows to run automated tasks remotely from a terminal
without a graphical display.
Execution in GUI mode will
display the VNCRobot GUI. You will have better control over the script
and you will be able to debug it's behavior. The GUI allows to pause or
stop the execution, set breakpoints in the code, make changes in the
script, validate the syntax etc. A screenshot of VNCRobot executing our
example script follows (click to enlarge).
Every time you execute the script, John receives an E-mail with
attached image similar to this one. If you use
cron (Linux task scheduler) to
invoke this task after the stock exchange trading hours every business
day, John
will be receiving his E-mails with the IBM stock data regularly.
This example is of course a bit silly and a similar result can be
achieved in many ways. It however demonstrates well VNCRobot's
capability to automate anything that can be performed on the remote
desktop. The idea is very
simple and the code is easy to understand. Thanks to Java and to wide
choice of operating systems supported by VNC servers you can set up a
similar task on virtually any configuration. The tradeoff is that the
script code will not be compatible across different systems, which is
however a nuisance common among almost all automated GUI testing tools.
Example #2: Automated
Functional And Regression Testing
Automated functional or regression testing typically involves execution
of larger test suites consisting of many steps and instructions. The
goal is to verify functionality of a software product.
Automated testing is more complicated than execution of automated task
described in the first example above.
There are more factors to consider:
- Limitations - Testing must be designed with
regards to the technology limitations,
- Verification
- Testing is more complex and there must be ways to verify the behavior
and
test results automatically,
- Reporting
- The test results and eventual failures need to be reported in an
appropriate way.
VNCRobot provides sufficient means to deal with all of these
requirements.
1. Limitations
Automated testing with VNCRobot must be
implemented with regards to limitations of the RFB protocol and VNC
technology. The following aspects need to be considered:
- The VNC client (i.e. VNCRobot) knows just the
image of the remote desktop. It is NOT
implicitly aware of which OS it displays or which applications or
windows are there. See the Verification chapter on how to analyze the
remote desktop content and verify the desktop and application behavior.
- Remote desktops behave differently depending on
OS version, installed software and user preferences. It means that if
you develop a test script on one VNC server, it may or may not work on
another one. To ensure stable and reliable behavior of VNCRobot
automated testing always use one VNC server to execute your test suite
(or eventually more VNC servers with identical desktop configuration).
- The RFB protocol is not able to transfer other
than ISO8859-1 (Latin-1) characters. It means that you will not be able
to simulate e.g. a user typing Chinese or Russian characters.
- It is not possible to execute two or more test
suites on one VNC server instance at a time. On Linux/Unix it's not a
problem because you can start as many server instances as you want and
you can use a single machine for parallel execution of many automated
tasks. Windows are more affected by this limitation because they can
typically run just one VNC server instance. To use a Windows machine
for parallel testing you may consider to take advantage of Windows
Terminal Services and connect VNCRobot through an RDP to VNC Protocol
Adaptor like rdp2vnc.
2.
Verification
There are three verification methods in
VNCRobot:
- Monitoring of
remote desktop image updates. Commands like Waifor update
allow you to verify update of the remote desktop image, e.g. when an
application window gets opened.
- Monitoring of
server beeps (bell characters). You may design CLI commands
executed on the remote desktop to beep (i.e. print the bell character)
in case of successful execution and take advantage of the Waifor bell
command to capture it.
- Image
comparison methods. VNCRobot 1.3 supports two methods called
"default" and "search". The first one is based on histogram comparison
and allows to generate a percentage indicating how much the remote
desktop image matches to a template image. The second one allows to
search the remote desktop image or its part for a smaller image, e.g.
an icon. Both methods are well described in the How to Use Image Comparison
document of the VNCRobot GUI documentation. Users may also allowed to
take advantage of the VNCRobot 1.3
Open API to write their own image comparison methods and plug them
into VNCRobot.
3.
Reporting
VNCRobot is able to capture screenshots of the
remote desktop or its part. The tool can also generate a simple HTML
report which includes the overall test result (pass/fail), screenshots,
descriptions, warnings and results of image comparison. For more
information see the corresponding scripting commands Screenshot, Report, Warning and image
comparison commands Compareto
and Waitfor
match.
For examples of simple automated test suites see the following
resources:
- Sample
test report which shows how to use VNCRobot reporting and
verification tools on a simple automated test case,
Example #3: Localization
Testing
Localization testing is usually functional testing of multilanguage
applications accross all supported locales or language packs. It
typically involves execution of the same test scenario on all supported
language configurations. The goal is to make sure that the application
behaves the same way with all supported languages and that translations
of the UI strings are correct and accurate.
Localization testing is a perfect candidate for automated testing
because you design the test suite once and then run it on many
configurations. Automation effectively separates the engineering work
from the linguistic one and you don't have to hire QA people who master
both the technical skills and foreign languages. It's enough to let the
engineers to automate the testing and then outsource review of the test
results like screenshots and reports to linguists.
VNCRobot is a good choice for automation of localization testing
provided that you keep the following rules:
- Design your test scripts to accept variable
values from the command line. See
-v/--variable switch in
the VNCRobot CLI Options
document for more information.
- When you create the automated test script,
avoid using mouse clicks, moves and drags to control the tested
application. Translations bring differencies into GUI component sizes
and behavior and mouse controls are then not reliable. Stick to the
keyboard where possible and control your application through simple
keys, shotcut/hot keys, mnemonic keys and focus controls like
Tab/Shift+Tab.
- Make sure your scripts leave the remote desktop
in the same state as before execution. A good idea is to start always
on a clean desktop which is not running any visible application.
Calling of Ctrl+F4 several times at the beginning of your script will
ensure it.
- If you decide to save all the output into a
single folder, make the language code part of the screenshot and report
file names to prevent them from overwriting.
The following example shows a simple localization testing of Firefox on
Ubuntu in several different languages: English (default language),
Spanish, French, German, Italian, Russian, Japanese and Simplified
Chinese. The script is very simple and follows the rules stated
earlier. We will take advantage of Firefox's CLI option -UILocale
which allows to start the browser in the language specified by a
language identifier (which is in fact identical with the user preferred
language sent withing the HTTP request header). Note that you must have
the specified Firefox language packs installed to achieve this. The
language identifier (e.g. en-us, es-es) is in the script defined by a
variable called LANG and its value can be later on
rewritten through the -v/--variable VNCRobot CLI option.
The script further on performs a very basic browser testing - it opens
the File menu,
starts the Preferences and Help windows and displays the Browse bar at
the bottom of the browser window. Three screenshots are taken along the
way and a report is generated for each language.
Var LANG=en-us
# This procedure
will open Firefox in the locale specified by argument.
# It will then open
the browser menu, Preferences and Help windows and Search bar.
# Screenshots will
be taken at major points.
# Parameter: {1} -
desired browser locale, e.g. en-us, es-es, de-de
procedure
test_firefox {
# Close all
windows and open the Run Application window on Gnome
Press Alt+F4 count=2 wait=1s
Press Alt+F2 wait=4s
# Start Firefox
in desired language
Typeline "firefox -UILocale
{1}"
# Wait for major
update of the remote screen
Waitfor update extent="50%" timeout="30s" ontimeout="Exit 1"
# Open the
Firefox menu - keys F10 and "arrow down"
Press F10 wait=200
Press Down wait=1s
Screenshot
firefox-menu-{1}.jpg
desc="Firefox File menu. It should be properly
localized into {1}."
# Open the
Firefox Preferences window.
# As the File menu is open, key "arrow right" will display the Edit menu
# and key "arrow up" will jump to the last menu item Preferences
Press Right wait=500
Press Up wait=500
Press Enter wait=5s
Screenshot
firefox-preferences-{1}.jpg
desc="Firefox Preferences window. It should be
properly localized into {1}."
# Escape will
dispose the Preferences window
Press Esc wait=500
# Display the
Firefox Search bar (F3 key) and open the Help window (F1).
Press F3 wait=1s
Press F1 wait=5s
Screenshot
firefox-help-{1}.jpg
desc="Firefox Help window and the Search bar
(at the bottom). It should be properly localized into {1}."
# Close the
browser so that we can rerun the script safely
Press Alt+F4 count=2 wait=1s
}
# Start an HTML
report
Report index-{LANG}.html
desc="Example of VNCRobot automated
localization testing of Firefox. See the <a
href=http://www.vncrobot.com/docs/examples/index.html#example3>How
To Use
VNCRobot</a> document for the list of all examples."
# Call the test
procedure.
# Value of the
argument variable can be passed from CLI.
test_firefox {LANG}
To execute the script on a local VNC server we need to pass the desired
language identifier through CLI. The command will look like this:
vncrobot.sh -c
localhost:1 -p <password> -r example3.txt -n -v
LANG=<language_identifier>
To test Firefox in the eight languages discussed above we need to
execute the command sequentially eight times, once per each language
identifier. The following table lists reports from automated execution
for all suggested languages.
Example #4: Availability
Monitoring
Another smart idea is to use VNCRobot to monitor availability of
services or applications and detect failures and server crashes. The
principle is to design and run an automated test script which
repeatedly accesses a service or an application in an infinite loop. If
the service or application fails to display, VNCRobot can raise an
alarm, e.g. send an E-mail to the administrator, or even restart the
service.
Note that there are many tools allowing to test availability of some
types of services or applications in a more effective way than
VNCRobot.There's for example a bunch of good tools for web
applications. VNCRobot testing is however easy to implement/maintain
and it can test virtually anything capable of displaying on
the remote desktop, regardless whether it is a browser, Java/C
application, CLI command etc. The tool is especially good to "glue"
testing of integrated heterogennous application stacks where you need a
special tool for each application type otherwise.
As an example we will test availability of the Apache web server on a
local machine. There's a page at http://localhost/vncrobot which
represents our tested service. It is displayed in a Firefox browser on
the following picture.
To monitor it's availability
consistently we need to develop a test script which repeatedly loads
the page in the browser and verifies that the it is displayed
correctly. It must be robust enough to deal with a browser crash.
Tho verify both browser and test site behavior we will take advantage
of two image templates. Both of them were cut from the remote desktop
image using the VNCRobot's Compareto
Command Window. You should be able to locate them easily on the
picture linked above.
Image Name
|
Image Description
|
Image
|
firefox_icon.png
|
Part
of the
Firefox "Go" icon.
|

|
apache.png
|
Part
of the
"Powered by Apache" logo image.
|

|
A sample code for a VNC server on
Linux Gnome
desktop follows:
# Send an E-mail
to the administrator.
# Param {1}: E-mail subject.
procedure send_mail {
Sendmail
subject="{1}" from="root@yourmachine.com" to="John.Doe@yourdomain.com"
server="smtpserver.yourdomain"
}
# This procedure performs cleanup and starts Firefox
procedure
start_firefox {
# Close all windows and start Firefox
Press Alt+F4 count=3 wait=3s
Press Alt+F2 wait=4s
Typeline "firefox
http://localhost/vncrobot"
# Wait max 15 seconds for Firefox to start.
# Presence
of the browser is verified by existence of the "Go" icon.
Waitfor match template="firefox_icon.png" interval=3s method="search" timeout=15s
# This will be executed when Firefox fails
to start.
# An E-mail
is sent to administrator and the script exits.
if ({_EXIT_CODE} > 0) {
send_mail "Firefox failed to start! Please fix and
restart the VNCRobot testing."
Exit 1
}
}
# Variable
indicating the test result
Var FAILED=0
# Infinite loop
for (; 1==1;) {
# Reload our test page and verify if it
gets correctly displayed.
# We are in
fact looking for part of the "Powered by Apache" image.
Press Ctrl+R wait=4s
Compareto
"apache.png" method="search"
# This will be executed when the last
command succeeds
if ({_EXIT_CODE} == 0) {
Var FAILED=0
} else if ({FAILED} == 0) {
# Verify if Firefox
is there and restart it if not.
Compareto
"firefox_icon.png"
method="search"
if
({_EXIT_CODE} == 0) {
# It is a first
detected failure so send an E-mail and set the test result to 1.
# It will prevent sending of multiple E-mails for each failure.
send_mail "Your test application is not available!
Please fix it."
Var FAILED=1
} else {
# Restart Firefox
start_firefox
}
}
# Wait 5 seconds before trying again
Wait 5s
}
The script code is be easy to understand. The start_firefox
procedure starts the Firefox browser and opens the test URL. The
procedure uses the Waitfor
match command together with the 'search'
image comparison
method to verify whether the Firefox icon is displayed on the
screen.
The script body consists of an infinite loop. Each cycle starts with
Ctrl+R which makes Firefox to reload the page. The script then looks
for
part of the Apache logo. If the image is not found, the script checks
if Firefox is visible to rule out browser crash and restarts it in case
of need. If the logo is not visible while Firefox is displayed
correctly, we detected a crash of our service and a warning E-mail is
sent to the administrator. A picture of VNCRobot executing the example
script in GUI mode follows.
To test the script functionality, kill the browser or the tested
service (Apache server).
You should get an E-mail notifying you of the crash. It is of course
possible to implement a service restart within the script. See the Exec command for more
information.
Example #5: Load Testing
VNCRobot can be successfuly used to generate load for stress testing.
The idea is to start
many parallel testing processes to simulate a number of users accessing
a
service or using an application. As VNCRobot simulates a real user
behavior in a graphical environment, such a scenario will
create load on the tested system which is very similar to real
production
environment.
VNCRobot
supports has a Java API which allows to create multiple testing threads
within a single Java Virtual Machine. Each thread may independently
execute one test script on a single VNC server instance. See the
VNCRobotThread interface of the VNCRobot
1.3 Open API for more information. Note that Unix/Linux systems are
prefered to host the load tests
because unlike Windows they are capable of running many VNC
server instances on a single machine.
Let's create an example program generating load on a tested
application. We need to create three components:
- Test script in
the VNCRobot scripting language which will represent a typical user
session with the tested application.
- Java wrapper which
will take advantage of the VNCRobot Open API to create and start
parallel test threads.
- Shell wrapper
which will start and eventually shut down the VNC servers (Linux/Unix
only).
1. Test Script
There is no test script included in this
example. You need to create it
before and integrate it with the Java and shell wrappers described
below. See the previous examples in this document on
how to create a test script in the VNCRobot scripting language. You may
easily use the Record&Replay
feature to do so. It is highly recommended to use a small remote
desktop size (e.g. 800x600 pixels) and avoid image comparison and
taking of screenshots. All these factors significantly increase the
amount of required memory and CPU time. Another good idea is to set
your remote desktop background (wallpaper) to a solid color.
In the next steps we will suppose that the test script already exists
and is saved in a file called client.txt.
2. Java Wrapper
Java wrapper is a program which creates and starts the automated
testing threads using the VNCRobot
1.3 Open API. A universal wrapper code is displayed below.
import com.vncrobot.VNCRobot; import com.vncrobot.api.VNCRobotThread;
public class LoadExample { final static int DEFAULT_THREAD_COUNT = 50; public static void main(String[] argv) { int threadCount = DEFAULT_THREAD_COUNT; String password = null; String script = null; if (argv.length > 1 && argv.length < 4) { try { threadCount = Integer.parseInt(argv[0]); script = argv[1]; if (argv.length > 2) { password = argv[2]; } } catch (NumberFormatException ex) { System.out.println("Invalid number of threads - default of "+DEFAULT_THREAD_COUNT+" will be used."); } } else { System.out.println("Usage: LoadExample <number_of_threads> <script_path> [<VNC_password>]"); return; } String[] threadArguments; if (password == null) { threadArguments = new String[] { "--connect", "", "--run", script, "--nodisplay" }; } else { threadArguments = new String[] { "--connect", "", "--run", script, "--nodisplay", "--password", password }; } VNCRobot robot = new VNCRobot();
VNCRobotThread runnable; Thread thread; for (int i=1; i<=threadCount; i++) { threadArguments[1] = "localhost:" + i; runnable = robot.createVNCRobotThread(threadArguments[1], threadArguments, System.out, false); thread = new Thread(runnable); thread.start(); } } }
To run the wrapper compile the Java code to .class or take advantage of
the precompiled class in the attached ZIP file.
The program accepts up to three parameters - desired number of threads
to be started (mandatory), path to the testing script (mandatory) and
VNC password (optional). The command should look like:
java -classpath
.:./vncrobot.jar LoadExample <thread_number> <script_path>
[<password>]
3. Shell Wrapper
Before you start the VNCRobot load testing, you always need to start
all the necessary VNC servers. You should also verify whether any
application involved in the testing allows only one instance per user.
A typical example is Firefox which will not allow a single user to run
more than one browser. To resolve this you will need to create a
different user for each test thread and start the VNC server under his
account.
All these necessary overhead tasks can be efficiently automated on
Linux/Unix by simple shell scripts. Let's suppose that our example
application doesn't have any instance limitations. All we need to do
then is to start the VNC servers, execute the Java load application and
kill the VNC servers after the testing gets finished. A Bash script
handling these tasks follows.
#!/bin/bash
# Number of threads to be executed
THREADCOUNT=10
# Path to the automated script
SCRIPT=client.txt
# VNC password. If your VNC server doesn't require password,
# leave the variable empty.
PASSWORD=welcome
# This 'for' statement will start VNC servers on local machine
# on ports 1 to THREADCOUNT
for ((i=1; i<=$THREADCOUNT; i++))
do
vncserver :$i
done
# The following command will execute the Java wrapper with VNCRobot
# automated threads
java -Xmx512m -classpath .:./vncrobot.jar LoadExample $THREADCOUNT
$SCRIPT $PASSWORD
# This 'for' statement will kill all the VNC servers started
# at the beginning of this script
for ((i=1; i<=$THREADCOUNT; i++))
do
vncserver -kill :$i
done
The shell wrapper source together with the Java wrapper are available
for download. Before you execute it,
set correct values of the THREADCOUNT, SCRIPT
and PASSWORD variables.
If you define the number of threads too high, some of them may start to
fail with java.lang.OutOfMemoryError. To fix this issue
increase the JVM memory heap size specified by the -Xmx
Java option in runLoad.sh (default value is set to 512MB).
|
|
|