Home | Roadmap | Documentation | Downloads | Contact Us 


      

How To Use VNCRobot

This document is intended to give you an overview of how VNCRobot can be used for various automated activities. Most of the use examples were created to fit the originally designed purpose of the tool. A few ones were submitted by community users who keep surpising me by inventing new and new ways to use VNCRobot. My kudos to you all!

Should you have any questions or suggestions, contact me through the contacts page.

Contents:

Example #1: Automated Tasks

VNCRobot can automate virtually any action that user does on the desktop. It's usage is not limited to software testing and it can easily automate simple tasks which can make your life and everyday job a lot easier. Let's for example consider a guy called John who likes to come to the office every morning and read his E-mails. As he owns some IBM stock, he'd love to get the trading results daily by E-mail.

VNCRobot offers a simple method to do it. The trick is to create an automated script which connects to a VNC server, opens a URL in a browser, takes a screenshot of the trade data and sends it to John. You may use the Record&Replay feature to generate the script in a matter of minutes. A sample code for VNC server on Linux Gnome desktop follows:

# Open the Run Application window on Gnome
Press Alt+F2 wait=4000

# Open the IBM stock report on Google
Typeline "firefox http://finance.google.com/finance?q=IBM&hl=en" wait=20000

# Take a screenshot and send it to John
Screenshot /tmp/ibm.jpg
Sendmail subject="IBM stock on {_DATE}" from="root@yourmachine" to="John.Doe@yourdomain"
    server="smtpserver.yourdomain" attach="/tmp/ibm.jpg"


# Close the browser so that we can rerun the script safely
Press Alt+F4 wait=500

Save the script to a file called example1.txt. You may then execute this task automatically any time by one of the following commands:

GUI mode execution:
vncrobot.sh -c localhost:1 -p <password> -r ~/example1.txt

CLI mode execution:
vncrobot.sh -c localhost:1 -p <password> -r ~/example1.txt -n

If you execute the task in CLI mode, no GUI will be shown and all log messages will be printed out to the console (command line, DOS prompt window). Running in CLI mode is faster and requires less memory and system resources. On Unix/Linux systems it also allows to run automated tasks remotely from a terminal without a graphical display.

Execution in GUI mode will display the VNCRobot GUI. You will have better control over the script and you will be able to debug it's behavior. The GUI allows to pause or stop the execution, set breakpoints in the code, make changes in the script, validate the syntax etc. A screenshot of VNCRobot executing our example script follows (click to enlarge).

VNCRobot executing example from use case #1

Every time you execute the script, John receives an E-mail with attached image similar to this one. If you use cron (Linux task scheduler) to invoke this task after the stock exchange trading hours every business day, John will be receiving his E-mails with the IBM stock data regularly.

This example is of course a bit silly and a similar result can be achieved in many ways. It however demonstrates well VNCRobot's capability to automate anything that can be performed on the remote desktop. The idea is very simple and the code is easy to understand. Thanks to Java and to wide choice of operating systems supported by VNC servers you can set up a similar task on virtually any configuration. The tradeoff is that the script code will not be compatible across different systems, which is however a nuisance common among almost all automated GUI testing tools.


Example #2: Automated Functional And Regression Testing

Automated functional or regression testing typically involves execution of larger test suites consisting of many steps and instructions. The goal is to verify functionality of a software product.

Automated testing is more complicated than execution of automated task described in the first example above. There are more factors to consider:
  1. Limitations - Testing must be designed with regards to the technology limitations,
  2. Verification - Testing is more complex and there must be ways to verify the behavior and test results automatically,
  3. Reporting - The test results and eventual failures need to be reported in an appropriate way.
VNCRobot provides sufficient means to deal with all of these requirements.

1. Limitations
Automated testing with VNCRobot must be implemented with regards to limitations of the RFB protocol and VNC technology. The following aspects need to be considered:
  • The VNC client (i.e. VNCRobot) knows just the image of the remote desktop. It is NOT implicitly aware of which OS it displays or which applications or windows are there. See the Verification chapter on how to analyze the remote desktop content and verify the desktop and application behavior.
  • Remote desktops behave differently depending on OS version, installed software and user preferences. It means that if you develop a test script on one VNC server, it may or may not work on another one. To ensure stable and reliable behavior of VNCRobot automated testing always use one VNC server to execute your test suite (or eventually more VNC servers with identical desktop configuration).
  • The RFB protocol is not able to transfer other than ISO8859-1 (Latin-1) characters. It means that you will not be able to simulate e.g. a user typing Chinese or Russian characters.
  • It is not possible to execute two or more test suites on one VNC server instance at a time. On Linux/Unix it's not a problem because you can start as many server instances as you want and you can use a single machine for parallel execution of many automated tasks. Windows are more affected by this limitation because they can typically run just one VNC server instance. To use a Windows machine for parallel testing you may consider to take advantage of Windows Terminal Services and connect VNCRobot through an RDP to VNC Protocol Adaptor like rdp2vnc.
2. Verification
There are three verification methods in VNCRobot:
  1. Monitoring of remote desktop image updates. Commands like Waifor update allow you to verify update of the remote desktop image, e.g. when an application window gets opened.
  2. Monitoring of server beeps (bell characters). You may design CLI commands executed on the remote desktop to beep (i.e. print the bell character) in case of successful execution and take advantage of the Waifor bell command to capture it.
  3. Image comparison methods. VNCRobot 1.3 supports two methods called "default" and "search". The first one is based on histogram comparison and allows to generate a percentage indicating how much the remote desktop image matches to a template image. The second one allows to search the remote desktop image or its part for a smaller image, e.g. an icon. Both methods are well described in the How to Use Image Comparison document of the VNCRobot GUI documentation. Users may also allowed to take advantage of the VNCRobot 1.3 Open API to write their own image comparison methods and plug them into VNCRobot.

3. Reporting
VNCRobot is able to capture screenshots of the remote desktop or its part. The tool can also generate a simple HTML report which includes the overall test result (pass/fail), screenshots, descriptions, warnings and results of image comparison. For more information see the corresponding scripting commands Screenshot, Report, Warning and image comparison commands Compareto and Waitfor match.

For examples of simple automated test suites see the following resources:
  • Sample test report which shows how to use VNCRobot reporting and verification tools on a simple automated test case,
Sample VNCRobot report
VNCRobot 1.2 Demo


Example #3: Localization Testing

Localization testing is usually functional testing of multilanguage applications accross all supported locales or language packs. It typically involves execution of the same test scenario on all supported language configurations. The goal is to make sure that the application behaves the same way with all supported languages and that translations of the UI strings are correct and accurate.

Localization testing is a perfect candidate for automated testing because you design the test suite once and then run it on many configurations. Automation effectively separates the engineering work from the linguistic one and you don't have to hire QA people who master both the technical skills and foreign languages. It's enough to let the engineers to automate the testing and then outsource review of the test results like screenshots and reports to linguists.

VNCRobot is a good choice for automation of localization testing provided that you keep the following rules:
  • Design your test scripts to accept variable values from the command line. See -v/--variable switch in the VNCRobot CLI Options document for more information.
  • When you create the automated test script, avoid using mouse clicks, moves and drags to control the tested application. Translations bring differencies into GUI component sizes and behavior and mouse controls are then not reliable. Stick to the keyboard where possible and control your application through simple keys, shotcut/hot keys, mnemonic keys and focus controls like Tab/Shift+Tab.
  • Make sure your scripts leave the remote desktop in the same state as before execution. A good idea is to start always on a clean desktop which is not running any visible application. Calling of Ctrl+F4 several times at the beginning of your script will ensure it.
  • If you decide to save all the output into a single folder, make the language code part of the screenshot and report file names to prevent them from overwriting.
The following example shows a simple localization testing of Firefox on Ubuntu in several different languages: English (default language), Spanish, French, German, Italian, Russian, Japanese and Simplified Chinese. The script is very simple and follows the rules stated earlier.  We will take advantage of Firefox's CLI option -UILocale which allows to start the browser in the language specified by a language identifier (which is in fact identical with the user preferred language sent withing the HTTP request header). Note that you must have the specified Firefox language packs installed to achieve this. The language identifier (e.g. en-us, es-es) is in the script defined by a variable called LANG and its value can be later on rewritten through the -v/--variable VNCRobot CLI option. The script further on performs a very basic browser testing - it opens the File menu, starts the Preferences and Help windows and displays the Browse bar at the bottom of the browser window. Three screenshots are taken along the way and a report is generated for each language.

example3.txt

Var LANG=en-us

# This procedure will open Firefox in the locale specified by argument.
# It will then open the browser menu, Preferences and Help windows and Search bar.
# Screenshots will be taken at major points.
# Parameter: {1} - desired browser locale, e.g. en-us, es-es, de-de
procedure test_firefox {
    # Close all windows and open the Run Application window on Gnome
    Press Alt+F4 count=2 wait=1s
    Press Alt+F2 wait=4s

    # Start Firefox in desired language
    Typeline "firefox -UILocale {1}"

    # Wait for major update of the remote screen
    Waitfor update extent="50%" timeout="30s" ontimeout="Exit 1"

    # Open the Firefox menu - keys F10 and "arrow down"
    Press F10 wait=200
    Press Down wait=1s
    Screenshot firefox-menu-{1}.jpg desc="Firefox File menu. It should be properly localized into {1}."

    # Open the Firefox Preferences window.
    # As the File menu is open, key "arrow right" will display the Edit menu
    # and key "arrow up" will jump to the last menu item Preferences
    Press Right wait=500
    Press Up wait=500
    Press Enter wait=5s
    Screenshot firefox-preferences-{1}.jpg desc="Firefox Preferences window. It should be properly localized into {1}."

    # Escape will dispose the Preferences window
    Press Esc wait=500

    # Display the Firefox Search bar (F3 key) and open the Help window (F1).
    Press F3 wait=1s
    Press F1 wait=5s
    Screenshot firefox-help-{1}.jpg desc="Firefox Help window and the Search bar (at the bottom). It should be properly localized into {1}."

    # Close the browser so that we can rerun the script safely
    Press Alt+F4 count=2 wait=1s
}

# Start an HTML report
Report index-{LANG}.html desc="Example of VNCRobot automated localization testing of Firefox. See the <a href=http://www.vncrobot.com/docs/examples/index.html#example3>How To Use VNCRobot</a> document for the list of all examples."

# Call the test procedure.
# Value of the argument variable can be passed from CLI.
test_firefox {LANG}


To execute the script on a local VNC server we need to pass the desired language identifier through CLI. The command will look like this:

vncrobot.sh -c localhost:1 -p <password> -r example3.txt -n -v LANG=<language_identifier>

To test Firefox in the eight languages discussed above we need to execute the command sequentially eight times, once per each language identifier. The following table lists reports from automated execution for all suggested languages.

Language Identifier
Description & Link To The Test Report
en-us
Firefox testing on English locale
es-es
Firefox testing on Spanish locale
fr-fr
Firefox testing on French locale
de-de
Firefox testing on German locale
it-it
Firefox testing on Italian locale
ru-ru
Firefox testing on Russian locale
ja-jp
Firefox testing on Japanese locale
zh-cn
Firefox testing on Simplified Chinese locale


Example #4: Availability Monitoring

Another smart idea is to use VNCRobot to monitor availability of services or applications and detect failures and server crashes. The principle is to design and run an automated test script which repeatedly accesses a service or an application in an infinite loop. If the service or application fails to display, VNCRobot can raise an alarm, e.g. send an E-mail to the administrator, or even restart the service.

Note that there are many tools allowing to test availability of some types of services or applications in a more effective way than VNCRobot.There's for example a bunch of good tools for web applications. VNCRobot testing is however easy to implement/maintain and it can test virtually anything capable of displaying on the remote desktop, regardless whether it is a browser, Java/C application, CLI command etc. The tool is especially good to "glue" testing of integrated heterogennous application stacks where you need a special tool for each application type otherwise.

As an example we will test availability of the Apache web server on a local machine. There's a page at http://localhost/vncrobot which represents our tested service. It is displayed in a Firefox browser on the following picture.

Test application

To monitor it's availability consistently we need to develop a test script which repeatedly loads the page in the browser and verifies that the it is displayed correctly. It must be robust enough to deal with a browser crash.

Tho verify both browser and test site behavior we will take advantage of two image templates. Both of them were cut from the remote desktop image using the VNCRobot's Compareto Command Window. You should be able to locate them easily on the picture linked above.

Image Name
Image Description
Image
firefox_icon.png
Part of the Firefox "Go" icon.
Firefox's "Go" icon
apache.png
Part of the "Powered by Apache" logo image.
Part of the Apache logo image

A sample code for a VNC server on Linux Gnome desktop follows:

# Send an E-mail to the administrator.
# Param {1}: E-mail subject.
procedure send_mail {
  Sendmail subject="{1}" from="root@yourmachine.com" to="
John.Doe@yourdomain.com"
    server=
"smtpserver.yourdomain"

}


# This procedure performs cleanup and starts Firefox

procedure start_firefox {

  # Close all windows and start Firefox
  Press Alt+F4 count=3 wait=3s
  Press Alt+F2 wait=4s
  Typeline "firefox http://localhost/vncrobot"

  # Wait max 15 seconds for Firefox to start.
  # Presence of the browser is verified by existence of the "Go" icon.
  Waitfor match template="firefox_icon.png" interval=3s method="search" timeout=15s

  # This will be executed when Firefox fails to start.
  # An E-mail is sent to administrator and the script exits.
  if ({_EXIT_CODE} > 0) {
    send_mail "Firefox failed to start! Please fix and restart the VNCRobot testing."
    Exit 1

  }
}

# Variable indicating the test result
Var FAILED=0

# Infinite loop
for (; 1==1;) {
  # Reload our test page and verify if it gets correctly displayed.
  # We are in fact looking for part of the "Powered by Apache" image.
  Press Ctrl+R wait=4s
  Compareto "apache.png" method="search"
 
  # This will be executed when the last command succeeds
  if ({_EXIT_CODE} == 0) {
    Var FAILED=0

  } else if ({FAILED} == 0) {
      # Verify if Firefox is there and restart it if not.
      Compareto "firefox_icon.png" method="search"

      if ({_EXIT_CODE} == 0) {
        # It is a first detected failure so send an E-mail and set the test result to 1.
        # It will prevent sending of multiple E-mails for each failure.
        send_mail "Your test application is not available! Please fix it."
        Var FAILED=1
      } else {
        # Restart Firefox
        start_firefox
      }
  }

  # Wait 5 seconds before trying again
  Wait 5s
}

The script code is be easy to understand. The start_firefox procedure starts the Firefox browser and opens the test URL. The procedure uses the Waitfor match command together with the 'search' image comparison method to verify whether the Firefox icon  is displayed on the screen.

The script body consists of an infinite loop. Each cycle starts with Ctrl+R which makes Firefox to reload the page. The script then looks for part of the Apache logo. If the image is not found, the script checks if Firefox is visible to rule out browser crash and restarts it in case of need. If the logo is not visible while Firefox is displayed correctly, we detected a crash of our service and a warning E-mail is sent to the administrator. A picture of VNCRobot executing the example script in GUI mode follows.

VNCRobot executing example script of availability testing

To test the script functionality, kill the browser or the tested service (Apache server). You should get an E-mail notifying you of the crash. It is of course possible to implement a service restart within the script. See the Exec command for more information.


Example #5: Load Testing

VNCRobot can be successfuly used to generate load for stress testing. The idea is to start many parallel testing processes to simulate a number of users accessing a service or using an application. As VNCRobot simulates a real user behavior in a graphical environment, such a scenario will  create load on the tested system which is very similar to real production environment.

VNCRobot supports has a Java API which allows to create multiple testing threads within a single Java Virtual Machine. Each thread may independently execute one test script on a single VNC server instance. See the VNCRobotThread interface of the VNCRobot 1.3 Open API for more information. Note that Unix/Linux systems are prefered to host the load tests because unlike Windows they are capable of  running many VNC server instances on a single machine.

Let's create an example program generating load on a tested application. We need to create three components:
  1. Test script in the VNCRobot scripting language which will represent a typical user session with the tested application.
  2. Java wrapper which will take advantage of the VNCRobot Open API to create and start parallel test threads.
  3. Shell wrapper which will start and eventually shut down the VNC servers (Linux/Unix only).
1. Test Script
There is no test script included in this example. You need to create it before and integrate it with the Java and shell wrappers described below. See the previous examples in this document on how to create a test script in the VNCRobot scripting language. You may easily use the Record&Replay feature to do so. It is highly recommended to use a small remote desktop size (e.g. 800x600 pixels) and avoid image comparison and taking of screenshots. All these factors significantly increase the amount of required memory and CPU time. Another good idea is to set your remote desktop background (wallpaper) to a solid color.

In the next steps we will suppose that the test script already exists and is saved in a file called client.txt.

2. Java Wrapper
Java wrapper is a program which creates and starts the automated testing threads using the VNCRobot 1.3 Open API. A universal wrapper code is displayed below.

LoadExample.java
import com.vncrobot.VNCRobot;
import com.vncrobot.api.VNCRobotThread;

/**
* LoadExample.java
*
* This is an example of load testing based on VNCRobot. For more information
* read the How To Use VNCRobot document at
* http://www.vncrobot.com/docs/examples/index.html#example5
*
* (C) 2007, Robert Pes
*/
public class LoadExample {

// Default number of threads.
final static int DEFAULT_THREAD_COUNT = 50;

// Main method.
public static void main(String[] argv) {

// Number of threads
int threadCount = DEFAULT_THREAD_COUNT;

// Password
String password = null;

// Script path
String script = null;

// Parse the program arguments. We expect an integer
// indicating how many threads we will start followed by script name.
// The last argument, VNC password, is not mandatory because some
// VNC servers can be configured not to require authentication.
if (argv.length > 1 && argv.length < 4) {
try {
threadCount = Integer.parseInt(argv[0]);
script = argv[1];
if (argv.length > 2) {
password = argv[2];
}
} catch (NumberFormatException ex) {
System.out.println("Invalid number of threads - default of "+DEFAULT_THREAD_COUNT+" will be used.");
}
} else {
System.out.println("Usage: LoadExample <number_of_threads> <script_path> [<VNC_password>]");
return;
}

// Thread arguments. They are identical with VNCRobot CLI options.
String[] threadArguments;
if (password == null) {
threadArguments = new String[] {
"--connect", // Connect option
"", // VNC host name and port - will be generated dynamically
"--run", // Run option
script, // Name and path of the script to run
"--nodisplay" // CLI execution mode option
};
} else {
threadArguments = new String[] {
"--connect", // Connect option
"", // VNC host name and port - will be generated dynamically
"--run", // Run option
script, // Name and path of the script to run
"--nodisplay", // CLI execution mode option
"--password", // Password option
password // Password
};
}

// Create an instance of VNCRobot
VNCRobot robot = new VNCRobot();

VNCRobotThread runnable;
Thread thread;

// Loop for index from 1 to specified number of threads
for (int i=1; i<=threadCount; i++) {

// Populate the VNC host name and port in the thread arguments
threadArguments[1] = "localhost:" + i;

// Create a new automated thread and start it
runnable = robot.createVNCRobotThread(threadArguments[1], threadArguments, System.out, false);
thread = new Thread(runnable);
thread.start();
}
}
}
To run the wrapper compile the Java code to .class or take advantage of the precompiled class in the attached ZIP file. The program accepts up to three parameters - desired number of threads to be started (mandatory), path to the testing script (mandatory) and VNC password (optional). The command should look like:

java -classpath .:./vncrobot.jar LoadExample <thread_number> <script_path> [<password>]


3. Shell Wrapper
Before you start the VNCRobot load testing, you always need to start all the necessary VNC servers. You should also verify whether any application involved in the testing allows only one instance per user. A typical example is Firefox which will not allow a single user to run more than one browser. To resolve this you will need to create a different user for each test thread and start the VNC server under his account.

All these necessary overhead tasks can be efficiently automated on Linux/Unix by simple shell scripts. Let's suppose that our example application doesn't have any instance limitations. All we need to do then is to start the VNC servers, execute the Java load application and kill the VNC servers after the testing gets finished. A Bash script handling these tasks follows.

runLoad.sh
#!/bin/bash

# Number of threads to be executed
THREADCOUNT=10

# Path to the automated script
SCRIPT=client.txt

# VNC password. If your VNC server doesn't require password,
# leave the variable empty.
PASSWORD=welcome

# This 'for' statement will start VNC servers on local machine
# on ports 1 to THREADCOUNT
for ((i=1; i<=$THREADCOUNT; i++))
do
  vncserver :$i
done

# The following command will execute the Java wrapper with VNCRobot
# automated threads
java -Xmx512m -classpath .:./vncrobot.jar LoadExample $THREADCOUNT $SCRIPT $PASSWORD

# This 'for' statement will kill all the VNC servers started
# at the beginning of this script
for ((i=1; i<=$THREADCOUNT; i++))
do
  vncserver -kill :$i
done


The shell wrapper source together with the Java wrapper are available for download. Before you execute it, set correct values of the THREADCOUNT, SCRIPT and PASSWORD variables.

If you define the number of threads too high, some of them may start to fail with java.lang.OutOfMemoryError. To fix this issue increase the JVM memory heap size specified by the -Xmx Java option in runLoad.sh (default value is set to 512MB).

     
 
© Copyright 2007 Robert Pes.