Automated Testing and BDD on BlackBerry 10

Posted by Samuel Truscott on

Testing at Truphone

At Labs we like to make stuff that works and BlackBerry is no exception.

To make sure the app is always working after all the improvements we add, we wanted to automate all our testing so that any bugs are found and fixed as early as possible so you don’t see them. We aim to test all the time; after every change and before every release. This means that when it lands on your device it just works and you get a seemless user experience on your mobile.

BB10 Automated Test Overview

The framework that we’ve started to work on lets us automate the User Interface testing on devices and simulators over the network and we can write scripts to reproduce those tests. We based the idea on Hooq, an old open source project that let you record and replay your actions on the UI, but we wanted to do it with BlackBerry! We’ve also added a Java layer so we can plug it into our Behaviour Driven Development testing environment.

Under the hood the framework has to cheat. Until BlackBerry release Automatic Input Control we have to dig right down into the Cascades API to get access to what’s going on.

How does it work?

So here’s how a basic record and reply works:

  • 1/ You ask the framework to record.
  • 2/The framework iterates over all the Cascades objects and installs hooks that let us know when things change.
  • 3/ When they change, we write it out over the network to record it.

Reply is similar, but backwards:

  • 1/ You (via the command-line program) stream the record file back to the framework.
  • 2/ The framework re-produces what happened.

Behaviour Driven Testing

If you’re into your Test Driven Development (TDD) or Behaviour Driven Development (BDD) then you can use a Java API to execute the commands (with timeouts specified). It lets us do neat things like this:

final int defaultTimeoutPeriod = 1000;
final int defaultSleepPeriod = 3000;
final SynchronousConnection conn 
      = new SynchronousConnection("192.168.70.131", 15000);
try {
    conn.connect(defaultTimeoutPeriod);

    IReply reply;

    reply = conn.transmit(
         new TextCommand("usernameField", "sam.truscott"),
         defaultTimeoutPeriod);
    System.out.println(reply.isSuccess() + ": " + reply.getMessage());

    reply = conn.transmit(
        new TextCommand("passwordField", "MyReallySecretPassword"),
        defaultTimeoutPeriod);
    System.out.println(reply.isSuccess() + ": " + reply.getMessage());

    reply = conn.transmit(
         new ClickCommand("loginButton"),
         defaultTimeoutPeriod);
    System.out.println(reply.isSuccess() + ": " + reply.getMessage());

    reply = conn.waitForCommand(
         new ToastCommand("Nice try"),
         defaultTimeoutPeriod);
    System.out.println(reply.isSuccess() + ": " + reply.getMessage());

    reply = conn.transmit(
         new SleepCommand(defaultSleepPeriod),
         defaultTimeoutPeriod);
    System.out.println(reply.isSuccess() + ": " + reply.getMessage());

    // make sure the toast has disappeared
    reply = conn.waitForCommand(
         new ToastCommand(false),
         defaultTimeoutPeriod);
    System.out.println(reply.isSuccess() + ": " + reply.getMessage());
} catch (TimeoutException toe_) {
    toe_.printStackTrace();
} finally {
    conn.close();
}

The trick here is the objectName in the QML (i.e. usernameField). If you have code you can’t change then you can use a full path to the QML object (this is something that Hooq did) which may look like Application::Pane::Window0:_:textField1.

Command Line Example

Of course if you’re a bit more geeky you can just have a play on the command line via tools like netcat and connect directly to the framework. This can be quite useful for the Signal Spy tools if you’re checking how many times signals have fired or if you want to use the ‘test’ command to query the Qt Properties of an object such as ‘visible’, ‘enable’ or ‘open’.

Command line dialing

Command line proof

So that’s me making a call, over the command line, on a Q5 - without even touching it.

It’s no silver bullet (yet) and it isn’t complete but we still think it might help others! In the future we’d like to integrate BlackBerry’s new Automatic Input Control as that may help to replace some of the voodoo-we-do to mimic stuff being used.

More information

Give it a go, have a read, tell us what you think! Fork it at https://github.com/trulabs/labs-truphone-cascades-test.

Need help getting going? Once you’ve got the library attached I highly suggest connecting via netcat and using the ‘help’ command. It’ll give you the syntax of all the commands. You can also use the ‘record’ command and see it spew what’s going on. If you want to get started with the Java code check out the Unit tests!

blog comments powered by Disqus