A Guide to Codeless Automation

A Guide to Codeless Automation

Salesforce Testing Tutorial for Beginners

Robotium Guide for Beginners

Robotium For Beginners

Test Android Sample App

Reasons Windows Phone Apps are Rejected

Incorrect app screenshots

Requirement 4.6 - App screenshots. Your app screenshots should not include any emulator chrome, frame rate counters, or debug information. They should not be altered or be transparent. Also, photos of your app running on a device or the emulator are not substitutes for a screenshot taken with the emulator.

• Tip: Use the built-in emulator screenshot tool to take the screenshots. The Windows Phone 8 SDK ships with emulators for WVGA, XVGA, and 720p resolutions.
• Tip: Don’t take WVGA (480 x 800) screenshots for Windows Phone 8 XAPs. Instead, use the Dev Center feature that automatically scales the XVGA screenshots down to the WVGA resolution.
• Tip: Screenshots are not required to be localized, however, you do need to provide at least one screenshot for each language supported.

Here are a couple of examples of bad screenshots:

Bad screenshot 1 – Note emulator chrome	Bad screenshot 2 - Note frame rate counter in upper right

And here are the correct screenshots:

Good screenshot 1 - No emulator chrome	Good screenshot 2 - No debug information

Missing information to test the app

Requirement 5.1.4 - App Testability. Many apps require login credentials to run. If your app requires an existing account, make sure you create a test account that can be used by the certification team during testing. Don’t forget to include the account credentials in the Certification notes in your Dev Center submission. Click More XAP Options on the Upload and describe your XAP page.

Include test account credentials in Certification notes field.

App crashes

Requirement 5.1.2 - App closure. This requirement is simply to verify that your app doesn’t crash during certification testing. As you know, if your app crashes in release mode, it just goes away without any user prompt. Certification will reject your app if it “unexpectedly terminates” during testing.

BugSense and Little Watson can help you collect unhandled exception telemetry data. See this KB article for more info on how to avoid crashes: Troubleshooting Windows Phone App Problems that occur after Submitting.

Incorrect icons and tile images

Requirement 4.7 – Required app images. Developers sometimes forget to replace default icons and tile images in an app created from a Visual Studio template. Other tools such as App Studio and Apache Cordova provide default images that are to be replaced with unique icons and tiles that reflect your app. The default images are useful in letting you know the required size for these assets.

Default tile provided by App Studio - replace before submitting

Default tile provided by Visual Studio – replace before submitting

• Tip: Visual Studio creates a default large tile when using the Project New template -FlipCycleTileLarge.png. This tile (691 x 336) is required only if your app supports large tiles. By default, this option is disabled in the WMAppManifest.xml settings. If your app doesn’t support large tiles, you can delete the tile from the project. This will save space and installation time for this unused asset.

Incorrect use of the Back button

Requirement 5.2.4 - Use of Back button. The Back button behavior is narrowly defined. The Back button should:

• Close the app only if the app is on the main page
• Go to the previous page only if not on the main page
• Close an open dialog
• Close the soft keyboard

There are two exceptions to this:

1) If the app is a game, and if during game play the Back button is pressed, game play can be paused and present a pause prompt. Pressing the Back button again should dismiss the pause prompt and restart game play.

2) If you need to confirm with the user that they really intend to close the app when the Back button is pressed, you can display a confirmation prompt (for example, “Are you sure you want to quit?”). This should only be done when the Back button is pressed and the user is on the main page. An affirmative response should exit the app. A negative response should return the user to the main page. You can do this by overriding OnBackKeyPressed.

• Tip: If you are writing a C++ app or game, see the Marble Maze sample for sample code on implementing the Back button behavior in a native app or game.

Insufficient localization information

Requirement 5.5.1 – Language validation. For each of the languages your app supports, you must provide a localized app description. This is the description entered as part of your Dev Center submission. Certification testing also verifies that your app displays properly for each of the languages your app supports.

• Tip: Use the emulator to quickly switch between phone languages. To do this, press Settings, and then press language + regions. Then change the language setting to one that your app supports. Don’t forget to tap restart phone – otherwise the language change will not persist. When the emulator reboots, deploy, start your app and verify the correct language is displayed. Note that the emulator supports all of the display languages supported by Windows Phone, whereas your development phone likely doesn’t.

Lack of support for both light and dark phone themes

Requirement 5.5.2 – Content and themes. This requirement ensures your app displays properly in both light and dark themes. You can switch between themes on your phone. To do this, press Settings, and then press theme.

• Tip: Use Blend and Visual Studio during layout to easily switch between themes.

Interview Questions

Testing Concepts

1. What is the main difference between Verification and Validation?
Verification is related to static testing, where as Validation covers dynamic testing.


2. Describe the difference between the Black box and White Box categories of testing.
Black Box testing is conducted with no knowledge of the of the application’s code or internal structure. White Box is conducted with knowledge of the code or internal structure.

3. Is it possible to use both Black Box and White Box Testing with Verification?
Yes, as documentation reviews are static inspection or ‘desk checks’ we might consider these Black Box and as Walkthroughs are done with the author these might be thought of as White Box. 


4. Name some types of testing that belong under the Functional testing group.
Types of Acceptance testing such as Alpha, Beta, UAT and OAT would fit under Functional Testing - though we’d expect some cross-over into Non-Functional also. It’s not common to assign Test Types as such as the focus is Features and Behaviours.


5. Name some types of testing that belong under the Non-Functional testing group.
Any Test Types that are related to 'emergent' characteristics and attributes of the application under test such as Accessibility, Localisation and Performance.

6. Summarise how Performance, Load and Stress testing types relate to each other.
Load and Stress testing are testing sub-types of Performance testing. Load testing focuses on normal operational profiles of the application, whereas Stress focuses on load outside of the expected operational profile.


7. When and against what might we use Static Black Box testing?
When performing an Inspection of Documentation, though ISTQB states this is White Box testing that would more accurately apply to Peer Reviews and Walkthroughs.


8. Are all forms of Validation testing dynamic? Explain why they are or aren’t.
Yes, because they involve running the code and when the code isn’t being run we’re doing Static Verification.

9. How does Regression Testing fit across the overall testing domain?
Regression Testing is not a specific Test Type but an Approach that covers both manual and automated execution and draws together Test Cases for many Test Types. 


10. Name the main ‘levels’ of testing as recognised by ISTQB, list any others you recognise.
There are four levels which are; Component, Integration, System and Acceptance.

Pros and Cons of iOS and Android Development & Testing

iOS and Android are very clearly the dominate mobile platforms at the moment. Many companies develop apps for both, but some companies still release apps on one platform first then expand the successful apps. But how do companies decide which OS to tackel first?

iOS Pros

‘There are mature APIs (application programming interface) in place which are well designed and debugged, and actually date back to NeXT system in the early ‘90s. These are complemented by good documentation, a decent dev community and lots of good quality open source code.’

‘There’s also a comprehensive tool chain which is easy to get started with, and a relatively quick code-compile-run turnaround when using the Simulator, all of which makes development faster.’

iOS Cons

The App Store review process is unpredictable and inconsistent.

Getting apps onto test devices can also be complicated at first, but third party tools are starting to make this simpler. Also, developer accounts are limited to 99 test devices a year, which can become a problem when you release multiple apps under one account.

Android Pros

There are definitely more inter-app integration possibilities. ‘

You have fewer restrictions on what your app is allowed to do, but it’s a double-edged sword, as this can open the door to malware.

With Android it’s much easier to get your app on devices and far easier to publish app to Google Play store.

Android Cons

The sheer number of Android devices out there in the field makes testing on all of them virtually impossible,’ laments Venn.

‘Dev tools are clunky and painfully slow to use – sometimes it can actually be quicker to deploy your app to a device rather than to the Android dev emulator, which is ironic when you consider that it’s there to make testing easier.’

To cap it all off, uptake of new OS versions has been very slow so far. This leads to frustrating incompatibility issues across different devices.

iOS and Android UX Design Similarities and Differences

The same app can be available for iOS and Android, but making sure the app looks, feels and functions correctly on both major platforms is a huge development and testing challenge. Android is famously more open in terms of its app standards while iOS has a specific set of guidelines that must be followed. This makes creating the same app for both platforms a bit challenging. To help you get your head around the issue, The Archer Group has put together a list of  iOS and Android app design similarities and differences. It’s changed a bit as new devices (specifically the iPhone 5) have hit the market, but they are still good points to know.

UX Similarities

·        Application structure: The basic flow of information can be similar in both iOS and Android platforms. It’s rarely necessary to define      completely unique information architecture for each platform.

·        Expected functionality of basic UI components: Many UI components, including tabs, sliders, pickers, text fields, checkboxes, switches, and buttons, are very similar across both platforms. The design treatment and placement of the UI components varies between platforms, but their expected functionality is quite similar.

·        Gestures: The most basic and common touch gesture controls used in applications are similar in both iOS and Android. The tap, drag, flick, swipe, double tap, and pinch gestures are typically used for similar actions across both platforms. The only gesture that has significantly different usage across platforms is the “tap and hold” gesture, which is much more commonly used in the Android platform to reveal a contextual menu of options or to enter a data selection mode.

UX Differences

·        “Back” navigation: “Back” is a UI element in iOS applications, placed in the upper-left hand corner of the navigation bar that navigates backward only within defined screens in an application, never across the entire device. In Android devices, there can be two different “back” actions: “up” and “back”. “Up” was introduced primarily for Android 3.0+ devices without hardware keys and is a UI element represented as an icon on the left-hand side of the top action bar. “Up” navigates back within an application. Android “back”, in contrast, is a button on the physical device that goes back in history across the entire device.

·        Tab navigation placement: Tab navigation is typically used to navigate through primary functions in an application. iOS tab navigation is represented through a tab bar at the bottom of the application. Android recommends that tabs be placed at the top of an application. In addition, by iOS standards, only 5 tabs can be displayed at a time. In Android, however, scrollable tabs can be used to display more tabs than can fit in the viewable screen width.

·        Search: In iOS, the standard UI control for searching within an application is a search bar that is placed at the top of a searchable screen. In Android, several different search options are available. A “search dialog” component can be used that is similar to the iOS approach and places a search bar at the top of the screen. However, this bar is hidden until the user presses a search button within the user      interface. An alternative search approach in Android 3.0+ is to use a “search widget” that allows search to be placed anywhere within the application interface, typically within the application’s action bar at the top of the screen.