Acceptance testing Framework: concordion

Acceptance testing Framework: concordion

“Next gen BDD”

Concordion lets you write tests in normal language use in paragraphs , tables and proper punctuations.

Tests are written in html and then instrumented with special attributes that concordion interprets to execute the test

Intro:

Concordion is a Java library that allows to bind an HTML document to methods in a Fixture - a Java class that does various binding work. When you execute that specification, it generates an HTML output of what has been tested, with errors in red and ‘test ok’ in green.

Sample Project:

https://github.com/kasthurikranthikumar/concordion.git

Selenium to Selenide Transition

1. Create a browser

Selenium WebDriver:

DesiredCapabilities desiredCapabilities = DesiredCapabilities.htmlUnit();

desiredCapabilities.setCapability(HtmlUnitDriver.INVALIDSELECTIONERROR, true);

desiredCapabilities.setCapability(HtmlUnitDriver.INVALIDXPATHERROR, false);

desiredCapabilities.setJavascriptEnabled(true);

WebDriver driver = new HtmlUnitDriver(desiredCapabilities);

Selenide:

open("/my-application/login");

// And run tests with option -Dbrowser=htmlunit (or "chrome" or "ie", default value is "firefox")

2. Shutdown a browser

Selenium WebDriver:

if (driver != null) {

    driver.close();

}

Selenide:

// Do not care! Selenide closes the browser automatically.

3. Find element by id

Selenium WebDriver:

WebElement customer = driver.findElement(By.id("customerContainer"));

Selenide:

WebElement customer = $("#customerContainer");

or a longer conservative option:

WebElement customer = $(By.id("customerContainer"));

4. Assert that element has a correct text

Selenium WebDriver:

assertEquals("Customer profile", driver.findElement(By.id("customerContainer")).getText());

Selenide:

$("#customerContainer").shouldHave(text("Customer profile"));

5. Ajax support (waiting for some event to happen)

Selenium WebDriver (OMG!):

FluentWait<By> fluentWait = new FluentWait<By>(By.tagName("TEXTAREA"));

fluentWait.pollingEvery(100, TimeUnit.MILLISECONDS);

fluentWait.withTimeout(1000, TimeUnit.MILLISECONDS);

fluentWait.until(new Predicate<By>() {

    public boolean apply(By by) {

        try {

            return browser.findElement(by).isDisplayed();

        } catch (NoSuchElementException ex) {

            return false;

        }

    }

});

assertEquals("John", browser.findElement(By.tagName("TEXTAREA")).getAttribute("value"));

Selenide:

$("TEXTAREA").shouldHave(value("John"));

This command automatically waits until element gets visible AND gets expected value.

Default timeout is 4 seconds and it's configurable.

6. Assert that element has a correct CSS class

Selenium WebDriver:

assertTrue(driver.findElement(By.id("customerContainer")).getAttribute("class").indexOf("errorField") > -1);

Selenide:

$("#customerContainer").shouldHave(cssClass("errorField"));

7. Find element by text

Selenium WebDriver:

No way (except XPath)

Selenide:

WebElement customer = $(byText("Customer profile"));

8. Assert that element text matches a regular expression

Selenium WebDriver:

WebElement element = driver.findElement(By.id("customerContainer"));

assertTrue(Pattern.compile(".*profile.*", DOTALL).matcher(element.getText()).matches());

Selenide:

$("#customerContainer").should(matchText("profile"));

9. Assert that element does not exist

Selenium WebDriver:

try {

    WebElement element = driver.findElement(By.id("customerContainer"));

    fail("Element should not exist: " + element);

}

catch (WebDriverException itsOk) {}

Selenide:

$("#customerContainer").shouldNot(exist);

10. Looking for element inside parent element

Selenium WebDriver:

WebElement parent = driver.findElement(By.id("customerContainer"));

WebElement element = parent.findElement(By.className("user_name"));

Selenide:

$("#customerContainer").find(".user_name");

11. Looking for Nth element

Selenium WebDriver:

WebElement element = driver.findElements(By.tagName("li")).get(5);

Selenide:

$("li", 5);

12. Click "Ok" in alert dialog

Selenium WebDriver:

    try {

      Alert alert = checkAlertMessage(expectedConfirmationText);

      alert.accept();

    } catch (UnsupportedOperationException alertIsNotSupportedInHtmlUnit) {

      return;

    }

    Thread.sleep(200); // sometimes it will fail

Selenide:

    confirm("Are you sure to delete your profile?");

or

    dismiss("Are you sure to delete your profile?");

13. Debugging info for elements

Selenium WebDriver:

    WebElement element = driver.findElement(By.id("customerContainer"));

    System.out.println("tag: " + element.getTag());

    System.out.println("text: " + element.getText());

    System.out.println("id: " + element.getAttribute("id"));

    System.out.println("name: " + element.getAttribute("name"));

    System.out.println("class: " + element.getAttribute("class"));

    System.out.println("value: " + element.getAttribute("value"));

    System.out.println("visible: " + element.isDisplayed());

    // etc.

Selenide:

    System.out.println($("#customerContainer"));

    // output looks like this: "<option value=livemail.ru checked=true selected:true>@livemail.ru</option>"

14. Take a screenshot

Selenium WebDriver:

    if (driver instanceof TakesScreenshot) {

      File scrFile = ((TakesScreenshot) webdriver).getScreenshotAs(OutputType.FILE);

      File targetFile = new File("c:\temp\" + fileName + ".png");

      FileUtils.copyFile(scrFile, targetFile);

    }

Selenide:

    takeScreenShot("my-test-case");

For JUnit users it's even more simpler:

    public class MyTest {

      @Rule // automatically takes screenshot of every failed test

      public ScreenShooter makeScreenshotOnFailure = ScreenShooter.failedTests();    }

15. Select a radio button

Selenium WebDriver:

    for (WebElement radio : driver.findElements(By.name("sex"))) {

      if ("woman".equals(radio.getAttribute("value"))) {

        radio.click();

      }

    }

    throw new NoSuchElementException("'sex' radio field has no value 'woman'");

Selenide:

    selectRadio(By.name("sex"), "woman");

16. Reload current page

Selenium WebDriver:

    webdriver.navigate().to(webdriver.getCurrentUrl());

Selenide:

    refresh();

17. Get the current page URL, title or source

Selenium WebDriver:

    webdriver.getCurrentUrl();

    webdriver.getTitle();

    webdriver.getPageSource();

Selenide:

    url();

    title();

    source();

My first program with Selenide:

Create a Maven project.

Add selenium-java client dependencies in pom.xml

Add TestNG or JUnit dependencies in pom.xml

And finally Selenide dependencies

<dependency>

    <groupId>com.codeborne</groupId>

    <artifactId>selenide</artifactId>

    <version>3.7</version>

    <scope>test</scope>

</dependency>

Add a simple TestNG/ Junit class

import static com.codeborne.selenide.Selenide.*;

import static com.codeborne.selenide.Condition.*;

public class Test

{

@Test

public void userCanLoginByUsername() {

  open("/login");

  $(By.name("user.name")).setValue("johny");

  $("#submit").click();

  $(".loading_progress").should(disappear); 

  $("#username").shouldHave(text("Hello, Johny!")); 

}

}

Just Run it.

What is Selenide?

Wrapper around Selenium Webdriver for below reasons.

Open Source, Stable and Concise API

Concise API: Very less code, we don’t need to write code to start and stop web driver.

Ajax based locator support by using $, $$

Smart Waits using ’shouldBe(visible)’, ‘ShouldHave’, ShouldNotHave

Multi file upload and Downloading a file is very easy.

Finds parent, child, sibling etc. elements very easily.

Selenide profiler helps to find execution time for each step.

Execution is very fast.

Many more features.

You can still use Webdriver directly.

Solved below issues:

Ajax issues, Time outs, Stale Element Exceptions and easy Screenshots

Supports all other features:

Parallel execution

POM design pattern

Use with Junit, TestNg, jBehave, ScalaTest

http://selenide.org/

here is sample code : https://github.com/kasthurikranthikumar/selenide.git

Selenium Remote WebDriver + Sauce Labs

Sauce Labs is a Monster Grid to run our tests remotely.

The Sauce Labs cloud-based testing platform helps you run your SeleniumWebdriver tests on the cloud, test web and mobile apps immediately on 350+ browsers and platforms, and easily debug your code with the help of videos and screenshots.

Lest quickly setup the things.

Step 1: Create a sauce labs account: https://saucelabs.com/home

Choose Free Trail, fill the info and then click on Create Account button.

Then verify your account by clicking link in the Email.

Now, you can see Dashboard screen.

Observe, Bottom left corner where you can see your account Firstname and Last name.

Click on it,

We need UserName and AccessKey to connect to sauce labs from our selenium code.

Step 2: Write Remote WebDriver code:

Here is the sample code.

package com.sauceLabs.automation;

import org.openqa.selenium.By;

import org.openqa.selenium.Platform;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.WebElement;

import org.openqa.selenium.remote.DesiredCapabilities;

import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.URL;

public class Test {

       public static final String USERNAME = "kXXXXXX";

       public static final String AUTOMATE_KEY = "d87XXXX-XXX-XXX-XXX-XXX";

       public static final String URL = "https://" + USERNAME + ":" + AUTOMATE_KEY + "@ondemand.saucelabs.com:80/wd/hub";

       public static void main(String[] args) throws Exception {

              DesiredCapabilities caps = new DesiredCapabilities();

              caps.setCapability("browser", "IE");

              caps.setCapability("browser_version", "7.0");

              caps.setCapability("os", "Windows");

              caps.setCapability("os_version", "XP");

              caps.setCapability("browserstack.debug", "true");

              WebDriver driver = new RemoteWebDriver(new URL(URL), caps);

              driver.get("http://www.google.com");

              WebElement element = driver.findElement(By.name("q"));

              element.sendKeys("BrowserStack");

              element.submit();

              System.out.println(driver.getTitle());

              driver.quit();

       }

}

Just run it, That’s all.

Parallel Testing thumb rules:

Parallel Testing thumb rules:

Each Test must be atomic, must have no dependency.

Implement Grouping mechanism to run in small batches.

Write meaningful comments for each test.

Choose a reliable GRID environment.

XPath vs. CSS Selectors

XPath vs. CSS Selectors

IE tests runs 2-3 times slower than the same tests on Firefox.

Reason is that, Internet Explorer has a very slow XPath engine.

To fix this issue, it's recommended that ids or CSS Selectors be used whenever trying to locate elements.

For more information on CSS Selectors and Selenium:

https://saucelabs.com/resources/selenium/css-selectors

To test CSS Selectors:

https://addons.mozilla.org/en-US/firefox/addon/firefinder-for-firebug/

Will Add more details soon.

WebDriver Architecture

All implementations of WebDriver that communicate with the browser, or a RemoteWebDriver server use a common wire protocol. This wire protocol defines a RESTful web service using JSON over HTTP.

So each WebDriver command is mapped to an HTTP method via the WebDriver service, and then passed on to the HTTP Command Processor to communicate with the browser. The Command responses are returned as HTTP/1.1 response messages via the WebDriver service.

Different drivers, such as the Firefox Driver and the IE Driver, have different implementations to accomplish the above.

The Selenium WebDriver architecture document linked below goes into further details on how these are implemented and how WebDrvier commands flow through to the browser and back. Read section 16.6 for details on the Firefox Driver.

Execution Flow:

Step 1. Automation Script is written using Client binding for different languages

Step 2. Once Test execution is started

Step 3. Browser driver is created

Step 3. Respective Driver.exe file starts “HTTP server” and starts listening for HTTP requests.

Step 4. A new HTTP request Creates, as each Selenium Method contains is mapped to respected webDriver API

Step 5. HTTP request is sent to the HTTP server

Step 6. HTTP server determines the steps needed for implementing the Selenium command

Step 7. The implementation steps are sent to the browser

Step 8. The HTTP server send the execution status back to the test script

Reason for not having FireFoxDriver.exe

Firefox driver is included in the selenium-server-stanalone.jar available in the downloads. The driver comes in the form of an xpi (firefox extension) which is added to the firefox profile when you start a new instance of FirefoxDriver

Selenium API

http://seleniumhq.github.io/selenium/docs/api/java/

Selenium RC architecture

Intro: Introduced in 2004 and lived until 2006 when WD born.

Selenium Remote Control is great for testing complex AJAX-based web user interfaces under a Continuous Integration system. It is also an ideal solution for users of Selenium IDE who want to write tests in a more expressive programming language than the Selenese HTML table format.

Main components:

-          Client libraries: which provide the interface between each programming language and the Selenium RC Server.

A Selenium client library provides a programming interface (API), i.e., a set of functions, which run Selenium commands from your own program.

The client library takes a Selenese command and passes it to the Selenium Server for processing a specific action or test against the application under test (AUT).

The client library also receives the result of that command and passes it back to your program

-          Selenium Server:  Drives the browser, Embeds Selenium Core and Injects into browser, Interprets commands,  sends back results to the client

which launches and kills browsers, interprets and runs the Selenese commands passed from the test program, and acts as an HTTP proxy, intercepting and verifying HTTP messages passed between the browser and the AUT.

The RC server bundles Selenium Core and automatically injects it into the browser. This occurs when your test program opens the browser (using a client library API function). Selenium-Core is a JavaScript program, actually a set of JavaScript functions which interprets and executes Selenese commands using the browser’s built-in JavaScript interpreter.

Architectural diagram: (Simplified)

Step 1: Client libraries communicate with the Server passing each Selenium command for execution.

Step 2: Then the server Launches browser and Injects ‘selenium core’ into browser from Server.

Step 3: Then Selenium + java code will be sent to RC server line by line and Interprets the commands.

Step 4: Corresponding javascript method will be executed

Step 5: Each request goes via HTTP proxy to overcome same-origin policy problem

Step 6: Corresponding application / web server handles and request and sends the response back

Step 7: response will be sent back to client library.

Advantages of RC:

      1. Supports multiple languages like Java, Ruby, C#, Perl, Python etc.

      2.    Core Written In JavaScript

      3.    Supports all browsers

Disadvantages:

1.       The same origin policy. It is a Javascript security policy that allows running the code only from the domain you're on. And since RC is fully written in Javascript, you can't easily switch between domains or work with some websites that redirect or use frames with content from many domains.

2.       Because of another security policy in Javascript, you can't fill in <input type='file' /> inputs and have to use several workarounds.

3.       You have to write your own methods when you need to wait for an element.

4.       Execution is Slow, duet to multiple layers of interactions

5.       Fails to mimic real life interaction ( Non Native Events)

6.       Cannot maximize the browser really

Tip:

Does really RC server holds Selenium Core init?

See here

How did Selenium RC handle same origin policy problem?

How did Selenium RC handle same origin policy problem?

Problem:

“Same Origin Policy or Cross-site scripting” is a browser security policy.

Same Origin policy prohibits JavaScript code from accessing elements from a domain that is different from where it was launched.

Example 1: The HTML code in www.google.com uses a JavaScript program "testScript.js". The same origin policy will only allow testScript.js to access pages within google.com such as google.com/mail, google.com/login, or google.com/signup. However, it cannot access pages from different sites such as yahoo.com/search or fbk.com because they belong to different domains.

Example 2: The browser is getting a script from Selenium which tells it that it wants to fetch resources from http://google.com. But the browser got this script from http://localhost:4444/selenium (for example). The browser says "hey this script came from “localhost” and now it's requesting a resource from some outside website. This violated the same-origin-policy.

Solution:

 So, In order to handle same origin policy, Selenium Remote Control was introduced.

In Detail:
So, how did Selenium RC handle this? To overcome this security restriction,
Selenium RC acts as an HTTP Proxy Server. When the test script asks to launch
a browser, Selenium RC server launches the browser and injects its JavaScript
(Selenium Core) into the browser. All the subsequent requests for the AUT go
through Selenium RC (acting as an HTTP Proxy Server) to the actual web server
hosting WAUT. Thus making the browser think that the web application is being
served from the Selenium RC’s server domain than the actual web server’s domain
and allowing Selenium Core to execute and drive the web application.

Understanding Selenium IDE & Architecture

Selenium IDE, is an extension for Firefox that allows users to record and playback tests.

A table-based syntax was placed over the raw Javascript and this allowed tests to be written by people with limited programming experience using a keyword-driven approach in HTML files.  

Command        –               Element             –            Text

type                                 name=q                       Selenium WebDriver

Selenium IDE is developed using Javascript, as all the browsers being tested supported Javascript. 

Do you want to see? How Selenium IDE code looks like?

      a. Download latest selenium IDE add-on file from here. (Ex: http://release.seleniumhq.org/selenium-ide/2.9.0/ )  selenium-ide-2.9.0.xpi  749 KB

      b.   Extract xpi file using a software, which you can download here. http://b1.org/

      c.    You would notice multiple xpi files under selenium-ide-2.9.0.xpi file, but we are mainly looking at selenium-ide.xpi file

      d.    Extract selenium-ide.xpi file

      e.     Browse below folder once you extract xpi file C:\Users\USER_ACCOUNT\Downloads\selenium-ide-     2.9.0\selenium-ide\chrome\content\selenium-core\scripts

      f.     All Java script files!! This is the selenium Core.

Selenium Core is the heart of the original Selenium implementation, and is a set of Javascript scripts that control the browser. This is sometimes referred to as "Selenium" and sometimes as "Core".

How selenium IDE works?

Test Runner will do the job.

https://github.com/SeleniumHQ/selenium/blob/master/javascript/selenium-core/TestRunner.html

or

C:\Users\ USER_ACCOUNT \Downloads\selenium-ide-2.9.0\selenium-ide\chrome\content\selenium-core/ TestRunner.html

Below are the the referenced javascript files to execute our scripts.

Lib/prototype.js, lib/sizzle.js, lib/snapsie.js, scripts/atoms.js, scripts/htmlutils.js, scripts/ui-element.js, lib/scriptaculous/scriptaculous.js, scripts/selenium-browserdetect.js, scripts/selenium-browserbot.js, scripts/find_matching_child.js, scripts/selenium-api.js, scripts/selenium-commandhandlers.js, scripts/selenium-executionloop.js, scripts/selenium-testrunner.js, scripts/selenium-logging.js, scripts/selenium-version.js, xpath/util.js, xpath/xmltoken.js, xpath/dom.js, xpath/xpath.js, xpath/javascript-xpath-0.1.12.js, scripts/user-extensions.js

Above all javascript libraries are heart of selenium IDE, which are very important to run selenium commands.

So, Do you want to add your own / custom functionality? Here is the way.

Because Selenium's core is developed in JavaScript, creating an extension follows the standard rules for prototypal languages. To create an extension, we create a function in the following design pattern.

Ex: C:\Users\ USER_ACCOUNT \Downloads\selenium-ide-2.9.0\selenium-ide\chrome\content\selenium-core/script/user-extensions.js.sample

Selenium.prototype.assertValueRepeated = function(locator, text) {

    // All locator-strategies are automatically handled by "findElement"

    var element = this.page().findElement(locator);

    // Create the text to verify

    var expectedValue = text + text;

    // Get the actual element value

    var actualValue = element.value;

    // Make sure the actual value matches the expected

    Assert.matches(expectedValue, actualValue);

};

Save the above code as 'user-extensions.js"

By default, Selenium looks for a file called "user-extensions.js", and loads and javascript  code found in that file.  user-extensions.js provides a convenient location for adding extensions to Selenium, like  new actions, checks and locator-strategies.

By default, this file does not exist. Users can create this file and place their extension code  in this common location.

How to use newly created core extension?

1.     Go to Selenium IDE - Click on Options -> Options…

2.     In General section select the location of the newly created Selenium Core Extension

3.     Click OK and restart Selenium IDE

You will find the extension in the command list

Architecture:

Advantages:

• Selenium IDE is very easy to use.
• It has the capability to convert the test to different programming languages such as html, java etc
• Programming language experience is not required for Selenium IDE
• Selenium IDE provides Logging capabilities using file login plug-in.
• In Selenium IDE, user can debug and set breakpoints
• Selenium IDE is flexible for the users.
• Auto complete for all common Selenium commands
• Debug and set breakpoints
• Option to automatically assert the title of every page
• Support for Selenium user-extensions.js file

Disadvantage:

• Selenium IDE is Firefox plugin, thus its support is limited to Firefox only
• It will not support iteration and conditional statement
• Selenium IDE doesn't support Exception and Error handling
• Doesn't support test script grouping
• No Reporting
• No Test Fixtures and data-driven tests
• No Test Dependencies
• No way to take screen shots
• Selenium IDE do not support Database testing
• It is not playing multiple windows when we record it ( switching between windows).
• limitations to open/ identifying the child windows,clicking on the pop ups, etc.,

Clue to Selenium Remote Controller:

Why not use Selenium IDE alone? For one thing, it works only with FireFox, version 1.5 or later. If we run the tests on the server, we can test our application using any browser.