Sunday, October 17, 2010

Optimization through video recording

Last week I got request from my colleague to run one of his automation on particular time at night. He told me he could not access internet at that time because he would be traveling. He was suspecting some timing issue because of which automation was hanging. Automation sometimes run at 12'o clock midnight which was changing the date and causing automation to fail.

I had some context of that automation but not complete knowledge, so he came to me and asked me to start automation by telling me the steps which I need to run. I was busy in some other stuff and in no position to listen to his instructions; suddenly I gave him my test machine with recordMyDesktop (It is a video recording tool) running on it. I asked him to run whatever steps he wants me to perform and that will get captured in video. I told him I will see the video and perform exact steps. I recorded all the steps in video and saved it in my laptop.
 
At his requested time in night I started the  video and ran through same steps without making any mistake. After that exercise, I felt it is good way to optimize due to following reasons,
-Those  instructions were not important for me to remember so it would have been waste of time to put my effort on that.
-I could  have missed some of the important steps if I would not have recorded the video.

According to me, these types of information or commands which are difficult and unimportant  to remember can be noted or recorded down and can be used in future to save some good amount time .
 



Friday, October 1, 2010

Practicing SFDPO Test Heuristic

Testing Heuristics are set of guidelines which helps you to generate test ideas more effectively and systematically. There are many different Test Heuristics and you can get lot of information about Testing Heuristics from internet. In this post I will talk about my experiences with SFDPO heuristic.


SFDPO mnemonic which stands for
S- Structure,F-Function,D-Data,P-Platform,O-Operation."SFDPO" is often called as "San Francisco Depot" to just remember the heuristic. It implies that when you start developing test idea for any feature or product you should give thought to all these aspects(SFDPO) mentioned above. It will help you to develop test ideas in systematic way and you will have lesser escapes.


I always found it difficult to apply test heuristic or I use to apply them with my common sense and always felt that I can miss any aspect if I don't follow any systematic way. So I devised some way to  apply this heuristic for feature I was going to test. This way helped me during my initial practice and  with this I can generate more useful test ideas.


These are steps I followed
Get the information:- Gather whatever information you can get about the feature.
  -I went through design docs,prepared questions and checked them with concerned developers.
  -Studied the behavior of similar feature in other products.
  -Explored underlying technology used by the feature.


Prepare the draft:- After gathering information I went ahead and prepared a draft with it. Draft will contain
 -What I understood about the feature.
 -About its internals.
 -How it is implemented.
 -How users have used same feature in other products.


Now I have categories according to SFDPO heuristic. Once I got my draft up, I went through the draft and found the terms which should belong to particular category in Heuristic.
-Like "frontend connects to backend through api" will go under Structure.
-Data in system will be impacted during upgrade will go under Data.
-All actions performed by user in draft can go under functional aspect.


As I got few basic ideas about each aspect then  I started generating more ideas about every aspect.

Saturday, September 25, 2010

ElementNotVisible exception in Selenium WebDriver

Problem I was facing 
Recently I was using Selenium Webdriver for one of my project and got ElementNotVisible exception, even though element was visible on screen.I went through the Webdriver old forum posts but could not find exact solution. But forum posts and webdriver FAQ gave me some good pointers to solve the problem.

Cause of the problem I figured out


This issue occur to me due to  2 reasons.


First reason:-I was using relative Xpath and same Xpath exists on page more than once.There could be many reason for that, but in my case element I was trying to click, was under a dialog box and HTML page has template for that dialog box. Template Dialog box was hidden on page.When I was using relative Xpath,selenium was trying to hit hidden element inside the template dialog box and giving exception ElementNotVisibleException. 


solution:- Change Xpath in your code, so  Selenium don't get confuse between two elements.


Second reason:-I was trying to click a button which changes on hovering mouse over the button. There where two elements related to button first one was shown without hovering mouse and other gets shown when you hover mouse over button. Element shown on hovering mouse was performing actual function using a javascript and that element was not visible until you hover mouse over button .Clicking on that directly was giving ElementNotVisibleException.


solution:- For hidden links and button you can directly run the javascript behind the hidden button which will serve the purpose. 

Friday, August 20, 2010

Knowing the bug

Testing involve many activities like finding issues,reporting status,exploring,understanding  your application and may be more. Among all I personally like  debugging or troubleshooting the bug. Some people might say that it is not tester's job and tester's job is to just report the issue. I don't want to argue on that but I would like to say that knowing and finding root cause of bugs help me in following ways,


-That experienced can be used when you test similar set of application.
- If you are aware that you are going to hit same bug with newly design feature, you can very well give your input at design time.
-You will get to know more about internals of AUT, which will help you to isolate bugs  and make your bug report stronger.

I love to find the root cause of bugs  before reporting the bug. I have developed some debugging experience  but if some new features going to come I always like to question developer. Question will be "How they will go about debugging if they found some bug in the new feature". We can use that information as much as possible until it is not that related to code debugging. I would like to put some debugging experience with web application which can be used in general. I also use this technique to isolate bugs. Suppose you hit issue while testing web based application.


-First thing will be to divide it in two parts. Issue can be either server side issue or client side issue .
Server side debugging can be started started with server logs. There can be two types of server logs for web based application
  • Web Server logs :-Sometimes these logs are useful to identify what type of HTTP request is coming to server. There are status code for each type of HTTP response, So if your application works over HTTP then status code can help you to check if every request coming from client is fine. You can make out from this logs that issue is client side or server side.
  • Application server logs :- Logs generated by application and often Application server  logs + Web server logs are combined to form one log. Logging in debug mode helps to find out any exception or error. Exceptions throws stack trace, which highlight origin of error. You can follow the stacktrace and if you have little bit idea of the code you can find the actual root cause of issue.
If you cannot find issue in server side then you can try debugging in client side.For web based client side application you will find most issues related to javascript.To debug that
-Firefox has Error console or you can use Firebug as Firefox extension, which gives fare amount of understanding of script errors.
-Chrome comes with embedded developer tools for debugging javascript errors.

It is always good to question your developers about debugging. Tester are the one to see the issues in first place, so they can be one of best person to debug and this can help you avoid getting "bug not reproducible" responses  from development.









Wednesday, August 4, 2010

Browser compatibility what i understood

Now a days most of desktop based application moving towards web based. I guess everyone sometime or other get to test web applications.

Assuming your web AUT(application under test) has lot of features which will result in huge number of testcases.  Let's say  AUT has hundreds of testcases as it has been the case for me, most of time. Basic requirement for test team is to make sure all features works fine on every web browser we support. Best way to achieve this is to run every testcase on every browser which will be an exhaustive way. I really don't like this way and I feel there should be a way to optimize this. Validating your web application for all web browser is callled cross-browser compatibility thats what i am going to focus on .


From my experience what  I have discovered is that there are two areas where behavior of your application changes from browser to browser. If I concentrate on those two areas then I can optimize  on testing for browser compatibility.


 Those areas are
  1. CSS 
  2. Javascript
  • CSS  is guidelines which tells browser how to display your page. These guidelines are stored  in a file( with .css extension) with web pages.There are standard for these guideline but different browser interpret these files differently, which causes your web pages  to display incorrectly. Most common issue you will find  due to CSS are,
    • label size not proper.
    • Text font vary across browser.
    • pop-up menu does not render properly. In general you can say all UI rendering issues.
         To find CSS issues,
    • navigate through  all pages/dialog box  and popup menus  on your web application.If there is any UI problem  due to CSS  you will easily observe it .
  •  Javascript is scripting language for browsers  to perform some dynamic operation on browser.Usually inputs given by user can be processed immediately on browser itself rather getting it processed from server .Every browser has there own javascript engine and browser sometimes fail to execute these javascripts in runtime.  Issues mostly due to javascript  are,
    • WebElements(links,buttons and images etc ) calling javascript does not work at all
          To find issues related to javascript,
    • Test all links which call javascript or uses javascript in some way .You can use developer tool like firebug .Firebug is add-on to firefox which helps you see internal of web pages you are browsing

According to me, if you attack these two areas you  can optimize on browser compatibility and avoid  running every  testcase on every browser.

Tuesday, July 13, 2010

When to automate? is equally important

Dhanasekar S's  recent blog which differentiates between importance of manual testing and automation testing triggered this blog. With all my automation experience I can say that there is one thing important in automation according to me.
That is 'When to Automate?'
As James Bach says "Test automation is any use of tools to aid testing". That is very much true, but to identify what are those tools and when to automate  is usually difficult job for me .So I came up with the following parameters to find out when to automate .  
  1. When I am doing any manual task repetitively , e.g.
    1.  Creating data set for my application.
    2.  Running same configuration every time after preparing the setup.
    3.  Even doing ssh(remote login) to same IP  frequently.
  2.  Smoke Testcases which need to be run  with every new build:- This case is also kind of a repetitive task where you need to run scripted testcases every time a new build comes. It is good to automate these as they give you more time to explore new areas in your application.


    I  would like to cite an example related to case 1 .
    Recently , I started writing testcases  for new features using mindmaps. Mind maps gives better overview of my testcases and their coverage.It also helps me getting quick review from development team .

    But the issue is when I need to write all my testcases again to my test management tool ie Testopia. Let me tell you about Testopia ,Testopia is awesome test management tool from mozilla which provide api's over XML::RPC  to update /modify  testcase database. 
    Anyways my above task was repetitive and obviously  boring which provoked me to  write a perl script to  parse test cases from mind map's file and directly update in Testopia. "Same script can be used in future" was also an another trigger point to automate the task. Scripts to achieve the above task got ready in just 45 minutes  and I updated 100+ testcases with the script to Testopia. In the above scenario I was not told to automate.When I felt bored with coping testcases , I decided to automate the Testopia update task and  that was my call .Many of my tester friends told me that they don't have any exposure to automation because management never told them to do any. According to me ,when to automate and what to automate  should come from tester and not from management. I usually see test managers  giving importance to the automation tools in a tester's skill set but according to me knowing when to automate is equally important as knowing the tools.

    Tuesday, June 8, 2010

    Selenium Webdriver running remotely

    Recently i started developing my automation code using Selenium2 code .Most of my code is for firefox driver .I need to confess that i liked Selenium2 really there are some stability issues in comparison to Selenium1 but it is good .

    Default usage of Selenium2 is that it will run server and client from same instance and that will control the browser elements,but Selenium2 provide a way to run server and client on remote location as you use to do in Selenium1.Remote Webdriver is the class which help you to run client and server separately .RemoteWebdriver provides a DriverServlet which if you add inside any servlet container you got your selenium server up.
    There are few examples for this in Selenium2 wiki site .As i have never ran servlet application before so i got in to little bit of issues following that .But finally i got it running and i used jetty as servlet container .Two ways you can include that in jetty

    1.Run jetty server on machine and put a war file on location pointed by jetty config file .
    2.Start jetty server from your application and add you servlet class to it .(I followed this method)

    I downloaded the selenium2 code and built remotewebdriver werver and client.
    command to built server
    ./go clean selenium-java selenium-server-standalone which will result in build/remote/server/server-standalone.jar
    command to built client
    ./go //remote/client:client:uber which will result in build/remote/client/client-standalone.jar
    *go is script comes with selenium2 code and code below i got it from wiki just made few changes .

    Server code
    AppServer.java
    =============================================
    import org.mortbay.jetty.Server;
    import org.mortbay.jetty.nio.SelectChannelConnector;
    import org.mortbay.jetty.webapp.WebAppContext;

    import java.io.File;

    import org.openqa.selenium.remote.server.DriverServlet;

    public class AppServer {

    public static void main(String s[]) throws Exception{
    Server server = new Server();


    WebAppContext context = new WebAppContext();
    context.setContextPath("");
    File st=new File(".");

    context.setWar(st.getPath());
    server.addHandler(context);

    context.addServlet(DriverServlet.class, "/wd/*");

    SelectChannelConnector connector = new SelectChannelConnector();
    connector.setPort(3002);
    server.addConnector(connector);

    server.start();
    }
    }
    ================================
    Add server-standalone.jar ,jetty-6.1.9.jar and jetty-util-5.1.9.jar(you can download from jetty site or selenium2 code also has in third_party folder) to classpath and run above code.
    Your server will up with following message ---
    2010-06-08 11:07:00.730::INFO: Started SelectChannelConnector@0.0.0.0:3002

    Client example code
    FirefoxClient.java
    ================================
    import java.net.URL;
    import org.openqa.selenium.remote.DesiredCapabilities;
    import org.openqa.selenium.remote.RemoteWebDriver;
    import org.openqa.selenium.WebDriver;

    public class FirefoxClient {

    public static void main(String s[]) throws Exception {

    URL url=new URL("http://127.0.0.1:3002/wd");
    DesiredCapabilities capabilities = new DesiredCapabilities();

    // ... but only if it supports javascript
    capabilities.setJavascriptEnabled(true);
    capabilities.setBrowserName("firefox");

    // Get a handle to the driver. This will throw an exception
    // if a matching driver cannot be located
    WebDriver driver = new RemoteWebDriver(url,capabilities);

    // Query the driver to find out more information
    //Capabilities actualCapabilities = ((RemoteWebDriver) driver).getCapabilities();

    // And now use it
    driver.get("http://www.google.com");
    }

    }
    =====================================
    Add client-standalone.jar to class path and run the above code .