Verify/ATI Conference Site

Gabriele Stauss, Software Test Consultant

Capability Maturity Model (CMM) and Capability Maturity Model Integration (CMMI) have been part of the software development environment and DOD Acquisition process ever since the Software Engineering Institute (SEI), Carnegie Mellon, Pittsburgh, PA published “The Capability Maturity Model: Guidelines for Improving the Software Process” in SEI 1995. This session provides a short overview of the history of the different CMM and CMMI models and describes the importance of the model using everyday situations. The main focus of this session are the CMMI process areas that relate to testing which are Verification and Validation.

Paul Ammann, George Mason University

Description forthcoming.

Bob Galen, iContact

Agile testing is hard and don't let anyone tell you otherwise! First you have to get over the misconception that you don't need testers within agile teams. Then you have to integrate with the developers and provide holistic quality services, all the while testing, testing, testing. And those are the challenges when the going is EASY!
 
There are a number of other challenges associated with more difficult contexts. Bob Galen explores how to handle testing in difficult contexts, including how to attack a total lack of test automation, how to remain agile in highly regulated environments and still serve your PMO or Testing COE, and how to effectively test when your team is spread globally and interactions are nearly impossible. Galen describes contexts and approaches for blending your traditional testing techniques with their agile counterparts, achieving an effective "balance" between the two - for example, in your test planning and quality metrics reporting.

Andrew Pollner, ALPI

As agile development changes the paradigm for development, so too must testing adapt. Of particular attention is the application of automated testing to support agile development practices. Creating a modular, sustainable, test automation framework within the short development cycles of agile seems to be at odds with what we’re told is required to build robust automated frameworks. We will look at automation approaches, tasks, and schedules necessary to adapt to a more iterative environment. Andrew Pollner provides insights into discovering how automation can evolve to support your new development paradigm by describing what techniques are most appropriate and what limitations you should be aware of.

Viranand Strout, Mandiant

Very often when we are challenged with building out our test automation strategies, we are confined by our limitations in our physical and virtual test environments. We end up designing our test automation tools within those confines or working around the constraints of the environment. When a testing function partners with the systems administration function to build out the testing environment, there are many opportunities to build and maintain a more holistic solution for the environment and automation suite. This session explores some of the considerations to building out a testing environment and test automation strategy. There are many network, cloud, and virtualization tools that can be coupled with automation tools that can provide some simple and more comprehensive development and testing capabilities for an architecture.

Nick Olivo, SmartBear

This presentation aims to provide a set of guidelines that testers and test managers can use to perform a thorough evaluation of automated testing software. Olivo will describe some of the common pitfalls that evaluators fall prey to, how those pitfalls may be avoided, as well as points to consider while evaluating both commercial and open source tools. Attendees will come away better prepared to choose the automation software that best meets their needs and automation goals.

Kevin Olbrich, iContact

Description forthcoming.

Tom Wimsatt, Software Process Technology, Inc.

Test management tools are expensive and nobody wants to spend money on testing. The typical fall back position is to manage with spreadsheets that get increasingly cumbersome and lead to duplication. Participate in this presentation to learn a more effective homegrown solution to the test management problem which involves creating a custom test procedure tracking and defect repository using the MS Office Suite on your desktop.

Sreedevi Sampath, University of Maryland, Baltimore County

Test prioritization is a regression testing technique where tests are ordered for execution with the goal of quickly finding faults. Prioritizing test cases plays an important role during software maintenance and regression testing, where a large number of tests accumulate over time from previous versions of the system. Accumulation of tests is exacerbated in user-session-based testing of web applications, where field usage data is logged and converted into test cases. This presentation describes a tool that allows testers to easily collect and prioritize user-session-based test cases.

Arin Sime, AgilityFeat

• Description forthcoming.

Dion Johnson, DiJohn IC

• Description forthcoming.

Ole Lensmar, SmartBear Software

Recent incidents at some of the webs largest websites show that potential security exploits are something you definitely cannot ignore if you are handling sensitive information on your web-site or application. If you’ve been thinking about how to test your application for SQL Injections, XML Bombs and Cross-Site Scripting exploits then this presentation is for you. It will give you a thorough understanding of functional security testing; what it is, how it works, and how it can be automated using popular free tools to detect if your system is vulnerable to common attacks as it grows and changes.

Edwin Reynolds, ALPI

Are your web sites ready for Success?  Users today will abandon you if the site or service is not available when they need it. A sites availability and ability to scale with viral success can make or break your initiative or in some cases your fledgling business.  Load & Performance testing is how you understand and minimize the risks of that abandonment. Join this presentation to obtain guidance on how to obtain the knowledge and techniques you need to be successful with Load and Performance testing.  Ed Reynolds will share best practices that help you: 1) Recognize the difference between load ,performance and stress testing, 2) Identify objectives for load or performance testing, 3) Identify key scenarios and metrics, 4) Learn how to create test cases, 5) Learn how to prepare/understand your load environment & how to run a load test, 6) Learn how to analyze and evaluate the results, 7) Make an informed decision when selecting load and performance tools

Peter Varhol, Seapine Software

A potential by-product of development is technical debt – a working implementation that does not conform to best practices for architecture and implementation. Almost all projects have some element of technical debt. Technical debt must typically be repaid, with interest, later in the application lifecycle.

Most teams are capable of recognizing that technical debt has been incurred, but assessing that debt can be difficult, especially under an aggressive development schedule. But having a good measure of at least one aspect of technical debt makes it possible for the team to assess whether it is better to address now or later.

One operational measure of technical debt is scalability. A robust architecture and implementation theoretically should be able to scale indefinitely. Knowing the ability of the application to scale gives the team significant information about the quality of the architecture and coding practices. Ideally, load testing can be a key indicator of when to pay down debt.

This session discusses the role of technical debt and how to assess technical debt with frequent use of load testing. It will provide a protocol for using load testing as a part of the testing process, and using it to assess the architecture and coding quality. Last, it provides a mechanism for testers and developers to collaborate in decisions surrounding design and coding practices.

Edward Bonver, Symantec

Threat Modeling is one of the most important security activities that a development/QA team needs to perform as part of a Security Development Lifecycle. This activity allows the team to build a complete security profile of the software system. More importantly, it can be used as a learning method to involve software testers early on, to make sure they develop a thorough understanding of the system. Threat Modeling is not always easy to get going for a team that has little or no security experience. In this presentation you’ll get to take a look at why Threat Modeling is so important; you'll also get to explore the process behind it, and how the process is being successfully implemented and followed across Bonver's organization, where development and deployment environments, as well QA team compositions vary drastically across hundreds of products.

Dan Bartow, SOASTA Inc

TBD

Linda Hayes, Worksoft, Inc.

Although almost every company professes to want test automation, not all are willing to invest what it takes to get it done. But don’t give up, get creative! Learn about real life examples of automation entrepreneurs who found unexpected sources of funding that led to expanded opportunities across – and outside - the enterprise.

This session will present four scenarios for expanding your horizons by capitalizing on the value of your automation skills. Attendees will learn how to: 1) Leverage existing automation to get more budget, 2) Identify new automation opportunities outside of IT, 3) Convert an automation project into a product, 4) Turn a job into a company

Burt LeJune, Innovative Defense Technologies

Description forthcoming.

Dorothy Graham, Software Consultant

There is a view that testers should be technical (i.e. have programming skills), especially if they are involved in test automation.

Although this can work well, particularly in an agile team, not all testers need or want to be technical, especially those with a business or application background – it is my belief that this should not preclude them from using test automation!

A tester who is also a software developer is, of course, a very useful person to have on the team. They are able to work with test automation scripts and understand both the testing aspects and the technical aspects of the automation code. But just because this has benefits in some cases, does not mean that this is the best in all cases.

The programming skills needed to work directly with a test automation tool do not need to reside in someone who is also a tester. The testing skills needed to produce good quality tests, those that will find bugs, give confidence and reduce risk, do not need to reside in someone who is also a developer. These are separate skills sets, separate roles.

Skilled testers should not be forced to abandon their vocation, to the detriment of their organizations, because tests are being automated. This presentation addresses how and why technically skilled test automators can and should support and enable non-technical testers to write and run automated tests.

Peter Varhol, Seapine Software

When do you ship an application under development? The question seems simple; you ship it when it's complete, right? But that begs the question "what does it mean to be complete." There are many possible definitions of complete and it is critical that teams and users agree on a definition before any work is done. This presentation will provide measurable criteria for determining when an application is ready to ship. It will discuss how to create objective measures of completeness and how to collect data and track progress toward that end. Peter Varhol will explain how testers and quality professionals may work with application stakeholders to contribute to both the definition and automated measurement that determines if an application is fit for its intended purpose.

Gurinder Negi, Statefarm Insurance Companies

Are you faced with inefficient test automation processes that leave your organization struggling to find consistent answers to questions such as, “where are the test scripts from a previous release?”, “what and how much should be automated?”, and “what concrete benefits have we received from all of our efforts?” A process framework for test automation enables different areas within your IT department to think in a consistent manner about automation. It can make execution of automation easy and collection of metrics for reporting even easier. Such a framework will allow for better testware management while also improving maintenance and execution of scripts. Some of the processes to be highlighted in this presentation include an automation backlog process for documenting and prioritizing automation tasks, an ROI estimation process for estimating/relaying automation ROI, an early engagement process for getting automation involved earlier in the system lifecycle, and an automation inventory management process for source control and configuration management of automation artifacts.

Rick Kuhn, NIST

Software developers frequently encounter failures that result from an interaction between components. Testers often use pairwise testing - all pairs of parameter values - to detect such interactions. Combinatorial testing beyond pairwise is less commonly used because good algorithms for higher strength combinations (e.g., 4-way or more) have not been available, but empirical evidence shows that some errors are triggered only by the interaction of three, four, or more parameters.  This presentation will cover new methods and tools that generate tests for all n-way combinations of parameter values, costs and time required, example applications and case studies, integration with existing test programs, and automated generation of test oracles.

Eric Pugh, Open Source Connections

Description forthcoming.

Patrick Quilter, Quilmont LLC

In today's economy, doing more with the same resources has become the norm. This is exactly what automated testing can deliver. Before it can become the magic wand many of us believe it could be, we need to identify and refine the typical pitfalls and perceptions. Automation frameworks come by the thousands but all share one basic property; they put data on the screen. However, the underlying architecture has the greatest impact on the return on investment. This session will detail a unique breed of automation framework that is centralized (client/server), has a high degree of code reusability with limited maintenance, and can be configured with many automation engines. In addition, this session will discuss integrating frameworks into a comprehensive process utilizing Test Driven Development (functional) and Business Processes Testing (regression). Also discussed will be strategies for creating positive team dynamics where senior level testers can mentor junior level testers while positioning the overall team to be an enterprise solution provider.

Jonathan Cage, iContact

We often hear the benefits of automated acceptance testing, but realizing that promise requires teams to actively embrace this new testing strategy.  Since acceptance testing requires significant effort and close collaboration between testers, developers, and product owners, getting buy-in from all groups can be a challenge.  So given these challenges, how can automated acceptance testing be successfully deployed in an organization?

In this session, Jonathan Cage will relate his experiences introducing acceptance testing as a grassroots effort from within Agile teams at two organizations, and discuss strategies used to overcome obstacles along the way.  Using real world examples, Jonathan will describe how a small group of testing enthusiasts, armed with tools like FitNesse and Cucumber, can incrementally deploy acceptance testing throughout an entire software engineering department to deliver tremendous business value. Participate in this presentation to learn the follow: 1) What is acceptance testing and why do it, 2) Real world experiences and strategies on how to begin, 3) Converting the skeptics by demonstrating value, 4) Acceptance testing patterns and anti-patterns, 5) Tooling and deployment with FitNesse and Cucumber.

Linda Hayes, Worksoft, Inc

Agile iterations present a special challenge for testers who must deal with cumulative functionality within a fixed time frame. Automation seems like the ideal solution, but most companies resist implementing it until the software is stable, reasoning that any savings from automation will be offset by the maintenance required to keep up with changes. Further, traditional record/script/replay approaches can’t be implemented until the code is functional, which is too late to be useful in an agile development environment. This tutorial will describe an incremental approach to test automation that allows automated tests to be written before the code, then rapidly updated as changes are introduced. This enables agile projects to deliver higher quality solutions more quickly by involving all team members in a coordinated and automated effort.

Dorothy Graham, Software Test Consultant

Many organizations never achieve the significant benefits that are promised from automated test execution tools. What are the secrets to test automation success? There are no secrets, but the paths to success are not commonly understood. Dorothy Graham describes the most important automation issues that you must address, both management and technical, and helps you understand and choose the best approaches for your organization—no matter which automation tools you use.

If you don’t begin with good objectives for your automation, you will set yourself up for failure later. If you don’t show Return on Investment from automation, your automation efforts may be doomed, no matter how good they are technically. Join Dorothy to learn how to identify achievable and realistic objectives for automation, show ROI from automation, understand technical issues such as testware architecture and staffing issues, pick up useful tips, learn what works in practice, and devise an effective automation strategy.

Bob Galen, iContact

The move from traditional tester to agile tester can be Extreme (pun intended). There are a wide variety of new skills that need to be acquired. But there are also established techniques that need to be re-honed or adapted as well. Beyond the specific skills however is a larger and more fundamental change-as the very mind of the agile tester is different!
You must move from independent tester to quality advocate. You can't simply verify requirements, but need to elicit and evolve them with your business stakeholders. If you are primarily a functional or manual tester, you'll need to significantly broaden your skills across all areas of software testing. In addition, your "courage" needs to change with respect to organizational and team dynamics. You can't simply be a wallflower who try's to "test in quality". In a word, everything shifts or changes. Join experienced agile coach & tester Bob Galen as we explore the key skill areas, both hard and soft, that you'll need to adjust in order to survive and thrive within agile teams.

Dharmalingam Ganesan & Mikael Lindvall, Fraunhofer CESE

Software testing is an important quality assurance activity. In practice, software testing consumes at least 50% of the development effort. Many engineers consider software testing a low-profile and boring task. For example, Java Software engineers manually write JUnit programs to test the system, which can be both tedious and error-prone. An analysis of such JUnit tests reveals that they typically follow a certain pattern. So why don't we generate JUnit tests? By the way, why stop there, why don't we generate all kinds of unit tests (i.e. xUnit)? In this tutorial, we introduce the Fraunhofer Approach for Software Testing (FAST) that has the capability to make testing a fun and a high-profile task by generating xUnit test cases. The goal of the FAST approach is to automatically generate ready-to-run executable test cases from lightweight models. The FAST approach has been applied to test case generation for a wide variety of systems including a) module-level testing of middle-ware, b) requirements testing of web-based systems, and c) testing of GUI-based standalone systems. We will present industrial experiences of generating xUnit tests using the FAST approach. In this tutorial, the participant can expect to learn state-of-the-art testing methods and tools that they can use in their jobs to make testing more fun, efficient and effective.

Don Goodman, Mandiant

By nature, QA infrastructures are more fluid than production infrastructures. New environments need to be quickly brought up, torn down, and revamped on a regular basis to meet the needs of the QA organization. Those in the systems administrator’s role must be able to respond quickly and deliver reliable test environments.  This session explores virtualization, automation concepts, tools, and scripts that can help a systems administrator build a responsive infrastructure for their QA organization.

Andrew Pollner, ALPI

Cloud testing offers several advantages over traditional testing. It shifts the burden of installing, configuring, maintaining, and updating testing tools to the vendor. It also lessens or eliminates the need to build and maintain servers to support testing functions. Testing through the cloud expands the reach of testing across geographical locations. With all these benefits come some additional challenges. Andrew Pollner provides insights on when cloud testing makes sense, how to best manage such an environment, and where a hybrid solution can benefit you.