To ensure that estimated effort and time to prepare for and execute testing are realistic, we need to support our estimates with facts, assumptions, an accurate understanding of necessary activities and constraints, historical experience, options and related risks. This enables us to understand the scope and amount of testing to be performed to deliver expected results within tolerable risks. In her presentation, Carolyn will describe an effective approach for developing realistic accurate estimates based on project type, size, scope, complexity, process, and experience on similar projects. She will cover identifying and capturing appropriate metrics during projects, and how to use these to support reaching agreement with project managers and stakeholders on the time and resources needed to do effective testing on future projects. Carolyn will also illustrate an estimating tool to help you capture estimates, track project changes that affect estimates, compare the accuracy of estimates with actual results, and adjust future estimates based on history.
Software is complex, and complex software doubly so. Our testing teams, comprised of fallible humans, are challenged with finding the tiniest of errors in a seemingly infinite domain. Once we’re satisfied with a version, we start all over again. It’s no wonder that teams are looking to automation in hopes of easing their workload. Beginning correctly is essential, however, and, in this session, Steven will explain how to do just that. He will begin with assessing team skills to determine the level of technical skills and interest among the testers and developers. He will discuss the need for finding allies and for getting management buy-in before starting – or forgiveness after getting underway. You’ll learn methods for identifying the areas of your project that would be good starting points for automation, and the importance of determining the right course of action.
Today many large enterprise applications are built and deployed in a private cloud application environment. A private cloud provides a flexible and cost effective solution to align resources with application priority and demand. But, a cloud based environment presents unique challenges. The tools for management of virtual machines and the application environment are constantly being matured. The techniques for load balancing, monitoring, and optimization in cloud environment are still evolving. The private cloud environments are usually built on engineered systems from vendors such as VCE V-Block, oracle Supercluster and others. In this presentation, Vic and Vinkesh will share real world experiences monitoring and optimizing in a private cloud environment.
Burgeoning customer expectations coupled with rapidly changing business environment are stumping the most time-honored IT strategies. Proliferation of new age consumer centric SMAC (Social, Mobile, Analytics and Cloud) technologies and agile IT models are disrupting the traditional ways of business delivery. These tectonic shifts are asking for a rethink of traditional software testing practices. In order to be relevant for tomorrow, enterprise quality has to go beyond software testing. Next-age testing will be driven with an engineering led outlook to assure end to end business processes. Using a collaborative, integrated and predictive approach, the end objective of quality assurance will be to assure all apps deliver world class user experience as against finding defects.
Agile methods are significantly transforming the software development landscape, but enthusiasts often maintain hopeful beliefs about agile practices that fall short of reality. By recognizing and avoiding five common misconceptions surrounding agile development, organizations can plan and implement agile initiatives more effectively. Join Stephen as he shows how agile methods can be difficult to use, stretching the abilities of practitioners, and how the results may be hard to compare to previous standards of success. You will learn that agile methods have drawbacks in addition to benefits and that it’s necessary to carefully consider both sides to avoid silver bullet thinking. Also, Stephen will argue that many supposedly new values of agile are not new at all, and failing to recognize this can lead organizations to think that they are driving change when they are simply recycling old pathologies under new names.
Leading testing means much more than getting a team to follow you. It means fostering teamwork that will enhance the effectiveness of your whole project team. It also means earning credibility for testing, so that managers and fellow team members understand testing’s value, and the importance of the information your testing uncovers. You don’t have to be a manager to be a leader—but if you’re not also a leader, you can’t be a good manager. In this mini-workshop you will work with others to answer the questions:
- What does it take to be a leader?
- Where do exceptional test leaders focus their energies?
- How can you get to be an exceptional test leader?
The key to successful adoption of any development methodology is a clear understanding of the roles and responsibilities of each team member. As agile continues its rapid adoption, it’s essential to define the role of QA in Scrum as concretely as we’ve defined the other team roles. Agile gives QA an opportunity for broader and deeper involvement in the software development lifecycle, enabling us better to ensure quality not only by finding defects, but also by preventing the introduction of defects in the first place. We can drive defect prevention by asking key questions during requirements definition, and building comprehensive acceptance criteria before implementation begins. This results in high quality development, testing, and story acceptance. Finally, to prevent the steady growth of technical debt, QA can help the team plan story points for fixing both known and unknown defects as part of every sprint. Join Karen to discuss prevention vs. detection in action…
Most applications have features to enable printing or saving of documents to files. The task of validating that the documents meet layout requirements and contain expected data is a tedious and time consuming manual task that is prone to human error. For these reasons, testing of printed reports and documents is often neglected until customers report issues with them. Automated validation has typically been challenging due to resolution issues and the dynamic nature of reports. A solution has been developed, however, to address these challenges by breaking down the verifications into two parts. First, using bitmap comparisons to validate sections of the printout or document that are not supposed to change against a known baseline and second, using OCR technology to validate sections of the document or printout that change with the data. These components are integrated into an automation framework allowing for a comprehensive process. Join Ivan and bring this helpful solution to your organization.
Improved access, cost effectiveness, reduction of execution time, flexibility, and other benefits yet to be defined make cloud computing the increasingly popular choice for developing and operating an application. As infrastructure becomes more costly and applications become more complex, the cloud offers organizations the scalability and availability that may not be achievable in a single, unique environment. This presentation will provide a basic understanding of cloud, the interconnection between requirements and cloud architectures, and the types of testing required as complexity increases. Topics will include benefits and weaknesses of cloud computing, basic cloud definitions such as Infrastructure-as-a-Service (IaaS), multitenancy, and failover architectural approaches. The requirements supported and types of testing to consider for maximum benefit will also be discussed.
We’ve all heard about strategic planning. It’s the preparation for battle plans or achieving goals. People talk about strategies when they want to change something or achieve something. So, when we talk about test planning and test strategies, are we talking about the same thing? If we have a plan do we need a strategy? If we have a strategy do we need a test plan? In today’s market with the need for leaner, quicker and effective testing what are options to consider with regards to test strategy and test plan documentation.
This session discusses test planning and test strategy development and suggests approaches for today’s testers and test managers. Clyneice will be discussing test strategy and planning definitions and to-do’s, how to use test strategies as part of your testing, and feasible formats for documenting the strategy and plan. Attend to learn how to apply these two concepts within your organization.
Why do so many companies cancel their test automation projects? It turns out that the continuous maintenance of the test automation scripts can be a nightmare! Instead of saving money and effort, test automation often results in chaos. Model-based testing (MBT) can resolve this issue. In MBT test cases are based on a simple model of the application. The advantage is obvious; instead of having to write test scripts from scratch and manually adapt them in each test cycle, all that is necessary is to keep the model up-to-date. The costs for test maintenance are reduced dramatically thus leading to shorter test cycles and better product quality. In his presentation, Stefan will show how MBT has been successfully applied at the European Space Agency. In this context, he will introduce the MBT tool TEMPPO Designer that can generate complete, executable test scripts in various formats including HP QuickTest Professional. Stefan will also highlight the recommended process for MBT and give practical hints for its efficient application.
These are exciting times to be a software tester! The testing community is energized to bolster our skills and provide greater value than ever before. But…there is a challenge. Despite the shift we are seeing in testers to understand the possibilities of their craft, our software peers and management are simply stuck. The tiresome conversations around the role of the tester to “assure product quality”, “ensure zero defects“, “achieve 100% coverage”, and many more are still prevalent. These misguided perceptions of testing can make it difficult for testers to implement exciting changes – big or small. Let’s discuss how can you raise awareness, reshape perspectives, and help raise the bar for you and your organization! Join Lynn and learn to identify stakeholders, key influencers and trusted advisors that form your organization’s views on testing. Understand the perspectives of others including what shapes and shifts them. Explore how to advocate for testing and enhance your ability to evolve the practice within your organization.
Have you ever poured your heart and soul into a masterpiece of a test strategy only to learn that your client or team has only skimmed it? Having a team or client who understands their responsibility and can actually execute on the strategy can make all the difference in a project’s go-live date. Above and beyond the document, successful strategies have a solid supporting framework of tools and templates that break down the strategy into chunks of executable tasks and messages. Building on the Tell-Show-Do adult learning model, this session will demonstrate how to create a strategy supporting framework that will have even the most senior leaders of your organization speaking “Testing!” This session will be very hands on. You should come prepared with a few key points from your current testing strategy. You will leave the session with outlines, tools and templates tailored to your own organization’s needs.
During the era in which organizations were evolving their architectures to meet service-oriented standards, the ROI of services was predicated on reuse. The difference between SOA and APIs is the prolific nature in which services, now known as APIs, are being used to extend interconnectivity beyond the corporate boundary. As services cross borders, your APIs and your brand become inextricably intertwined. Thus, while SOA required trust, APIs demand integrity. The more secure, reliable, and dependable your API, the better the chance of consumption and the greater the potential for business expansion. However, if you’re providing a questionable interface, you are likely to lose business since switching costs associated with API integration are so low. This presentation explores the top three challenges to API integrity and outlines strategies and best practices for reducing the risks associated with each challenge. The challenges discussed will include a significantly increased attack surface area, elevated potential for unintentional or malicious misuse, and validating performance in the event of the erratic or surging demand.
The agile manifesto values individuals and interactions over processes and tools, but how do we build strong relationships within teams that are not collocated – is it even possible? Can a distributed agile team be successful, or is failure inevitable? In this presentation, Heidi and Michael will share their experience working as testers on a distributed agile team and explain how they made it successful. Regardless of project methodology, communication is the key, and it has never been more important than in the case of distributed teams. Not being able to casually stroll over to a colleague’s desk requires a different set of tactics though. Join Heidi and Michael for a walk-through of what they see as the key points in making a distributed agile team successful, covering everything from team building tips to communication tools. Learn to create trust and team spirit, adopt flexibility without losing structure and how to get the most out of some valuable communication tools.
Performance agile testing transformation is a journey that is as much about people and organizational culture as it is about process change. Most people in the technology field understand that agile is a group of software development methodologies based on iterative and incremental development. It is a methodology where requirements and solutions evolve through collaboration between self-organizing, cross functional teams. However, waterfall and agile projects take fundamentally different approaches in completing requirements, design, development, and testing activities. Because of the short time frames in agile, performance testing should be considered in all stages. A well-executed agile performance testing strategy can detect performance defects early and help achieve faster time to market. In this discussion you will look at best practices for achieving performance testing in an agile environment. Kim and Bryant will cover an overview of the agile methodology, performance testing within sprints, and agile performance testing challenges and benefits.
The average cost to fix a defect at the end of the lifecycle is 400-800 times greater than if it were addressed earlier. On average, poor requirement practices account for 60 percent of a project’s time and budget. Organizations with well-defined, closely managed, and effectively measured quality activities succeed and continuously improve. Yet, in a recent survey, 77 percent of managers reported that bad decisions have been made due to a lack of accurate information. During this presentation, David will discuss the TMMi model which is the de facto international standard to assess and improve test maturity, featuring independent best practices from more than 14 quality and test models. He will review the best practices and identify key quality test measures that should be institutionalized to collect the data needed to determine effective and efficient organizational performance. You will explore how TMMI can be used in conjunction with the GQM model to ensure that upper management is provided with the information they need to make informed business decisions.
In these challenging economic times, your organization needs to nail down the relationships and processes between requirements management and quality activities. These relationships and activities will help your organization better calculate and increase the return on investment of these processes. Attend this presentation to discover best practices around increasing requirements and quality management efficiencies in order to lower overall cost. Topics will include incorporating requirement peer reviews and static testing, risk reviews and prioritization, adding a requirements management plan to your processes and better understanding the costs of requirements errors. Gain the competitive edge needed to address the wide-ranging challenges of requirements and quality management in your organization.
Worried that your mobile app has a quality gap? An alarming 65% of all apps today have a 1 star rating or less. That’s over 700 million apps! According to the fourth annual World Quality Report, “a troubling two-thirds of organizations (65%) do not have the right tools to test mobile applications and 52% do not have access to the required devices.” If not addressed, these gaps will have a major impact on the quality of your mobile applications. Join Costa as he discusses the unique challenges of mobility and proper risk mitigation, the importance of maintaining a proper device lab, and how to get the maximum test coverage across thousands of unique devices. Costa will explain how to leverage existing tools and evaluate automation options, how to increase the long term quality of your apps, and decrease time to market and testing costs by up to 60%. Stop tossing and turning at night over poor quality. Closing the mobile app quality gap takes action. Join us!
Quality does not start or stop with testing. It begins with the design of the code through production. This is the cultural change many companies would like to make. Going from testers testing to everyone testing, from manual testing to automation, from commercial tools to open source, from four week release cycles to releasing every day, these are some of the major changes that Anaf has successfully implemented. Join Anaf as he walks you through a change in mentality that will help you to enable your organization to inculcate quality into every step. Anaf will explain how to make things easier, using the right tools for the right job. You will discuss how to hold vendors accountable creating a true partnership. Learn how to fail fast, recover quickly, and to be perfectly transparent using data as your guide.
Before diving into a software quality metrics program, learn from the mistakes of others. In this session, you’ll review some of the most common mistakes when implementing a software quality metrics program. And, in the process, you’ll see the difference between measurements and metrics and why indicators are needed for meaningful evaluation. Philip will also cover 10 common test metrics and see how they can not only help us monitor and evaluate our testing and product quality, but also answer questions important to our development process. You’ll also learn about Basili’s infamous GQM paradigm, and how we can use it to tie our metrics to goals and questions so that your metrics have real meaning for stakeholders. After all, the last thing you want is to spend hours pounding out a spreadsheet that no one looks at or uses to take action.
Ambiguity reviews are a proactive approach in the requirements phase to ensure requirements are quantitative, clear, and concise. This results in defect reduction during design, construction, and testing phases. In this session, Susan will discuss how to perform an ambiguity review, how to structure an ambiguity review template, providing basic metrics for the review, and keyword tips to find ambiguities in your requirements. Attend this session and you can expect to learn how to improve the effectiveness of your analysis to define test strategies and test cases, how to Increase the requirements understanding of your offshore development and QA teams, how to extend ambiguity review principles to other lifecycle documentation, and how to increase test coverage through an increased understanding of the requirements.
Throughout the history of software testing, the profession has evolved from expectations of just meeting requirements to ensuring fitness of use and overall quality. In this presentation, Joseph will discuss the challenges our stakeholders have in understanding our craft, as well as some of the negative perceptions of value. You will see how this situation is driven by our typical methodology agnostic way of testing and how our history has driven what most believe to be our purpose. Joseph will lead you through a redefinition of that purpose with advice on how to implement a new mindset. You will discuss embracing a newly defined purpose of software testing, where the emphasis is placed on providing information, not just raw data, to stakeholders. This new intent will transform testing into a service minded group whose value is transparent and ultimately empowered by their stakeholders, instead of just tolerated by them.
Some of the top challenges of developing and deploying mobile apps and websites include managing many different devices, supporting multiple operating systems, and integrating the right set of testing tools. While many companies are aggressively figuring out how to monetize their mobile channels, those tasked with quality assurance need to support these efforts for all types of devices, and in an efficient and cost-effective manner. Join this session to learn how leading enterprises are approaching their testing challenges and have integrated mobile into their existing test plans. You will learn why mobile requires a unique approach and how to ensure that apps have optimal functionality across various devices, operating systems and platforms. Techniques to leverage your existing test tools such as HP UFT and IBM Rational to maximize any existing investment will be discussed as well.
With an ever increasing amount of functionality and content being accessed from mobile devices, performance testing of mobile apps has become critical. According to the 2012-2013 World Quality Report, 64% of firms identified performance as a focus for mobile testing over both functional and security testing. Consumers expect their mobile content to be delivered fast and they won’t hesitate to find other options when it’s not. Making the assumption that mobile traffic will impact the system the same way as traditional web users will produce inaccurate results, fail to expose performance issues, or worse, result in a poor user experience. This session will discuss how to address unique mobile web and application challenges such as device / OS diversity, mobile user behavior and network variability as well as the specific differences between performance testing mobile apps and traditional web apps. Attendees will walk away with an understanding of best practices for ensuring the performance of their business critical mobile applications.
Gartner’s IT Key Metrics indicate that development and unit testing, system testing, defect removal and quality management comprise around 50% of application development. This combined with a “not just right but right the first time” expectation, makes agility and faster time to market without compromising on quality a key driver for Industrialization of testing services. In this approach testing services are provided in a factory like manner, as an assembly line, with prebuilt assets and out of the box solutions. This, in turn, is leading to formation of Specialized Testing Functions or CoEs. In this presentation Deepika will highlight best practices, process levers, and consulting techniques based on her experience in Industrialization of testing services. Attend to learn about best practices in setting up CoEs and techniques to define test organization roadmaps. Deepika will also present sample process and tool assets from her consulting engagements.
Anyone leading a test organization has heard from senior management on the “high cost of testing.” This cost, however, is overwhelmingly associated with dealing with failures. The question is what do you do about it? Books have been written on the Cost of Quality (CoQ) breaking down the CoQ into prevention, assessment and failure. A software quality organization should be spending time testing and not recovering from failures. This presentation will offer ways to apply the principles of CoQ to first identify those costs that are hidden in plain view and then strategies to reduce the cost of failure. It is not always easy to reduce the failures, at least not as a first pass improvement. So, reducing the cost of the impact these failures have on the test organization is key. Beyond that, what are the responsibilities of a test organization toward preventing failures (not just finding them) and how can test organizations best achieve that goal?
As enterprises go mobile, integrating strong security into mobile applications is imperative. Join Manish as he talks about the spread of mobility and the challenges and options to providing a robust security strategy around it. This presentation will help software and mobile QA teams learn about different types of mobile vulnerabilities and challenges that are driving the need for mobile security and for providing safe mobile access to business apps. Manish will showcase a mobile threat model to demonstrate the weak spots in mobile apps. He will cover industry statistics, the developer and attacker’s perspectives, mobile app risk areas, the consequences of vulnerabilities, measures to take to secure your app, and the security analysis of mobile apps.
How many testers have bugs in their defect queue that were opened more than a year ago? More than two or three years ago? The fact is software development efforts are almost always focused on delivering new functionality, leaving the “work arounds” in place for bugs released in prior versions of applications. Want to know a good way to close out those bugs once and for all? The answer is a Bugfest! Join Shaun Bradshaw as he explains how he has implemented a bugfest using Kanban to bring visibility to the myriad of outstanding problems left over from previous releases, to order and prioritize the work to clear out the nastiest offenders, and ultimately to exterminate those pesky bugs!
When it’s critical to deliver a well-performing application on-time, the first time, the traditional process of load testing the release-candidate is simply too little, too late. The sooner we start measuring and tuning performance, the lower the risk of experiencing unpleasant and costly surprises shortly before or after going live. We know it is between unreasonable and impossible to conduct traditional load testing before the application is at least functionally viable. Scott shares his T4APM™ lifecycle monitoring and trending framework designed to both dramatically reduce the risk of late-stage performance surprises and to streamline late-lifecycle load testing. This framework has been shown to integrate seamlessly into a variety of processes with or without specialized tools or customization. Better still, begin implementing the framework in a few minutes and maintain it with even fewer minutes of ongoing effort. If delivering well performing applications, faster, cheaper, easier and more reliably matters to your organization, then this talk is for you.
If a website or mobile app is not accessible to all potential visitors, what is its quality? Services, products, information, and entertainment on the web can all be made available to millions of consumers with vision loss, hearing loss, motor control loss, or even information processing challenges by complying with web accessibility standards. Assistive technologies enable access by converting web pages’ or mobile screens’ text and images into computerized voice. But, these technologies cannot interpret websites or mobile apps that are not built and tested for compliance to Section 508 Web Accessibility Standards. Join Nancy to learn about the Section 508 standards and how to test for web accessibility using screen readers and open source tools. Experience screen reader technology on both an accessible and non-accessible site and the impact on usability. Learn how your QA team can be advocates of accessible websites and mobile apps throughout the project life cycle and add accessibility testing to your testing capabilities.
Deciding to use outsourced testing to alleviate some of the challenges facing businesses today still raises questions. What is the best way to incorporate outsourced testing into the client’s setting? One solution is to build a long-term relationship with a testing vendor to provide testing as a service. Representing the client and vendor perspectives respectively, Monica and Christin share their experiences in building a true partnership that allows both sides to grow together. This unique presentation provides rare insight from both the client and vendor viewpoint into how partnering around testing as a service has provided the flexibility and specialized expertise that various projects require, while also bringing additional value in terms of new ideas and influences. Join Monica and Christin as they recount where their joint journey has taken them so far, discussing not only the challenges and obstacles, but also the unexpected benefits and pleasant surprises.
Test data management is often a forgotten requirement. That is, until the consequences of mismanaging or failure to manage come home to roost. In today’s world, IT organizations produce and rely on huge volumes of data. As systems become more complex, so do the test cases needed to validate them as well as the test data needed to execute the test cases. In this presentation, you will discuss the test data management lifecycle and real world examples of test data management gone awry in order to better understand the importance of good test data management practices. You will learn common challenges of unmanaged test data and discuss the benefits and drawbacks of various methods of instantiation, management, use, and disuse of test data. It is important to note, that this session will not cover test case management, data element profiling, identification techniques. Rather, it is about data in terms of data sets and what is needed to support a testing effort.
With the advent of business process outsourcing, more firms prefer to leverage vendor products rather than building in-house technology as it saves time and money. Whether it is a new product launch or enhancing existing business processes, most new development projects involve integration of third party systems with in-house applications. Testing is a challenge in these projects with unforeseen issues occurring during test execution. Join Karen and Sujata as they share some lessons learned and best practices that can help to overcome these challenges. Learn the importance of testing early by identifying critical paths in the integration and enabling project planning to place testing in early project phases. Understand that no assumptions should be made. Use any existing case studies of previous vendor integration for better understanding of interfaces. Finally, consider the importance of effective communication across QA, business, and development teams on both sides right from the beginning of the project.
In today’s global economy, your app’s footprint is growing faster than ever. While that’s great news, you now have a new challenge of making sure your app works as intended when it launches in new markets. The apps that succeed are the ones that understand the context in which they are used. Understanding localization demonstrates to your users that you understand them, and that can keep them coming back. In this talk, uTest’s Peter Shih will demonstrate the importance of excellent localization testing beyond simple language translation, reveal the pros and cons of current localization options, and provide actionable information that can help you ensure your app is delighting the people who use it, no matter where they are in the world.
A question that gets asked more than any other is, “How do you talk to your CIO about software testing?” As software testers typically are not in positions of authority, this question seems natural. What is more important though is to find out why there aren’t more testers in management positions. Why do CIOs and senior IT management put people with non-testing backgrounds in charge of such an important function? In this talk, Keith will attempt to answer those questions through the profiles of CIOs he has worked with and the approach he has taken to tell the testing story. He will offer his opinion on why more testers aren’t in management positions and what can be done about it. So, put your bias aside and join Keith to take a hard look at what’s working (and what’s not) in software testing management, the culture of skilled testing, and how to join the ranks of management without losing your soul.
Quality assurance testing and user acceptance testing have obvious similarities, yet each has unique objectives. Ideally, QA and UAT are performed by different teams at different intervals in the project timeline. QA analysts are the SMEs in defect detection, removal, and testing process governance, while the UAT team are the functional SMEs in their respective business area but could benefit from QA’s guidance to make their testing a success as well as a well-defined repeatable process. This presentation will show how QA can effectively engage with the UAT team and project stakeholders before and during the UAT test event to maximize the effectiveness and efficiencies of both teams. Join Elizabeth to learn about setting expectations and avoiding assumptions, engaging UAT participants to define a repeatable process, and creating effective metrics and reporting to help the UAT team determine if their testing was a success.
Test Data Management (TDM) is a significant proportion of overall test effort. In recent years, the need for reliable, accurate, and complete test data has increased due to the increased complexity of systems and their underlying technical landscape. This includes the proliferation of web services and other interfaces with loosely coupled systems. Reduced cycle times for testing driven by agile approaches and business demand for change have only added to the problem. For many software systems having the right test data is critical to obtaining sufficient test coverage. In this presentation, Bernd and Paul will review some of the key challenges of TDM, outline a standard and holistic approach, discuss the role of a test data architect, and review some of the industry tools available to support the process.