patterns & practices: Performance Testing Guidance

UPDATED WIKI: Home

rboucher — Wed, 30 Apr 2008 20:46:47 GMT

NOTE: If you like this guide, check out our Windows Communication Foundation Security Guide, currently in development.

patterns & practices Performance Testing Guidance Project

Welcome to the patterns & practices Performance Testing Guidance project Community site. The purpose of this project is to build some insightful and practical guidance around doing performance testing and using Visual Studio 2005. It's a collaborative effort between industry experts, Microsoft ACE, patterns & practices, Premier, and VSTS team members. - J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, Dennis Rea

Usage Scenarios

Improve efficiency and effectiveness of your performance testing
Learn expert techniques performance testing
Apply expert techniques for performance testing using Visual Studio Team System

What's New

Guides

Performance Testing Guidance for Web Applications

Views

Topics

Knowledge Base

Videos

Video Index

Scenario Frames

We use Scenario Frames to frame out the problem space and map out the guidance opportunities, as well as measure effectiveness.

Scenario Frames

Reference

Glossary

Community

Guidance Share

Feedback

Send mail to PerfTest@microsoft.com
Post comments on the Discussions page.

Team

Core Team: J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, Dennis Rea
Contributors and Reviewers

Related Projects

UPDATED WIKI: Home

mycodeplexuser — Fri, 11 Apr 2008 17:43:24 GMT

patterns & practices Performance Testing Guidance Project

Usage Scenarios

Improve efficiency and effectiveness of your performance testing
Learn expert techniques performance testing
Apply expert techniques for performance testing using Visual Studio Team System

What's New

Guides

Performance Testing Guidance for Web Applications

Views

Topics

Knowledge Base

Videos

Video Index

Scenario Frames

We use Scenario Frames to frame out the problem space and map out the guidance opportunities, as well as measure effectiveness.

Scenario Frames

Reference

Glossary

Community

Guidance Share

Feedback

Send mail to PerfTest@microsoft.com
Post comments on the Discussions page.

Team

Core Team: J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, Dennis Rea
Contributors and Reviewers

Related Projects

UPDATED WIKI: Home

mycodeplexuser — Sun, 30 Mar 2008 19:33:51 GMT

patterns & practices Performance Testing Guidance Project

Usage Scenarios

Improve efficiency and effectiveness of your performance testing
Learn expert techniques performance testing
Apply expert techniques for performance testing using Visual Studio Team System

What's New

Guides

Performance Testing Guidance for Web Applications

Views

Topics

Knowledge Base

Videos

Video Index

Scenario Frames

We use Scenario Frames to frame out the problem space and map out the guidance opportunities, as well as measure effectiveness.

Scenario Frames

Reference

Glossary

Community

Guidance Share

Feedback

Send mail to PerfTest@microsoft.com
Post comments on the Discussions page.

Team

Core Team: J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, Dennis Rea
Contributors and Reviewers

Related Projects

UPDATED WIKI: Home

mycodeplexuser — Sun, 30 Mar 2008 19:32:55 GMT

patterns & practices Performance Testing Guidance Project

Usage Scenarios

Improve efficiency and effectiveness of your performance testing
Learn expert techniques performance testing
Apply expert techniques for performance testing using Visual Studio Team System

What's New

Guides

Performance Testing Guidance for Web Applications

Views

Topics

Knowledge Base

Videos

Video Index

Scenario Frames

We use Scenario Frames to frame out the problem space and map out the guidance opportunities, as well as measure effectiveness.

Scenario Frames

Reference

Glossary

Community

Guidance Share

Feedback

Send mail to PerfTest@microsoft.com
Post comments on the Discussions page.

Team

Core Team: J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, Dennis Rea
Contributors and Reviewers

Related Projects

Visual Studio 2005 Team System Guidance

UPDATED WIKI: Home

mycodeplexuser — Sun, 03 Feb 2008 02:25:39 GMT

patterns & practices Performance Testing Guidance Project

Usage Scenarios

Improve efficiency and effectiveness of your performance testing
Learn expert techniques performance testing
Apply expert techniques for performance testing using Visual Studio Team System

What's New

Guides

Performance Testing Guidance for Web Applications

Views

Topics

Knowledge Base

Videos

Video Index

Scenario Frames

We use Scenario Frames to frame out the problem space and map out the guidance opportunities, as well as measure effectiveness.

Scenario Frames

Reference

Glossary

Community

Guidance Share

Feedback

Send mail to PerfTest@microsoft.com
Post comments on the Discussions page.

Team

Core Team: J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, Dennis Rea
Contributors and Reviewers

Related Projects

Visual Studio 2005 Team System Guidance

UPDATED WIKI: Home

mycodeplexuser — Sun, 03 Feb 2008 02:25:21 GMT

patterns & practices Performance Testing Guidance Project

Usage Scenarios

Improve efficiency and effectiveness of your performance testing
Learn expert techniques performance testing
Apply expert techniques for performance testing using Visual Studio Team System

What's New

Guides

Performance Testing Guidance for Web Applications

Views

Topics

Knowledge Base

Videos

Video Index

Scenario Frames

We use Scenario Frames to frame out the problem space and map out the guidance opportunities, as well as measure effectiveness.

Scenario Frames

Reference

Glossary

Community

Guidance Share

Feedback

Send mail to PerfTest@microsoft.com
Post comments on the Discussions page.

Team

Core Team: J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, Dennis Rea
Contributors and Reviewers

Related Projects

Visual Studio 2005 Team System Guidance

UPDATED WIKI: Performance Testing Scenario Frame

prashantbansode — Tue, 30 Oct 2007 18:23:38 GMT

Performance Testing Scenarios Frame (Performance Testing Activity Patterns)

- J.D. Meier, Carlos Farre, Prashant Bansode

Personas

Performance Tester
System Administrator
Business analysts
Stakeholder
Developer
Architect
Development Lead
Tester

Activities/Tasks

Plan / design
Install / Configure
Script
Execute
Analyze
Report

Plan / Design

Scenario	Persona (s)	Priority
Identify and collect information on physical deployment.	Performance Tester
Identify and collect information on system architecture.	Performance Tester
Identify and collect information on key scenarios.	Performance Tester
Identify and collect information on batch processes.	Performance Tester
Identify and collect security requirements for tests.	Performance Tester
Identify goals and constraints for performance critical scenarios.	Business Analyst ; Architect; Performance Tester
Determine if you need tests to investigate or validate/verify.	Performance Tester
Determine if you need end-to-end or component level tests.	Performance Tester
Determine if you need tests for times or resource usage.	Performance Tester
Determine if you need tests for stressful or under-anticipated condition.	Performance Tester
Construct a workload based on frequency.	Performance Tester
Construct a workload based on performance intensity.	Performance Tester
Construct a workload based on legal implications.	Performance Tester
Construct a workload based on areas of technical risks.	Performance Tester
Construct a workload based on stakeholder mandate.	Performance Tester
Identify or collect information on user activities.	Performance Tester
Identify or collect activities within a user session.	Performance Tester
Identify or collect user experience patterns.	Performance Tester
Identify or collect the key performance goals and constraints for the user experience.	Performance Tester
Identify or collect the key performance goals and constraints for the business goals.	Performance Tester
Identify or collect the key performance goals and constraints for the technical goals.	Performance Tester
Identify or collect the key performance goals and constraints for compliance or industry goals.	Performance Tester
Identify or collect resource utilization goals (processor, network, disk and memory.)	Performance Tester
Identify response time goals.	Performance Tester
Identify throughput goals (e.g. “orders within a time window”)	Business Analyst; Architect; Performance Tester
If capacity testing is in scope, identify or collect information around future growth requirements.	Business Analyst; Architect; Performance Tester

Install / Configure

Scenario	Persona (s)	Priority
Prepare the testing servers and clients.	System Admin; Performance Tester
Prepare the testing network.	System Admin; Performance Tester
Prepare the test infrastructure.	Performance Tester
Validate the test environment.	Performance Tester
Deploy the load test infrastructure.	Performance Tester
Troubleshoot the load test infrastructure.	Performance Tester
Verify that the load is being distributed among the load balancers.	Performance Tester
Install the system under test.	Performance Tester
Configure the system under test.	Performance Tester
Validate the tests.	Performance Tester

Script

Scenario	Persona (s)	Priority
Implement the test design considering functionality.	Performance Tester
Implement the test design considering input.	Performance Tester
Implement the test design considering navigation.	Performance Tester
Implement the test design considering data.	Performance Tester
Implement the test design consideriong human variability.	Performance Tester
Implement the test design considering abandonment.	Performance Tester
Implement the test design considering ramping and marching.	Performance Tester
Implement the test design considering ramping and marching.	Performance Tester
Implement the test design considering maintainability.	Performance Tester
Identify the amount and type of data the application.	Performance Tester
Identify the amount and type of data the test should consume.	Performance Tester
Identify data to be validated from Web responses.	Performance Tester
Identify the dynamic data to be extracted from Web responses.	Performance Tester
Identify the user credentials data to be used in authentication scenarios.	Performance Tester
Create Web test scripts to simulate a user scenario.	Performance Tester
Create Web test scripts with goals for page load times.	Performance Tester
Create Web test scripts to validate content from Web responses.	Performance Tester
Create Web test scripts to handle dynamic data from Web responses.	Performance Tester
Create Web test scripts to simulate forms-based authentication.	Performance Tester
Create Web test scripts to simulate certificate authentication.	Performance Tester
Create Web test scripts to simulate windows authentication.	Performance Tester
Create Web services test scripts to simulate a user scenario.	Performance Tester
Create Web services test scripts with goals for page load times.	Performance Tester
Create Web services test scripts to simulate forms-based authentication.	Performance Tester
Create Web services test scripts to simulate certificate authentication.	Performance Tester
Create Web Services test scripts to simulate windows authentication.	Performance Tester

Execute

Scenario	Persona (s)	Priority
While executing tests, evaluate results.	Performance Tester
While executing tests, identify trends.	Performance Tester
While exeucting tests, identify variations around a central value (oscillation.)	Performance Tester
While executing tests, identify the baseline.	Performance Tester
While executing tests, incrementally ramp the user load.	Performance Tester
While executing tests, explore unanticipated scenarios, workloads and conditions.	Performance Tester
While executing tests, tune the configuration against goals and constraints.	Performance Tester
Implement the test design considering ramping and marching.	Performance Tester
While executing tests, exploit potential vulnerabilities and performance threats.	Performance Tester
Perform an application performance walkthrough.	Performance Tester
Analyze first-page load time to evaluate client-side performance.	Performance Tester
Analyze second-page load time to evaluate client-side performance.	Performance Tester
Create a baseline for a single user scenario.	Performance Tester
Create baselines for backend / batch process scenarios.	Performance Tester
Perform a manual execution of the scenario to verify the user interface.	Performance Tester
Analzye code coverage for key scenarios.	Performance Tester
Perform network analysis for a single user.	Performance Tester
Troubleshoot missing parameters.	Performance Tester
Troubleshoot application errors.	Performance Tester
Create a load test to simulate a percentage of users to a particular activity	Performance Tester
Create a load test to simulate a number of users for a particular activity.	Performance Tester
Run a load test to complete a certain number of transactions.	Performance Tester
Run a load test to simulate a controlled number of transactions within a certain time limit.	Performance Tester
Create a load test to simulate user abandonment.	Performance Tester
Simulate different browsers during load test execution.	Performance Tester
Troubleshoot a load test.	Performance Tester
Determine how much load can be generated before the load generator is the bottleneck.	Performance Tester
Run a load test until reaching a processor utilization value threshold.	Performance Tester
Run a load test until reaching a network utilization value threshold.	Performance Tester
Run a load test until you reach a request per second value threshold.	Performance Tester
Run a load test with performance metrics and threshold violation limits.	Performance Tester
Identify key scenarios that need to be stress tested.	Performance Tester
Create tests to simulate synchronization and race issues.	Performance Tester
Create tests to timulate memory leaks.	Performance Tester
Create tests to simulate loss of data during network congestion.	Performance Tester
Create a stress test a Web application.	Performance Tester
Create a stress test a Web service.	Performance Tester
Create a stress test OLAP database scenarios.	Performance Tester
Create a stress test OLTP database scenarios.	Performance Tester

Analyze

Scenario	Persona (s)	Priority
During analysis, consider configurations.	Performance Tester
During analysis, consider significance and repeatability.	Performance Tester
During analysis, consider trends.	Performance Tester
During analysis, consider outliers.	Performance Tester
During analysis, consider patterns.	Performance Tester
During analysis, consider compliance.	Performance Tester
During analysis, consider accuracy	Performance Tester
During analysis, consider resource and times.	Performance Tester
During analysis, consider errors and functionality.	Performance Tester

Report

Scenario	Persona (s)	Priority
Create a report that is timely.	Performance Tester
Create a report that is relevant.	Performance Tester
Create a report that is audience specific.	Performance Tester
Create a report that is visual.	Performance Tester
Create a report that is intuitive.	Performance Tester
During analysis, consider compliance.	Performance Tester
Create a report that is supported.	Performance Tester
Analzye performance test results after test runs.	Performance Tester
Summarize results across test runs.	Performance Tester
Share performance results across the team and stakeholders.	Performance Tester
Report degradation curves.	Performance Tester
Report abandonment rate.	Performance Tester
Report averages and percentiles of page response times.	Performance Tester
Report trending analysis using multipile load levels.	Performance Tester
Customize reports using the data store.	Business Analyst; Performance Tester

UPDATED WIKI: Performance Testing Scenario Frame

prashantbansode — Tue, 30 Oct 2007 18:11:22 GMT

Performance Testing Scenarios Frame (Performance Testing Activity Patterns)

Personas

Performance Tester
System Administrator
Business analysts
Stakeholder
Developer
Architect
Development Lead
Tester

Activities/Tasks

Plan / design
Install / Configure
Script
Execute
Analyze
Report

Plan / Design

Scenario	Persona (s)	Priority
Identify and collect information on physical deployment.	Performance Tester
Identify and collect information on system architecture.	Performance Tester
Identify and collect information on key scenarios.	Performance Tester
Identify and collect information on batch processes.	Performance Tester
Identify and collect security requirements for tests.	Performance Tester
Identify goals and constraints for performance critical scenarios.	Business Analyst ; Architect; Performance Tester
Determine if you need tests to investigate or validate/verify.	Performance Tester
Determine if you need end-to-end or component level tests.	Performance Tester
Determine if you need tests for times or resource usage.	Performance Tester
Determine if you need tests for stressful or under-anticipated condition.	Performance Tester
Construct a workload based on frequency.	Performance Tester
Construct a workload based on performance intensity.	Performance Tester
Construct a workload based on legal implications.	Performance Tester
Construct a workload based on areas of technical risks.	Performance Tester
Construct a workload based on stakeholder mandate.	Performance Tester
Identify or collect information on user activities.	Performance Tester
Identify or collect activities within a user session.	Performance Tester
Identify or collect user experience patterns.	Performance Tester
Identify or collect the key performance goals and constraints for the user experience.	Performance Tester
Identify or collect the key performance goals and constraints for the business goals.	Performance Tester
Identify or collect the key performance goals and constraints for the technical goals.	Performance Tester
Identify or collect the key performance goals and constraints for compliance or industry goals.	Performance Tester
Identify or collect resource utilization goals (processor, network, disk and memory.)	Performance Tester
Identify response time goals.	Performance Tester
Identify throughput goals (e.g. “orders within a time window”)	Business Analyst; Architect; Performance Tester
If capacity testing is in scope, identify or collect information around future growth requirements.	Business Analyst; Architect; Performance Tester

Install / Configure

Scenario	Persona (s)	Priority
Prepare the testing servers and clients.	System Admin; Performance Tester
Prepare the testing network.	System Admin; Performance Tester
Prepare the test infrastructure.	Performance Tester
Validate the test environment.	Performance Tester
Deploy the load test infrastructure.	Performance Tester
Troubleshoot the load test infrastructure.	Performance Tester
Verify that the load is being distributed among the load balancers.	Performance Tester
Install the system under test.	Performance Tester
Configure the system under test.	Performance Tester
Validate the tests.	Performance Tester

Script

Scenario	Persona (s)	Priority
Implement the test design considering functionality.	Performance Tester
Implement the test design considering input.	Performance Tester
Implement the test design considering navigation.	Performance Tester
Implement the test design considering data.	Performance Tester
Implement the test design consideriong human variability.	Performance Tester
Implement the test design considering abandonment.	Performance Tester
Implement the test design considering ramping and marching.	Performance Tester
Implement the test design considering ramping and marching.	Performance Tester
Implement the test design considering maintainability.	Performance Tester
Identify the amount and type of data the application.	Performance Tester
Identify the amount and type of data the test should consume.	Performance Tester
Identify data to be validated from Web responses.	Performance Tester
Identify the dynamic data to be extracted from Web responses.	Performance Tester
Identify the user credentials data to be used in authentication scenarios.	Performance Tester
Create Web test scripts to simulate a user scenario.	Performance Tester
Create Web test scripts with goals for page load times.	Performance Tester
Create Web test scripts to validate content from Web responses.	Performance Tester
Create Web test scripts to handle dynamic data from Web responses.	Performance Tester
Create Web test scripts to simulate forms-based authentication.	Performance Tester
Create Web test scripts to simulate certificate authentication.	Performance Tester
Create Web test scripts to simulate windows authentication.	Performance Tester
Create Web services test scripts to simulate a user scenario.	Performance Tester
Create Web services test scripts with goals for page load times.	Performance Tester
Create Web services test scripts to simulate forms-based authentication.	Performance Tester
Create Web services test scripts to simulate certificate authentication.	Performance Tester
Create Web Services test scripts to simulate windows authentication.	Performance Tester

Execute

Scenario	Persona (s)	Priority
While executing tests, evaluate results.	Performance Tester
While executing tests, identify trends.	Performance Tester
While exeucting tests, identify variations around a central value (oscillation.)	Performance Tester

|While executing tests, identify the baseline. |Performance Tester ||

While executing tests, incrementally ramp the user load.	Performance Tester
While executing tests, explore unanticipated scenarios, workloads and conditions.	Performance Tester
While executing tests, tune the configuration against goals and constraints.	Performance Tester
Implement the test design considering ramping and marching.	Performance Tester
While executing tests, exploit potential vulnerabilities and performance threats.	Performance Tester
Perform an application performance walkthrough.	Performance Tester
Analyze first-page load time to evaluate client-side performance.	Performance Tester
Analyze second-page load time to evaluate client-side performance.	Performance Tester
Create a baseline for a single user scenario.	Performance Tester
Create baselines for backend / batch process scenarios.	Performance Tester
Perform a manual execution of the scenario to verify the user interface.	Performance Tester
Analzye code coverage for key scenarios.	Performance Tester
Perform network analysis for a single user.	Performance Tester
Troubleshoot missing parameters.	Performance Tester
Troubleshoot application errors.	Performance Tester
Create a load test to simulate a percentage of users to a particular activity	Performance Tester
Create a load test to simulate a number of users for a particular activity.	Performance Tester
Run a load test to complete a certain number of transactions.	Performance Tester
Run a load test to simulate a controlled number of transactions within a certain time limit.	Performance Tester
Create a load test to simulate user abandonment.	Performance Tester
Simulate different browsers during load test execution.	Performance Tester
Troubleshoot a load test.	Performance Tester
Determine how much load can be generated before the load generator is the bottleneck.	Performance Tester
Run a load test until reaching a processor utilization value threshold.	Performance Tester
Run a load test until reaching a network utilization value threshold.	Performance Tester
Run a load test until you reach a request per second value threshold.	Performance Tester
Run a load test with performance metrics and threshold violation limits.	Performance Tester
Identify key scenarios that need to be stress tested.	Performance Tester
Create tests to simulate synchronization and race issues.	Performance Tester
Create tests to timulate memory leaks.	Performance Tester
Create tests to simulate loss of data during network congestion.	Performance Tester
Create a stress test a Web application.	Performance Tester
Create a stress test a Web service.	Performance Tester
Create a stress test OLAP database scenarios.	Performance Tester
Create a stress test OLTP database scenarios.	Performance Tester

Analyze

Scenario	Persona (s)	Priority
During analysis, consider configurations.	Performance Tester
During analysis, consider significance and repeatability.	Performance Tester
During analysis, consider trends.	Performance Tester
During analysis, consider outliers.	Performance Tester
During analysis, consider patterns.	Performance Tester
During analysis, consider compliance.	Performance Tester
During analysis, consider accuracy	Performance Tester
During analysis, consider resource and times.	Performance Tester
During analysis, consider errors and functionality.	Performance Tester

Report

Scenario	Persona (s)	Priority
Create a report that is timely.	Performance Tester
Create a report that is relevant.	Performance Tester
Create a report that is audience specific.	Performance Tester
Create a report that is visual.	Performance Tester
Create a report that is intuitive.	Performance Tester
During analysis, consider compliance.	Performance Tester
Create a report that is supported.	Performance Tester
Analzye performance test results after test runs.	Performance Tester
Summarize results across test runs.	Performance Tester
Share performance results across the team and stakeholders.	Performance Tester
Report degradation curves.	Performance Tester
Report abandonment rate.	Performance Tester
Report averages and percentiles of page response times.	Performance Tester
Report trending analysis using multipile load levels.	Performance Tester
Customize reports using the data store.	Business Analyst; Performance Tester

UPDATED WIKI: Performance Testing Scenario Frame

prashantbansode — Tue, 30 Oct 2007 18:07:24 GMT

Performance Testing Scenarios Frame (Performance Testing Activity Patterns)

Personas

Performance Tester
System Administrator
Business analysts
Stakeholder
Developer
Architect
Development Lead
Tester

Activities/Tasks

Plan / design
Install / Configure
Script
Execute
Analyze
Report

Plan / Design

Scenario

Persona (s)

Priority

|Identify and collect information on physical deployment. |Performance Tester ||
|Identify and collect information on system architecture. |Performance Tester ||
|Identify and collect information on key scenarios. |Performance Tester ||
|Identify and collect information on batch processes. |Performance Tester ||
|Identify and collect security requirements for tests. |Performance Tester ||
|Identify goals and constraints for performance critical scenarios. |Business Analyst ; Architect; Performance Tester ||
|Determine if you need tests to investigate or validate/verify. |Performance Tester ||
|Determine if you need end-to-end or component level tests. |Performance Tester ||
|Determine if you need tests for times or resource usage. |Performance Tester ||
|Determine if you need tests for stressful or under-anticipated condition. |Performance Tester ||
|Construct a workload based on frequency. |Performance Tester ||
|Construct a workload based on performance intensity. |Performance Tester ||
|Construct a workload based on legal implications. |Performance Tester ||
|Construct a workload based on areas of technical risks. |Performance Tester ||
|Construct a workload based on stakeholder mandate. |Performance Tester ||
|Identify or collect information on user activities. |Performance Tester ||
|Identify or collect activities within a user session. |Performance Tester ||
|Identify or collect user experience patterns. |Performance Tester ||
|Identify or collect the key performance goals and constraints for the user experience. |Performance Tester ||
|Identify or collect the key performance goals and constraints for the business goals. |Performance Tester ||
|Identify or collect the key performance goals and constraints for the technical goals. |Performance Tester ||
|Identify or collect the key performance goals and constraints for compliance or industry goals. |Performance Tester ||
|Identify or collect resource utilization goals (processor, network, disk and memory.) |Performance Tester ||
|Identify response time goals. |Performance Tester ||
|Identify throughput goals (e.g. “orders within a time window”) |Business Analyst; Architect; Performance Tester ||
|If capacity testing is in scope, identify or collect information around future growth requirements. |Business Analyst; Architect; Performance Tester ||

Install / Configure

Scenario

Persona (s)

Priority

|Prepare the testing servers and clients. |System Admin; Performance Tester ||
|Prepare the testing network. |System Admin; Performance Tester ||
|Prepare the test infrastructure. |Performance Tester ||
|Validate the test environment. |Performance Tester ||
|Deploy the load test infrastructure. |Performance Tester ||
|Troubleshoot the load test infrastructure. |Performance Tester ||
|Verify that the load is being distributed among the load balancers. |Performance Tester ||
|Install the system under test. |Performance Tester ||
|Configure the system under test. |Performance Tester ||
|Validate the tests. |Performance Tester ||

Script

Scenario

Persona (s)

Priority

|Implement the test design considering functionality. |Performance Tester ||
|Implement the test design considering input. |Performance Tester ||
|Implement the test design considering navigation. |Performance Tester ||
|Implement the test design considering data. |Performance Tester ||
|Implement the test design consideriong human variability. |Performance Tester ||
|Implement the test design considering abandonment. |Performance Tester ||
|Implement the test design considering ramping and marching. |Performance Tester ||
|Implement the test design considering ramping and marching. |Performance Tester ||
|Implement the test design considering maintainability. |Performance Tester ||
|Identify the amount and type of data the application. |Performance Tester ||
|Identify the amount and type of data the test should consume. |Performance Tester ||
|Identify data to be validated from Web responses. |Performance Tester ||
|Identify the dynamic data to be extracted from Web responses. |Performance Tester ||
|Identify the user credentials data to be used in authentication scenarios. |Performance Tester ||
|Create Web test scripts to simulate a user scenario. |Performance Tester ||
|Create Web test scripts with goals for page load times. |Performance Tester ||
|Create Web test scripts to validate content from Web responses. |Performance Tester ||
|Create Web test scripts to handle dynamic data from Web responses. |Performance Tester ||
|Create Web test scripts to simulate forms-based authentication. |Performance Tester ||
|Create Web test scripts to simulate certificate authentication. |Performance Tester ||
|Create Web test scripts to simulate windows authentication. |Performance Tester ||
|Create Web services test scripts to simulate a user scenario. |Performance Tester ||
|Create Web services test scripts with goals for page load times. |Performance Tester ||
|Create Web services test scripts to simulate forms-based authentication. |Performance Tester ||
|Create Web services test scripts to simulate certificate authentication. |Performance Tester ||
|Create Web Services test scripts to simulate windows authentication. |Performance Tester ||

Execute

Scenario

Persona (s)

Priority

|While executing tests, evaluate results. |Performance Tester ||
|While executing tests, identify trends. |Performance Tester ||
|While exeucting tests, identify variations around a central value (oscillation.) |Performance Tester ||
|While executing tests, identify the baseline. |Performance Tester ||
|While executing tests, incrementally ramp the user load. |Performance Tester ||
|While executing tests, explore unanticipated scenarios, workloads and conditions. |Performance Tester ||
|While executing tests, tune the configuration against goals and constraints. |Performance Tester ||
|Implement the test design considering ramping and marching. |Performance Tester ||
|While executing tests, exploit potential vulnerabilities and performance threats. |Performance Tester ||
|Perform an application performance walkthrough. |Performance Tester ||
|Analyze first-page load time to evaluate client-side performance. |Performance Tester ||
|Analyze second-page load time to evaluate client-side performance. |Performance Tester ||
|Create a baseline for a single user scenario. |Performance Tester ||
|Create baselines for backend / batch process scenarios. |Performance Tester ||
|Perform a manual execution of the scenario to verify the user interface. |Performance Tester ||
|Analzye code coverage for key scenarios. |Performance Tester ||
|Perform network analysis for a single user. |Performance Tester ||
|Troubleshoot missing parameters. |Performance Tester ||
|Troubleshoot application errors. |Performance Tester ||
|Create a load test to simulate a percentage of users to a particular activity |Performance Tester ||
|Create a load test to simulate a number of users for a particular activity. |Performance Tester ||
|Run a load test to complete a certain number of transactions. |Performance Tester ||
|Run a load test to simulate a controlled number of transactions within a certain time limit. |Performance Tester ||
|Create a load test to simulate user abandonment. |Performance Tester ||
|Simulate different browsers during load test execution. |Performance Tester ||
|Troubleshoot a load test. |Performance Tester ||
|Determine how much load can be generated before the load generator is the bottleneck. |Performance Tester ||
|Run a load test until reaching a processor utilization value threshold. |Performance Tester ||
|Run a load test until reaching a network utilization value threshold. |Performance Tester ||
|Run a load test until you reach a request per second value threshold. |Performance Tester ||
|Run a load test with performance metrics and threshold violation limits. |Performance Tester ||
|Identify key scenarios that need to be stress tested. |Performance Tester ||
|Create tests to simulate synchronization and race issues. |Performance Tester ||
|Create tests to timulate memory leaks. |Performance Tester ||
|Create tests to simulate loss of data during network congestion. |Performance Tester ||
|Create a stress test a Web application. |Performance Tester ||
|Create a stress test a Web service. |Performance Tester ||
|Create a stress test OLAP database scenarios. |Performance Tester ||
|Create a stress test OLTP database scenarios. |Performance Tester ||

Analyze

Scenario

Persona (s)

Priority

|During analysis, consider configurations. |Performance Tester ||
|During analysis, consider significance and repeatability. |Performance Tester ||
|During analysis, consider trends. |Performance Tester ||
|During analysis, consider outliers. |Performance Tester ||
|During analysis, consider patterns. |Performance Tester ||
|During analysis, consider compliance. |Performance Tester ||
|During analysis, consider accuracy |Performance Tester ||
|During analysis, consider resource and times. |Performance Tester ||
|During analysis, consider errors and functionality. |Performance Tester ||

Report

Scenario

Persona (s)

Priority

|Create a report that is timely. |Performance Tester ||
|Create a report that is relevant. |Performance Tester ||
|Create a report that is audience specific. |Performance Tester ||
|Create a report that is visual. |Performance Tester ||
|Create a report that is intuitive. |Performance Tester ||
|During analysis, consider compliance. |Performance Tester ||
|Create a report that is supported. |Performance Tester ||
|Analzye performance test results after test runs. |Performance Tester ||
|Summarize results across test runs. |Performance Tester ||
|Share performance results across the team and stakeholders. |Performance Tester ||
|Report degradation curves. |Performance Tester ||
|Report abandonment rate. |Performance Tester ||
|Report averages and percentiles of page response times. |Performance Tester ||
|Report trending analysis using multipile load levels. |Performance Tester ||
|Customize reports using the data store. |Business Analyst; Performance Tester ||

UPDATED WIKI: Performance Testing Scenario Frame

prashantbansode — Tue, 30 Oct 2007 18:05:32 GMT

Performance Testing Scenarios Frame (Performance Testing Activity Patterns)

Personas

Performance Tester
System Administrator
Business analysts
Stakeholder
Developer
Architect
Development Lead
Tester

Activities/Tasks

Plan / design
Install / Configure
Script
Execute
Analyze
Report

Plan / Design

Scenario	Persona (s)	Priority

Install / Configure

Scenario	Persona (s)	Priority

Script

Scenario	Persona (s)	Priority

Execute

Scenario	Persona (s)	Priority

Analyze

Scenario	Persona (s)	Priority

Report

Scenario	Persona (s)	Priority

UPDATED WIKI: Scenario Frames

prashantbansode — Tue, 30 Oct 2007 18:03:47 GMT

Scenario Frames

- J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber

We organize our scenarios for key problem areas into a frame. We use the scenarios to figure out where customers need more help, and to test how well the guidance, tools, and platform address the problems.

Performance Testing Scenario Frame