This project is read-only.
Note that the information on this page is the BETA 1 of a guide that is now released. See http://www.codeplex.com/AppArchGuide for the latest PDF and HTML content.

Chapter 6 – Quality Attributes

- J.D. Meier, Alex Homer, David Hill, Jason Taylor, Prashant Bansode, Lonnie Wall, Rob Boucher Jr, Akshay Bogawat

Objectives

  • Learn the key quality attributes, and how they apply to applications.
  • Learn how to identify the quality attributes that should be considered along with the Key Issues, decisions, and techniques associated with each one.

Overview

Quality attributes are the cross cutting concerns that affect runtime performance, system design and user experience. Quality attributes are important for the overall usability, performance, reliability, and security of your application.

Quality Attribute Frame

The following table frames out the quality attributes that will be covered in this chapter. You can use this table to understand what each of the quality attributes means in terms of your application design.

Category Description
Availability The application should be available and operating at all times specified in an SLA, and when users will expect to be able to access it. Availability includes both software and infrastructure concerns.
Conceptual Integrity The application should avoid concrete dependencies on other systems and applications where possible, and provide features that allow it and any data it holds to be upgraded, migrated, and re-factored.
Flexibility The application should be configurable to allow use in varying environments and situations, and to cope with changes to business policies and rules.
Interoperability The application should interoperate with other associated applications, including sending and receiving data, using services, and cope with refactoring and upgrades to other applications.
Maintainability The application should be designed and constructed in such a way as to allow changes to components, services, features, and interfaces as may be required when adding or changing the functionality, fixing bugs, and meeting new business requirements.
Manageability The application should be easy to manage, by exposing sufficient and useful instrumentation for use in monitoring systems and for debugging and performance tuning.
Performance The application should perform to requirements and expectations, making efficient use of resources and infrastructure.
Reliability The application should continue to work correctly, and work without interruption where possible, when encountering issues such as invalid input data, survivable infrastructure faults, varying network bandwidth and availability, and other external factors.
Reusability The components and services in the application should be designed in such a way as to be suitable for use in other applications and scenarios whenever possible, and to minimize duplication of components in the application.
Scalability The application should support increasing usage and load by working correctly when scaled up to more powerful hardware, and when scaled out in a Web or Application Farm.
Security The application must prevent access by unauthorized users, and protect data and resources from unauthorized users and all types of malicious use.
Supportability The application must be easy for help desk staff and developers to provide support when it fails to work correctly, and be as intuitive as possible to minimize support requirements.
Testability The application must be designed in such a way that all individual components as well as entire subsystems can be testing using automated methods, test frameworks, and user-level testing.
Usability The application interfaces must be designed with the user and consumer in mind so that they are intuitive; can be localized and globalized; provide access to disabled users; and provide a good overall user experience.

Availability

The application should be available and operating at all times specified in a Service Level Agreement (SLA), and when users will expect to be able to access it. Availability includes both software and infrastructure concerns.

Key Issues

  • A physical tier such as the database server or application server can fail or become unresponsive, causing the whole system to fail.
  • Security vulnerabilities could allow Denial of Service (DoS) attacks, which will prevent access to the system.
  • Inappropriate use of resources can cause poor application performance. For example, resources acquired too early and held for too long cause resource starvation and inability to handle additional work.
  • Bugs or faults in the application can cause a system wide failure.
  • Frequent updates can reduce availability of the system.
  • A network fault causes the application to be unavailable.

Key Decisions

  • How to design for failovers related to different tiers in the system?
  • How to design for data storage failures, such as database failures or hard disk failures?
  • How to implement request queuing at a service interface layer to support input when the internal system is offline due to upgrades or faults?
  • How to decide if I need a geographically-separate redundant site to fail over to in the case of natural disasters such as earthquakes or tornados?
  • How to instrument the system for system health monitoring such as memory availability?

Key Techniques

  • Use Network Load Balancing (NLB) for web servers to distribute the load and prevent requests from being sent to a server that is down.
  • Use database clustering to improve the availability of database servers.
  • Use a Redundant Array of Independent Disks (RAID) to mitigate system failure in the event that a disk fails.
  • Use Reliable Sessions for service interfaces in order to implement guaranteed delivery of service requests.
  • Deploy the system at geographically separate sites and balance requests across all sites that are available. This is an example of advanced networking design.
  • Schedule planned system outages, such as maintenance activities, system upgrades, software patch installation, and data backup to have minimum impact on system availability.
  • To minimize security vulnerabilities, reduce the attack surface area, identify malicious behavior, use application instrumentation to expose unintended behavior, and implement comprehensive data validation.
  • Design clients with occasionally-connected capabilities, such as a Rich Client.

Conceptual Integrity

The application should avoid concrete dependencies on other systems and applications where possible, and provide features that allow it and any data it holds to be upgraded, migrated, and re-factored.

Key Issues

  • Designing the architecture of a new system.
  • Existing (legacy) system demands prevent both refactoring and progression towards a new platform or paradigm.

Key Decisions

  • How to design the system to reduce dependencies?
  • How to evolve the system without breaking clients?
  • How to create a migration path away from legacy technologies?
  • How to isolate applications from external dependencies?

Key Techniques

  • Use loose coupling and abstraction between layers to minimize dependencies.
  • Use abstraction within layers to support dynamic behavior and reduce dependencies.
  • Design interaction with external dependencies to handle instances when the external resource is not available.

Flexibility

The application should be configurable to allow use in varying environments and situations, and to cope with changes to business policies and rules.

Key Issues

  • The code base is large, unmanageable, and fragile.
  • Refactoring is burdensome due to regression requirements for a large and growing code base.
  • The existing code is over-complex.
  • The same logic is implemented in many different ways.

Key Decisions

  • How to handle dynamic business rules, such as changes based on authorization, data, or process?
  • How to handle dynamic UI, such as changes based on authorization, data, or process?
  • How to respond to changes in data and logic processing?
  • How to ensure that components and services have well defined responsibilities and relationships?

Key Techniques

  • Use an external source, such as a business rules engine, to handle dynamic business rules.
  • Use standard techniques to store users and roles, such as Active Directory or ASP.NET profiles.
  • Design systems as well-defined layers, or areas of concern, that delineate the User Interface, business processes, and data access functionality of the system.
  • Design components to be cohesive and with low coupling to maximize flexibility and allow simpler replacement and reusability.

Interoperability

The application should interoperate with other associated applications, including sending and receiving data, using services, and cope with refactoring and upgrades to other applications.

Key Issues

  • Interaction with external or legacy systems that use different data formats.
  • Boundary blurring, which allows artifacts from one layer, tier, or system to bleed into another.

Key Decisions

  • How to handle different data formats from external or legacy systems?
  • How to enable systems to interoperate, yet evolving separately or even be replaced?
  • How to isolate the participants by using service interfaces, layer super-types, and/or by mapping layers?

Key Techniques

  • Use orchestration with adaptors to connect with external or legacy systems and translate data between systems.
  • Use a canonical data model to handle interaction with a large number of different data formats.
  • Expose services using interfaces based on XML or standard types to support interoperability with other systems.
  • Design components to be cohesive and with low coupling to maximize flexibility and allow simpler replacement and reusability.

Maintainability

The application should be designed and constructed in such a way as to allow changes to components, services, features, and interfaces as may be required when adding or changing the functionality, fixing bugs, and meeting new business requirements

Key Issues

  • Excessive dependencies between components and layers prevent easy replacement, updates, and changes.
  • Use of direct communication prevents changes to the physical deployment of components and layers.
  • Reliance on custom implementations of features such as authentication and authorization prevents reuse and hampers maintenance.
  • Over-complex code makes maintenance and reuse difficult.
  • Mixing the implementation of cross-cutting concerns with application-specific components makes maintenance harder and reuse difficult.
  • Components are not cohesive, which makes them difficult to replace and causes unnecessary dependencies on child components.

Key Decisions

  • How to reduce dependencies between components and layers?
  • How to implement a pluggable architecture that allows easy upgrades and maintenance, and improved testing capabilities?
  • How to separate the functionality for cross-cutting concerns from application-specific code?
  • How to choose an appropriate communication model, format, and protocol?
  • How to create cohesive components?

Key Techniques

  • Design systems as well-defined layers, or areas of concern, that delineate the User Interface, business processes, and data access functionality of the system.
  • Design components to be cohesive and with low coupling to maximize flexibility and allow simpler replacement and reusability.
  • Design interfaces that allow the use of plug-in modules or adapters to maximize flexibility and extendibility.

Manageability

The application should be easy to manage, by exposing sufficient and useful instrumentation for use in monitoring systems and for debugging and performance tuning.

Key Issues

  • Lack of diagnostic information.
  • Lack of troubleshooting tools.
  • Lack of performance and scale metrics.
  • Lack of tracing ability.
  • Lack of health monitoring.

Key Decisions

  • How to discover the key types of failure?
  • How to enable the system behavior to change based on operational environment requirements, such as infrastructure or deployment changes?
  • How to enable the system behavior to change at run time based on system load, such as queuing requests and processing them when the system is available?
  • How to create a snapshot of the system's state to use for troubleshooting?
  • How to monitor aspects of the system's operation and health?
  • How to create custom instrumentation to provide detailed operational reports?
  • How to measure how many transactions are being processed per hour?
  • How to discover details of the requests sent to the system?

Key Techniques

  • Consider creating a health model that defines the significant state changes that can affect application performance, and use this to specify management instrumentation requirements.
  • Implement instrumentation, such as events and performance counters, that detects state changes and expose these changes through standard systems such as Event Logs, Trace files, or WMI.
  • Capture and report sufficient information about errors and state changes to enable accurate monitoring, debugging, and management.
  • Consider implementing fine-grained debugging instrumentation that can be enabled for debugging and disabled for normal operations to provide extra information.
  • Consider creating management packs that administrators can use in their monitoring environments to manage the application.
  • Consider creating probe and heartbeat methods that can extract working data or allow specialized monitoring of specific functions for debugging.
  • Consider logging and auditing information that may be useful for maintenance and debugging, such as request details or module outputs and calls to other systems and services.
  • Use configuration settings that change the way the application works, such as using a different service, failing over to another system, or accessing a spare or back-up system should the usual one fail.

Performance

The application should perform to requirements and expectations, making efficient use of resources and infrastructure.

Key Issues

  • Roundtrips to the data store for every user request increase the load on the data store.
  • Increased client response time, reduced throughput, and server resource over-utilization.
  • Increased memory consumption, resulting in reduced performance, cache misses, and increased data store access.
  • Processor and memory overhead increases due to the context switching and thread management overhead.
  • Lack of appropriate concurrency capabilities causes increased contention and reduced concurrency.
  • Poor choice of isolation levels results in contention, extended response times, timeouts, and deadlocks.
  • Mixing the logic for different areas of functionally, such as presentation and business logic, without clear and logical partitioning limits scalability options.
  • Increases in database server processing may cause reduced throughput.
  • Increases in network bandwidth consumption may cause delayed response times, and increased load for client and server systems.
  • Inefficient queries, or fetching all the data when only a portion is displayed, may incur unnecessary load on the database server, failure to meet performance objectives, and costs in excess of budget allocations.
  • Poor design can cause extra roundtrips to servers and expensive calls to methods when errors occur.
  • Using exceptions to control logic is expensive compared to returning an enumeration member or a Boolean value.
  • Poor design adds to the performance overhead and can conceal information unnecessarily.
  • Poor resource management can result in the creation of multiple instances of resources, with the corresponding connection overhead, and increase the response time of the application.
  • Failing to release, or delaying the release of shared resources such as connections leads to resource drain and limits scalability.
  • Retrieving large amounts of data from the resource over narrow bandwidth networks affects response time, and reduces the capacity to serve multiple concurrent users.
  • Poor resource management leads to resource shortages and increased memory consumption; both of these affect scalability.
  • Large numbers of clients can cause resource starvation and overload the server.
  • Components that hold state information can force affinity in a server farm, which reduces scalability options.

Key Decisions

  • How to determine what and where to cache?
  • How to communicate between layers?
  • How to determine effective choices for transport, boundaries, remote interface design, roundtrips, serialization and bandwidth?
  • How to handle concurrent user interactions?
  • How to make effective choices for transactions, locks, threading, and queuing?
  • How to structure the application?
  • How to achieve coupling, and high cohesion among components and layers?
  • How to determine effective data access techniques?
  • How to make effective choices for schema design, paging, hierarchies, indexes, amount of data to retrieve, and roundtrips?
  • How to handle data?
  • How to make effective choices for data structures; for example, choosing between arrays and collections?
  • How to handle exceptions?
  • How to make effective choices for catching, throwing, and handling exceptions?
  • How to manage resources?
  • How to effectively allocate, create, destroy, and pool application resources?
  • How to determine where to maintain state and what to store?
  • How to make effective choices for per-user and application-wide persistence techniques and locations?

Key Techniques

  • Choose the appropriate remote communication mechanism.
  • Design coarse-grained interfaces that require the minimum number of calls (preferably just one) to execute a specific task.
  • Minimize the amount of data sent over the network.
  • Batch work to reduce calls over the network.
  • Reduce transitions across boundaries.
  • Consider asynchronous communication.
  • Evaluate a stateful versus stateless design.
  • Treat threads as a shared resource.
  • Pool shared or scarce resources.
  • Acquire resources as late as possible, and release them as early as possible.

Reliability

The application should continue to work correctly, and work without interruption where possible, when encountering issues such as invalid input data, survivable infrastructure faults, varying network bandwidth and availability, and other external factors

Key Issues

  • System crashes.
  • System becomes unresponsive at times.
  • Output is inconsistent.
  • System fails due to non-availability of other externalities such as systems, networks, and databases.

Key Decisions

  • How to handle external systems that are unreliable?
  • How to handle replay and retry operations?
  • How to track the status of requests and jobs that are being processed?
  • How to audit requests and jobs?
  • How to detect failures and automatically initiate a failover?
  • How to redirect load under extreme circumstances?
  • How to take the system offline but still queue pending requests?

Key Techniques

  • Implement instrumentation, such as events and performance counters, that detects poor performance or failures of requests sent to external systems, and expose information through standard systems such as Event Logs, Trace files, or WMI.
  • Log performance and auditing information about calls made to other systems and services.
  • Consider implementing configuration settings that change the way the application works, such as using a different service, failing over to another system, or accessing a spare or back-up system should the usual one fail.
  • Consider implementing code that uses alternative systems when it detects a specific number of failed requests to an existing system.
  • Implement store-and-forward or cached message-based communication systems that allow requests to be stored when the target system is unavailable, and replayed when it is online.
  • Consider using Windows Message Queuing or BizTalk Server to provide a reliable once-only delivery mechanism for asynchronous requests.

Reusability

The components and services in the application should be designed in such a way as to be suitable for use in other applications and scenarios whenever possible, and to minimize duplication of components in the application.

Key Issues

  • Using different code or components to achieve the same result in different places.
  • Using multiple similar methods instead of parameters to implement tasks that vary slightly.
  • Using several systems to implement the same feature or function.

Key Decisions

  • How to reduce duplication of similar logic in multiple components?
  • How to reduce duplication of similar logic in multiple layers or sub-systems?
  • How to reuse functionality in another system?
  • How to share functionality across multiple systems?
  • How to share functionality across different sub-systems within an application?

Key Techniques

  • Examine the application design to identify cross-cutting concerns such as validation, logging, and authentication, and implement these functions as separate components.
  • Examine the application design to identify common functionality, and implement this in separate components that you can reuse.
  • Consider exposing functionality from components, layers, and subsystems through service interfaces that other layers and systems can use.
  • Consider using platform-agnostic data types and structures that can be accessed and understood on different platforms.

Scalability

The application should support increasing usage and load by working correctly when scaled up to more powerful hardware, and when scaled out in a Web or Application Farm

Key Issues

  • Applications cannot handle increasing load.
  • Users incur delays in response and longer completion times.
  • The system fails.
  • The system cannot queue excess work and process it during periods of reduced load.

Key Decisions

  • How to scale-up or scale-out an application?
  • How to scale the database?
  • How to scale the user interface?
  • How to take the system off-line, but still queue requests?
  • How to handle spikes in traffic and load?
  • How to decouple processing and queue requests in extreme circumstances?

Key Techniques

  • Avoid stateful components and subsystems where possible to reduce server affinity.
  • Consider locating layers on the same physical tier to reduce the number of servers required while maximizing load sharing and failover capabilities.
  • Consider implementing configuration settings that change the way the application works, such as using a different service, failing over to another system, or accessing a spare or back-up system should the usual one fail.
  • Consider implementing code that uses alternative systems when it detects a specific number of failed requests to an existing system.
  • Consider implementing code that uses alternative systems when it detects a pre-defined service load or number of pending requests to an existing system.
  • Implement store-and-forward or cached message-based communication systems that allow requests to be stored when the target system is unavailable, and replayed when it is online.
  • Consider partitioning data across more than one database server to maximize scale-up opportunities and allow flexible location of data subsets.

Security

The application must prevent access by unauthorized users, and protect data and resources from unauthorized users and all types of malicious use

Key Issues

  • Spoofing user identity.
  • Tampering with data.
  • Repudiation.
  • Information disclosure.
  • Denial of service.
  • Elevation of privilege

Key Decisions

  • How to protect from SQL injection?
  • How to protect from cross-site scripting?
  • How to audit failed logons?
  • How to audit across application tiers?
  • How to keep user input out of the control path?
  • How to avoid storing clear text credentials in configuration files?
  • How to avoid passing clear text credentials over the network?
  • How to avoid mixing personalization with authentication?
  • How to avoid reliance on a single gatekeeper?
  • How to avoid inadequate separation of privileges?
  • How to avoid insecure administration interfaces?
  • How to avoid insecure configuration stores?
  • How to avoid the use of custom cryptography?
  • How to avoid revealing too much information to the client?
  • How to access databases, file shares, and other network resources?
  • How to validate input from all sources, including cookies, query string parameters, HTTP headers, databases, and network resources?

Key Techniques

  • Validate input for length, range, format, and type.
  • Constrain and reject, input.
  • Sanitize output.
  • Do not store secrets in software.
  • Do not reveal sensitive system or application information.
  • Use application instrumentation to expose behavior that can be monitored.
  • Use the platform features to generate random numbers.
  • Avoid key management.
  • Partition the site by anonymous, identified, and authenticated users.
  • Reduce session timeouts.
  • Authenticate and authorize access to the session store.

Supportability

The application must be easy for help desk staff and developers to provide support when it fails to work correctly, and be as intuitive as possible to minimize support requirements.

Key Issues

  • Lack of diagnostic information.
  • Lack of troubleshooting tools.
  • Lack of performance and scale metrics.
  • Lack of tracing ability.
  • Lack of health monitoring

Key Decisions

  • How to monitor system activity?
  • How to monitor system performance?
  • How to implement tracing?
  • How to provide troubleshooting support?

Key Techniques

  • Consider a system monitoring application, such as Microsoft System Center.
  • Use performance counters to monitor system performance?
  • Enable tracing in web applications to troubleshoot errors.
  • Use common components to provide tracing support in code.
  • Use Aspect Oriented Programming (AOP) techniques to implement tracing.

Testability

The application must be designed in such a way that all individual components as well as entire subsystems can be testing using automated methods, test frameworks, and user-level testing.

Key Issues

  • Regulatory forces impose demands on testing and release dates.
  • Test cases are not documented, and testing is random and incomplete.
  • Testing does not focus on the appropriate components.
  • It is difficult to test and verify asynchronous components.
  • Complex applications with many processing permutations are not tested consistently.
  • Human testing does not cover enough of the application prior to release to prove it is ready.
  • Automated or granular testing cannot be performed because application has a monolithic design.
  • Testing cannot be performed properly because there is no suitable environment.
  • Lack of test automation systems.
  • Lack of test planning.
  • Poor test coverage.
  • Input inconsistencies; for the same input, the output is not same.
  • Output inconsistencies - output does not fully cover the output domain, even though all known variations of input are provided.

Key Decisions

  • How to ensure an early start to testing during the development lifecycle?
  • How to handle regression testing for each release?
  • How to automate user interaction tests?
  • How to handle test automation and detailed reporting for highly complex functionality, rules, or calculations?
  • How to address the concerns of regulatory agencies?
  • How to automated unit testing?
  • How to test asynchronous user interactions?
  • How to test asynchronous components?
  • How to test highly concurrent components and methods?
  • How to prove that a minimum amount of the code has been tested?
  • How to separately test each layer or tier?
  • How to manage multiple environments for development, test, and production?
  • How to instrument the code in order to obtain test data?
  • How to create a modular system to isolate testing effort?
  • How to measure code usage of code using code coverage tools?
  • How to selectively use parts of the system in order to construct test cases with more flexibility?
  • How to make it easy to specify and understand system inputs and outputs to facilitate the construction of test cases?
  • How to clearly define component and communication interfaces?

Key Techniques

  • Implement a Contract by Design strategy.
  • Use mock objects during testing.
  • Automate unit tests.
  • Construct simple, structured solutions.
  • Design systems to be modular to support testing.
  • Ensure that appropriate documentation exists for the system.
  • Provide instrumentation or implement probes for testing.
  • Provide mechanisms to debug output.
  • Design for low coupling between modules.
  • Provide ways to specify inputs easily.
  • Ensure that the output is understandable.

User Experience / Usability

The application interfaces must be designed with the user and consumer in mind so that they are intuitive; can be localized and globalized; provide access to disabled users; and provide a good overall user experience

Key Issues

  • Too much interaction (excessive number of "clicks") is required for a task.
  • There is an incorrect flow to the interface.
  • Data elements and controls are poorly grouped.
  • Feedback to the user is poor, especially for errors and exceptions.
  • The application is unresponsive.

Key Decisions

  • How to leverage effective interaction patterns?
  • How to improve user empowerment?
  • How to determine user experience acceptance criteria?
  • How to improve responsiveness for the user?
  • How to prioritize the user experience?
  • How to avoid common user experience architectural issues?
  • How to determine the most effective UI technology?
  • How to determine an effective layout?
  • How to enhance the visual experience?
  • How to choose the appropriate types of controls?
  • How to choose patterns for controls?

Key Techniques

  • Spend time to understand the user requirements of the application and design the interface for maximum productivity.
  • Design the screen and input flows and user interaction patterns to maximize ease of use.
  • Incorporate workflows where appropriate to simplify multi-step operations.
  • Consider implementing dummy or test interfaces to gain user acceptance and feedback.
  • Design the interface to support different levels of user permissions by hiding inappropriate controls and content.
  • Choose appropriate control types (such as option groups and check boxes) and lay out controls and content using the accepted UI design patterns.
  • Consider requirements for globalization and localization, including text direction.
  • Consider the needs of disabled users.
  • Implement technologies and techniques that provide maximum user interactivity, such as AJAX in Web pages and client-side input validation.
  • Use asynchronous techniques for background tasks, and tasks such as populating controls or performing long-running tasks.
  • Consider the requirements for richness and functionality in the interface, such as media and graphics, and choose an appropriate technology for the interface.
  • Consider how reach, deployment, and network capabilities will affect the choice of user interface technology.

Additional Resources

For more information on implementing and auditing quality attributes, see the following resources:

Last edited Dec 16, 2008 at 8:14 AM by rboucher, version 2

Comments

No comments yet.