A Quality Partnership

By Peter Varhol

More than 2 million lines of code were required to successfully guide NASA’s Rover Curiosity to a safe landing on Mars, but Curiosity doesn’t have to contend with airbags, turn signals or streaming audio from mobile phones like the Chevy Volt does. When the Volt was released, General Motors compared its development to a rocket program. The car had 10 million lines of software code and 100 electronic controllers—and that’s on the model that is now two years old.

General Motors’ 33,000 sq.-ft. Global Battery Systems Lab includes 160 test channels and 42 thermal chambers that duplicate extreme real-world driving patterns, hot and cold temperatures and calendar life. Photo by John F. Martin for General Motors.

Engineers are creating increasingly complex products, and as a result, automobiles, aircraft, medical devices, consumer electronics and more depend on software driving the hardware components. Shorter design cycle—like the Volt’s record-setting 29 months from the ground up—leave no time to waste. On the other hand, the increased complexity of embedded systems requires more testing than ever before. Design engineers need a solution to tie together design and testing of embedded systems, and they may have an answer in National Instruments’ partnership with IBM.

General Motors Battery Lab engineers prepare batteries for testing in the thermal chamber at the GM Tech Center in Warren, MI, in 2008. Photo by John F. Martin for General Motors.

IBM’s Rational software, which GM used to design and test the Volt, is being integrated with National Instruments’ LabVIEW, VeriStand and TestStand testing environments. The goal is to provide an end-to-end quality management and collaborative real-time testing solution that will increase productivity and quality while reducing time to market. In short, the companies hope to optimize upfront design and testing of embedded systems.

Linking Design and Testing

National Instruments has several technologies that are used to implement test and validation applications for these complex hardware/software systems.

NI LabVIEW is a graphical programming environment with tight integration to I/O hardware and support for IP developed in other languages such as C. Using drag-and-drop graphical icons, engineers can quickly create intelligent test programs tailored to their specific application needs.

NI DIAdem is a tool for engineering data management, analysis, and reporting that allows engineers to produce information they can make decisions with faster.

NI VeriStand is a software environment for configuring real-time testing applications, such as hardware-in-the-loop simulations or test cell control and monitoring. Out of the box, VeriStand helps teams configure a multicore-ready, real-time engine to execute tasks such as real-time stimulus generation; analog, digital and communication bus interfaces; deterministic model execution; and event alarming and alarm response routines.

NI DIAdem is a test data analysis and reporting tool. It can be used to quickly locate, load, visualize, analyze and report measurement data collected during data acquisition or generated during simulations. Reporting is vital for the team as a whole; it keeps stakeholders informed on what has been implemented, and whether those features meet requirements.

NI TestStand is a test management software tool that helps develop automated test and validation systems. TestStand can be used to develop, execute and deploy test system software. In addition, teams can develop test sequences that integrate code modules written in any test programming language.

As a result of their collaboration with National Instruments, IBM’s Rational Quality Manager, which creates and maintains relationships among product design artifacts (such as requirements and embedded software) and product test artifacts (such as stimulus profiles and test programs). It also provides views for insightful decision-making through the lifecycle. Its ability to identify and close gaps in coverage helps improve product quality and the efficiency with which this quality is achieved, especially in an environment where product requirements are changing.

Collaboration Across Silos

Collaboration breaks down the walls between quality management silos, which allows software bugs to be fixed quickly and cost-effectively before a product is deployed. IBM estimates the cost of fixing a software bug post-release can be 100 times higher than if you discover it early in the design process.

To catch those bugs early, information on design and software changes and their impact has to be communicated across the team. Engineers often work in distributed teams, sometimes around the world. It’s not always possible to get the entire team in the same room, or even on the phone at the same time. Teams building regulated systems have to keep up with changing standards, and sometimes even completely new regulations. No matter what the product, teams are constantly dealing with new versions of software, electronic systems and new technologies in general.

Collaboration is probably the most important part of any quality product development effort today. No one person can keep all of the project details in his or her head, so a project knowledge repository is essential. And because many people contribute to that repository, they have to maintain lines of communication so that all team members are fully informed about status, changes, and results.

NI TestStand automates the execution and results collection of test applications created in any programming environment.

IBM’s Rational Quality Manager brings collaborative features to product validation efforts implemented using National Instruments tools. Rational Quality Manager enables the capturing of threaded discussions directly on an asset like a test case, providing the ability for teams to have a common understanding of the state of individual aspects of the project. Link previews and selected feeds give team members insight into development status, without overwhelming them with too much information. Further, teams collaborate among themselves (and with their stakeholders) using web-based reviews and approvals.

Tackling Traceability

Collaboration is most useful when it’s automated. Team members can’t manually track requirements through specifications, development and tests, to successful product delivery. That’s why automatic traceability across project assets is necessary. National Instruments’ testing tools, combined with Rational Quality Manager, automates both collaborative activities and asset traceability.

Most development efforts go through several distinct, high-level phases, such as requirements, design, implementation, test and delivery. At each transition point, it is possible to lose information. Knowledge about requirements can be lost when moving to design, and knowledge of design can be lost during implementation.

Traceability works by providing links from requirements and similar business needs statements to downstream development artifacts, including specifications, code, tests and defects. These links let the project team immediately understand the relationship between requirements and the actual product being built. They provide a direct information connection between product lifecycle phases that is often lacking.

The IBM and National Instruments partnership promises bi-directional traceability across the entire development cycle. Links run both ways, downstream and upstream. The value with upstream linking is that it becomes possible to tell when a requirement has been satisfied. If requirements are linked to system tests, and tests are linked to defects, then team members can tell when requirements have been satisfied. This traceability also enables requirements and test engineers alike to easily perform impact analysis to determine the effects of changing a requirement or updating a test component. Upstream traceability allows developers at all stages to understand how their changes impact other aspects of the project and plan accordingly or eliminate the change altogether.

Traceability helps to ensure that teams build what was intended. This isn’t as absurd as it sounds, because the loss of information at each phase in the lifecycle means that it is possible to lose sight of some of the original needs. Also, and more importantly, requirements are apt to change during the course of a project.

A Moving Target

With traceability, teams can quickly and easily identify downstream code changes that might have to change in response to requirement changes. And requirements often change during projects. Whether the intended target market is adjusted, or inaccurate or vague requirements are fine-tuned, the era of frozen requirements and long-lead product development efforts is long past.

Product design teams building products that use a combination of hardware and software need to be cognizant of how requirements changes can affect downstream assets. This is an essential part of product teams becoming more agile and responsive to user and market needs during the development lifecycle.

NI VeriStand allows test engineers to configure real-time test applications that require stimulus generation, data logging, and/or simulation model execution.

The pressures on engineers surrounding complex designs are high, and still growing. In some cases, there are safety implications. Software failures could result in injury or death if the product has the ability to do harm. Defects or failures may also cost the company a good deal of money in recalls and lost business. The company’s reputation is also likely to be harmed, often severely, with poor quality products.

But as complex as today’s products are, they’re only going to become more complicated. Some luxury cars are already estimated to use 100 million lines of code. And it’s not just automotive and aerospace engineers who have to work with embedded software because it is making its way into more and more products that were once strictly mechanical.

The National Instruments/IBM partnership provides design engineers with a collaborative, end-to-end quality management and real-time testing solution to ensure embedded software doesn’t derail the design process. It can help reduce time to resolve defects and facilitate improved test component reuse. The partnership could help optimize the embedded system design and test process by improving quality and reducing time to market.

Contributing Editor Peter Varhol covers the HPC and IT beat for DE. His expertise is software development, math systems, and systems management. You can reach him at [email protected].

INFO

IBM

National Instruments