5 Ways to Fail When Prototyping a New Design

By P.J. Tanzillo

Product developments tend to come in one of two forms: the evolutionary and the revolutionary. In evolutionary projects, incremental improvements or small additional feature sets are added to an already existing and released product. The effort and risk tend to be much better understood in the product definition phase, and therefore, the timelines and results are easier to predict.

Figure 1: This first prototype of a pipeline inspection gauge (PIG) focused only on the high risk aspects of the geolocation algorithm. The bicycle was a creative way to move the electronic system at a rate comparable to the PIG.

The revolutionary designs begin with a pipe dream or a moment of inspiration They tend to be higher risk and often venture into uncharted technical and business territory. These are the projects that require experimental prototypes before the solution can even be specified, and these are the projects that have the most volatile schedules.

  Revolutionary designs are the inventions that can change the way we live. They are the cotton gin and the steam engine, the cell phone, the robotic surgeon and the electric car. National Instruments provides prototyping platforms that are used in innovative research and design projects, from the CERN Large Hadron Collider to the space elevator. This article will describe some common pitfalls to digital prototyping the company has discovered, so you can avoid them on your next revolutionary project.

1: Treat Failed Experiments as Project Failures
What better inspiration for invention than Thomas Edison, a classic example and a role model for innovators everywhere.  In Harper’s Magazine in 1890, Edison spoke of the many failures leading up to his invention of the electric light bulb.

“I would construct a theory and work on its lines until I found it was untenable. Then it would be discarded at once and another theory evolved. This was the only possible way for me to work out the problem. I speak without exaggeration when I say that I have constructed 3,000 different theories in connection with the electric light, each one of them reasonable and apparently likely to be true. Yet only in two cases did my experiments prove the truth of my theory.” 

For truly revolutionary designs, one sometimes needs to rule out many of the invalid solutions before arriving at a valid one. Each failed experiment is an opportunity for learning, and though frustrating, this is the only way to achieve truly revolutionary results. Mangers should celebrate failures alongside successes as progress toward an eventual solution.

Figure 2: This second prototype of the EngeMOVI PIG reduced the degrees of freedom only to those that would be present in a pipeline by mounting the system on rails.

2: Optimize for Cost too Early
Project managers and business owners love to shave cost from a project. Time and time again, we see engineers making decisions based on the cost of a first prototype at the expense of weeks of development. For experimental prototypes,  most of the initial work will likely be thrown out, so reducing the schedule has a far larger impact on overall project cost than the cost of goods.

If we speak in practical terms, a good rule of thumb for the cost of an “engineering year” is $200,000. This includes salary, benefits,  opportunity cost, support staff, etc. and although it varies greatly between regions and companies, it’s a good conservative estimate. If there are 250 workdays in a year, each engineer costs about $800 per day. If you can make a decision that gets your first prototype to your customer a week earlier, for a five-engineer team, that is worth $20,000. Haggling over pennies on a revolutionary project is simply impractical before you reach volume production.

3: Optimize for Performance too Early
Similar to the project manager’s desire to optimize cost early, software engineers love to optimize for performance so they can use their hardware resources in the most efficient way. Though this is an important step in the deployment phase of product development, we need to recognize that most code in an experimental prototype is “throw away” code, so time spent optimizing is time wasted. Instead, it’s often better practice to prototype on a system with far greater resources than you think you need so optimizations of large subsets of code can be delayed to later iterations.

4: Try to Solve the Entire Problem in the First Prototype
In most designs, it’s unnecessary to implement every aspect of the design in the first prototype. Instead, it’s best to start with the highest risk element and work from there. 

  A good example of this can be found in the design of a pipeline inspection gauge (PIG) by the Brazilian company EngeMOVI. A PIG is a tool for oil and gas pipeline inspection that is inserted into the pipeline and propelled by the pressure of the circulating product. The PIG examines deformations and corrosion anomalies, helping prevent failures that can cause ecological accidents. To reduce costs and quickly replace the pipeline after the detection of an anomaly, it is necessary to know its georeferenced position.

  A continuous GPS is unavailable inside the pipe where the PIG is located, so a more sophisticated georeferencing algorithm combining the acceleration and angular rate measurements of the PIG with external sensors needed to be developed. This was the highest risk software component, and therefore, where they began development.

  The mechanical design of the PIG required a complex suspension system to guarantee the survival of the unit in extreme conditions,  so to avoid schedule delays, they devised a platform to test the algorithms under development. In figure 1 you see an NI CompactRIO controller and sensors on a bicycle, which they used as a first prototype to map the path the same way they would with the PIG inside a pipeline.

  After a few rounds of collecting data and modifying the algorithm, it became clear that the bicycle has more degrees of freedom than a pipeline; therefore, they needed a prototyping platform with movement more like the final conditions. For this, they transferred their electronics to a railway cart (see Figure 2) to further develop a georeferencing scheme accurate within 1m.

  Finally, after assembling the PIG mechanics, they conducted a field test of the system prototype on an actual pipeline. After nearly a year of commercial operation in Brazil and Colombia, the PIG is still operational. The success of the initial prototype is constantly being improved and has inspired an entire family of tools such as submarine robots for deep-water pipeline inspection, welding robots with redundant kinematics and, more recently, a motorized PIG.

5: Forget the Reason you Prototype
Engineers build experimental prototypes for a variety of reasons, but most often, it involves either proving technical feasibility or gathering user feedback. The most successful designs often iterate on these prototypes multiple times, each time gathering more information that helps them improve the next version of the system. Once a solution is well understood and the engineer has a deep understanding of both the problem and the solution, a detailed specification can be made.

  Therefore, the prototype is really a sophisticated means of requirements gathering. Code reuse from the prototype to the deployed system is desirable, but generally should not be the focus of development.

  Not all engineers are working on the next light bulb or phonograph; however, we can take the lessons of the innovators like Edison and apply them to our day-to-day projects. Any time we’re creating a design from scratch, it’s best to fully understand the problem before deciding on the implementation strategy. In our experience at National Instruments, the fastest way to such an understanding is through iterative and experimental prototypes where schedule is the primary concern and cost and optimizations are secondary.

More Info:
National Instruments

P.J. Tanzillo is a group manager for the industrial embedded software team at National Instruments.  He received his BSECE from the Ohio State University in 2003, and he currently lives in Austin, TX.