A Funny Thing Happened on the Way to Parallel Processing

By Peter Varhol

We need to make better use of parallel processing. Moore’s Law as we’ve understood it for the past 30 years is more or less tapped out. For most of those three decades,  processors rode the wave of miniaturization to smaller components and increased clock speeds enabled by shorter distances for electrons to travel.

But there are tradeoffs involved in maintaining the pace of miniaturization, and the integrated circuit experts seem to have reached the limits of miniaturization and clock speed before introducing undesirable side effects. Among those side effects is complexity; a single processor with millions of components is difficult to debug and prove correct.

Yet miniaturization has enabled processor vendors to add multiple but simpler cores onto a single die. Conceptually, these cores could each take a workload and run that workload. If they were separate workloads, they could run independently. If they were parts of a single job,  they could be combined at the end to produce a single result.

If only real life could be explained that simply. The computer on which I normally work has two cores. Looking at the Perfmon counters for them during normal operation reveals that only one of the cores typically shoulders the bulk of the processing load. There are many reasons for this – the inability of the operating system to allocate the processing load efficiently, and the poor parallel nature of a user-driven computing environment in general.

But the fact is that too much useful processing power goes wasted today, and it’s getting worse: Four-way cores on the desktop are starting to become commonplace, and personal computers will follow servers up the scaling curve quickly. Barring computing breakthroughs that make this unnecessary, we will almost certainly see 32 cores and more sitting in front of us in the next few years.

Yet anyone who programs knows that writing code that can run in parallel is very difficult. It requires the use of semaphores and critical sections, and a careful accounting of system resources. It opens the door to race conditions, where the result is dependent upon which processing thread completes first. The error can seem to be random, and is one of the most difficult programming bugs to track down and fix.

With this potential in mind, I talked to James Reinders, chief evangelist and director of marketing for Intel Software Development Products, about the need to parallelize far better than we’ve been able to do in the past. He believes that parallelism will come in three waves.

“The first wave is problems that are naturally parallel,” he said. “This class will be able to be managed within the processor itself,” likely with the help of the operating system. We see some of this now, and expect it to be more pervasive in the near future.

The second class of problems, according to Reinders, is one that will be largely handled by the compiler. These are problems that don’t naturally break into discrete parts, but there are enough heuristics for the compiler to understand the nature of the problem and do some parallelism. It will take longer to develop the compiler technology necessary to be able to break a program down into parallel threads for execution, but he believes that this will be achievable in the not-too-distant future.

The last class of parallel problems is the most intractable, and will require both new compiler techniques and programming skills. This will likely require at least a generation to develop new skills and then make sure these skills are taught to the next generation of programmers and compiler developers.

If Reinders is correct, it means that the issue of programs taking advantage of multiple cores may not be as difficult as it first appears. If a good portion of the programs can be managed by the processor and operating system, or through compiler heuristics, then perhaps software can more easily catch up with advances in processor performance.

Of course, it may well be that the number of programs in the first two categories is trivial, and that by far the largest number of programs will be the most difficult to solve. But I don’t think so. I think Reinders is onto something. It will take some work on the parts of the processor and compiler developers, but fundamentally it’s easier for them than it is for individual programmers. But the end result will be faster programs and more productivity for engineers.

Peter Varhol has been involved with software development and systems management for many years. Send comments about this column to [email protected].