More value at lower cost. Doesn’t that seem like a good deal?

Here’s what Cassidy wrote that triggered this blog:

“Clay Christensen has an idea: Scare the hell out of yourselves.

OK, that’s not precisely the way he put it. But the author of “The Innovator’s Dilemma” is all about new ideas. Not just new — but different, unorthodox, radical, uncertain, frightening and disruptive. You don’t solve old problems with old ideas. The other day, Christensen held a one-man teach-in for non-profits and their supporters at San Jose’s Mexican Heritage Plaza, preaching the gospel of “disruptive innovation.” It’s an idea that is embedded in Silicon Valley’s DNA. It is also an idea that is a lot easier to talk about than to actually deploy.”

So what’s the connection with “intelligent design”? I was immediately reminded of a great new product, a home thermostat of all things, that I’d just written up from last month’s CES show.(See “Friday Video: Two minutes of system-design expertise from Matt Rogers, VP of Engineering and founder of Nest and designer of the thermostat of the future”) The thermostat is from a new company called Nest and the product is called the Nest Learning Thermostat. It looks like an updated 21^st century version of the old golden Honeywell manual thermostats that were common in the 1950s and 1960s. (Noted industrial designer Henry Drefuss created the Honeywell thermostat’s iconic circular industrial design in 1952.)

However, unlike those old Honeywell thermostats that were based on bimetallic temperature-sensing coil springs and mercury tilt switches, the Nest Learning Thermostat is based on a 32-bit microprocessor. And a TCP/IP stack. And WiFi. In adding these specific technologies to its Learning Thermostat, Nest demonstrates a grasp of Christensen’s concept of the “Innovator’s Dilemma.”

How?

First, understand that the context of what we mean by “home” has changed. Frequently in the modern Western world, there’s no one home. That means the home’s heating and cooling requirements are different. Also, our definition of “home” increasingly includes a home WiFi network, possibly with access to the outside world through the home’s broadband router. Add some intelligence and connectivity to a thermostat, and you can disruptively change how we heat and cool our homes with a large resulting energy savings.

After installation, the Nest Learning Thermostat needs to know three things:

1. What’s your Zip code (Yes, I know there’s a US bias built in here. Early intelligence has its limits.)
2. Should the thermostat start to heat or cool your home?
3. What temperatures should the Nest Thermostat use to heat or cool your home while you are away?

After that, you set the thermostat to the desired current temperature and the Nest Learning Thermostat then starts to observe your daily habits (with respect to heating and cooling only!). It monitors when you turn up the heat (like in the morning) and when you turn it down (like before you go to sleep). It notes when you turn on the cooling (like when the afternoon sun starts to make it overly warm for you).

The Thermostat learns your daily routing and your weekend routine (weekend habits are different for most people) and by the eight day, the thermostat has developed a pretty good idea of your habits and strives to maintain a comfortable home for you by catering to those habits. The Nest Learning Thermostat can start to heat your home several minutes before you rise in the morning. (This is a major feature for those of us whose first job in the morning is to turn up the heat for our spouses so they can get up.)

You can log into your thermostat from your office to adjust the heat so that your home is warm by the time you return from work or to let the thermostat know you’re going out for dinner and the heat can be delayed for a few hours. And of course, you can do the same from your smartphone no matter where you are on the planet (assuming you’ve got cellular coverage wherever you are).

The Nest Learning Thermostat also tries to train you after it learns your habits. It wants you to learn better habits in terms of energy consumption. It does this by starting to display a small green leaf when you turn down the heat or taper off on the air conditioning relative to your habitual heating and cooling use. This leaf tells you that you’re saving energy and thus money. It’s a subtle form of coercion and some people won’t like being told what to do by a thermostat. Others—the ones most likely to buy this product—will appreciate the watchful eye.

None of this would be possible if the Nest Learning Thermostat did not have a 32-bit microprocessor and a WiFi connection. Note that it took an entirely new company to build a thermostat like this. Although Honeywell still makes thermostats, microprocessor-based ones at that, it’s not building anything like the Nest Learning Thermostat. At least not this year. We just had to replace the thermostat in our condo and it is a Honeywell thermostat. It’s a standard setback thermostat design with an LCD. I have to tell it the time. I have to program it for a setback sequence. And Honeywell knew that the user interface on this product was so unintuitive that it kindly included a fold-out instruction booklet that pokes out of the top left of the thermostat, as you can see from this photo.

Frankly, I find the display of the new Honeywell thermostat extremely confusing. It shows the current temperature on the left side in large characters, the setpoint temperature in somewhat smaller characters on the right side, and the time in even smaller characters in the middle. To me, it looks like the display information was simply thrown on the display with little consideration to how the information is portrayed. The time is not the central piece of information on a thermostat yet that information is central to the display, albeit in small characters. The two important and conjoined pieces of information, the current and setpoint temperatures, are spaced nearly as far apart on the display as possible. And there’s no fixing this design with a firmware update. That LCD’s permanently configured to save manufacturing cost.

Not, repeat not intelligent design.

You can bet that the Nest Intelligent Thermostat does not have an ungainly instruction booklet poking out of its sleek, smooth industrial design. To anyone who has ever used a regular dumb thermostat, the Nest Learning Thermostat’s everyday operational use appears to be entirely intuitive. And if you want to do more complex things with the Nest Learning Thermostat, you interact with it through a Web page, not a handful of multi-use rubber buttons and a limited (for cost reasons) LCD. For these reasons (and a few more), I think the Nest Learning Thermostat is one of the best examples of 21^st century intelligent design I’ve seen. It offers up several lessons in thoughtful design that I hope you will appreciate as much as I do.

Note: To read Mike Cassidy’s entire column, click on the following link: “Clay Christensen sees Silicon Valley non-profits’ dilemma as the innovator’s dilemma”

Low-power microcontrollers

If there ever was a year for microcontroller advancement, this was it. Every major microcontroller vendor had something new and exciting on the low-power front. So many developments that I can only hit the highlights:

In August, ARM’s Alan Rampon wrote a blog post listing 17 microcontroller vendors that were offering a broad range of low-power devices based on various ARM Cortex-M series processor cores. The vendor list includes:

• Analog Devices (Cortex-M3)
• Atmel (Cortex-M3)
• Broadcom (Cortex-M3)
• Cypress Semiconductor (Cortex-M3)
• Dust Networks (Cortex-M3)
• Ember (Cortex-M3)
• Energy Micro (Cortex-M0, M3)
• Freescale Semiconductor (Cortex-M4)
• Fujitsu (Cortex-M3)
• Holtek (Cortex-M3)
• Nuvoton (Cortex-M0)
• NXP (Cortex-M0, M3, M4)
• ON Semiconductor (Cortex-M3)
• Samsung (Sortex-M0, M3)
• ST Microelectronics (Cortex-M3)
• Texas Instruments (Cortex-M3)
• Toshiba (Cortex-M3)

That list is probably somewhat dated already, but you get the idea. The proliferation of low-power microcontrollers greatly accelerated during 2011. One such device that really sticks in my mind (because it’s recent), is the onset of shipments of the NXP Semiconductor LPC4350, which packs an ARM Cortex-M4 and an ARM Cortex-M0 into one microcontroller that costs less than $4 in quantities of 10,000.

This microcontroller is on the forefront of a new wave of processor design called “asymmetric multiprocessing” and there’s a real “wave-of-the-future” look to this development. (See “More news on the asymmetric processing SoC front”)

Asymmetric Multiprocessing

The microprocessor is 40 years old (last month!) and silicon microprocessor implementations have really advanced over those four decades while many of our design memes have not. In particular, I’m thinking of the meme that says “processors are expensive, so layer as many tasks as possible on a processor to save money.” The net effect of this meme is to make us develop increasingly complex multitasking schemes in an attempt to get processor utilization up to 80% or 90% or perhaps even 95%.

Now any engineer can tell you that when you load any component to near 100%, you have just sent and engraved, gold-plated invitation to Murphy, asking for an audience. In other words, something will go wrong. You won’t always get the latency you expected. You won’t always get the bandwidth you need.

So you’d better ask yourself: Are complex multitasking systems really worth the effort when I can get two 32-bit microprocessor cores in one device for less than $4? You’d better be serious coming up with that answer. I believe that asymmetric multiprocessing will remake all of design, including low-power design, during this coming decade.

Asymmetric multiprocessor design wasn’t the only innovation that loomed in 2011. Xilinx finally announced the first four members of its new Zynq 7000 EPP (Extensible Processing Platform) family, which fuses a processor complex containing two ARM Cortex-A9 processor cores with an FPGA fabric.

Now assembling systems with microprocessors and FPGAs isn’t new. In fact, putting processor cores and FPGA fabrics onto the same piece of silicon isn’t particularly new either. However, doing it right? That is new. And this development fits into the low-power design world because putting the processor complex and the FPGA fabric on chip with a massive on-chip interconnect between the two cuts interface power significantly by reducing the interconnect frequency. You don’t need GHz interconnect clock rates when you have thousands of wires for parallel interconnect.

2.5D IC Assembly

Speaking of Xilinx, the company started shipping engineering samples of the Virtex-7 2000T FPGAs to customers last month and this too is a low-power design story. The story is completely told in this graphic:

The Xilinx Virtex-7 2000T is a very large FPGA with two million logic elements. But it’s not a monolithic piece of silicon. Rather, the Virtex-7 2000T consists of four “identical” FPGA tiles, each with half a million logic elements (and a ton of other stuff). The FPGA tiles are mounted on a silicon interposer, which establishes more than 10,000 connections between each tile (56,000 connections in total). The silicon interposer is a fascinating piece of technology. It’s a 65nm IC with four layers of metal on each side of the die and no transistors. It’s a silicon circuit board that must be made in a wafer fab. In this case, TSMC owns the fab. The interposer is as large as the stepper reticule will allow. The advantage here is that each FPGA tile is a quarter of the size of the interposer, and die yield has an exponential relationship to die size. The smaller the die, the better the yield percentage. So 2.5D assembly makes a lot of sense in several different ways.

The 2.5D IC assembly-with-interposer approach taken to create the Xilinx Virtex-7 2000T allows the FPGA tiles to use lower power I/O drivers because these drivers will only be driving short, closely controlled traces between adjacent tiles. That system-design knowledge saves power. Although the Xilinx Virtex-7 2000T uses four identical die fabricated with a 28nm process technology to realize the active elements, 2.5D IC assembly permits heterogeneous die assembly as well, as shown in this image from Xilinx:

As you can see, 2.5D IC assembly allows designers the freedom to intermix die from radically different IC technologies such as logic, memory (DRAM, Flash, SRAM, etc.), analog, and RF. It’s a pc-board-like technology but on a much smaller scale. The resulting 2.5D device may well be better optimized and cost less than it might if the design team attempted to place everything on one monolithic die. That’s a topic I’ll take up in Part B of this blog entry.

“We want to use the dense, reliable silicon interconnect inherent in Moore’s Law to integrate the dissimilar chips used in today’s systems, but we face an economic barrier because the interconnect on silicon chips is 1,000 times more expensive than the interconnect on PC boards.

“We could enable a new silicon-based interconnect paradigm if we could make silicon interconnect wafers for $10, just what silicon solar wafers cost today. The problem of mapping solar technology onto Moore’s Law is straightforward, but difficult, and we believe DecaTech has the answer.”

Now don’t take that $10 per interposer wafer to the bank. I get the impression that’s a long-term goal, not a short-term pricing roadmap. However, even a 10x drop in interposer costs will have a big influence on the future of 2.5D assembly technology.

And why should we as low-power systems designers care? Because interconnect is expensive and because interconnect now largely determines system performance. First, think about expense. Let your mind go back 40 years (if it can) to the birth of the microprocessor, which we celebrate this month. In the 1970s, microprocessor interconnect meant a bus. Not on a board but in a system. One of the most successful early microprocessor buses was the S-100 bus. It was named for the 100-pin edge connectors and the 100-conductor bus used to interconnect system boards in the original Altair 8800 microcomputer introduced in 1975 and subsequently adopted by several microcomputer vendors including Imsai, Vector Graphics, North Star Computers (formerly known as Kentucky Fried Computers), and Processor Technology and by board vendors including Godbout Electronics/Compupro and Morrow Micro-Stuff/ThinkerToys.

Back then, due to the nascent state of semiconductor integration, you would to have a processor board, one or likely more than one memory boards, a video board, and one or more I/O boards. A major system expense was just the half dozen or so 100-pin edge connectors and the simple but large circuit board that implemented the bus. The S-100 connectors were expensive and you needed a lot of energy to drive the bus lines because they were physically large and because—as bus speeds increased—they required resistive termination to prevent ringing and you needed even more energy to drive the termination resistors.

By 1981 when the IBM PC appeared, things were getting somewhat better. We still had half a dozen edge connectors but we were down to 62 edge-connector pins (for 8-bit systems). Add another 36 pins when we jumped to 16 bits and we found ourselves right back at close to a 100-pin bus. So much for progress.

For board-to-board interconnect, things are going serial (think of the PCI evolution to PCIe) but chip-to-chip interconnect on a board is still largely parallel with lots of pins on a chip looking to connect to lots of pins on other chips. There is still impedance in those pcb traces and you still need relatively big drivers that consume significant amounts of energy to drive those traces. Hence a movement to go serial for chip-to-chip interconnect on a board—an extension of the migration of buses to serialized versions.

High-speed serial buses incur their own costs. There’s the energy cost of driving even a few wires at multi-GHz speeds and there’s the performance hit in the form of latency increase that you get when you serialize and then deserialize a data stream. It’s not all wine and roses.

So we try to put as much as we can on one IC, but that’s not an ideal solution either. Not all IC processing is the same and chips with different functions are really better off being fabricated with different IC fabrication processes. For example, NAND Flash and DRAM processes push as far down the Moore’s Law curve as they can get, as fast as they can to boost density and cut cost per bit. CMOS logic processes are right behind the memory processes but use more layers of on-chip metal interconnect. Because they require more random connectivity than memories. Analog ICs typically operate at higher supply voltages and they’re nowhere near the leading/bleeding edge of IC processor technology. It’s not economical to put all of these different functions on one die, and so we see renewed interest in multichip modules, known by the 21^st-centrury name: 2.5D IC assembly using silicon interposers.

2.5D assembly using bare semiconductor die attached to small interposers instead of big circuit boards significantly changes the parallel/serial I/O equation. Suddenly, you don’t need such big I/O drivers on the chip because there’s no wire bond, no IC package interconnect, and significantly shorter traces to drive. Suddenly, massively parallel I/O consumes only a fraction of the energy it previously needed and the balancing equation that calculates the breakeven point between parallel I/O and serialized, high-speed I/O alters. The balance alters to favor parallel I/O more and serial I/O less.

And when major changes like that happen to such equations, the way we design systems also changes.

Between those two years, Cochran worked on a huge number of projects including the HP 204B audio oscillator where he used transistors and a hugely ingenious double-spiral-cam potentiometer actuator to design the famous tungsten light bulb out of Bill Hewlett’s original audio oscillator design. (That double-cam actuator makes a linear pot do unnatural things and you need to see the short, 3-second video to believe it!) But it’s what he did in the 1960s and 1970s that make him the topic of this particular blog post.

Cochran was working at HP Labs when Malcolm McMillan and Tom Osborne dropped by with two very different calculator prototypes. It was the legendary Barney Oliver, Grand Wizard of HP Labs who conjured the idea of merging these two very different machines into one massively powerful scientific calculator. McMillan’s design—called Athena—could perform transcendentals using Jack Volder’s CORDIC algorithms but it had a fixed-point architecture so it was not deemed accurate enough for repeated engineering calculations. Osborne’s design was barely more than a simple 4-banger calculator but it had a really elegant, floating-point hardware design based on the 1960s version of a VLIW processor.

Cochran got the job of trying to come up with a way to unify the two architectures. He writes “I was looking at Osborne’s architecture and trying to figure out what an algorithm was. I even flew down to Southern California to talk with Jack Volder who had developed the CORDIC  transcendental functions used in the Athena machine and talked to him for about an hour. He referred me to the original papers by Meggitt where he’d gotten the pseudo division, pseudo multiplication generalized functions.

My job was to determine how many digits and what the operation time was required; what the architecture had to be; how many registers did we need, clock speed, etc? Other people were coming back with their inputs on cathode ray tube display, keyboards [and so on]. Should we use transistors or small-scale integration? There was no large-scale integration, but there was medium-scale integration, MSI which meant maybe 10 transistors in a chip.”

From Cochran’s architectural contributions, plus substantial work from other engineers in HP Labs and Tom Osborne (who remained an HP consultant for many more years), HP introduced the HP 9100 Scientific Calculator in late 1968. It’s a marvelous machine and it was a real design breakthrough for its day. It also weighed 40 pounds and drew 70 Watts out of a wall socket.

Now 70W isn’t all that much compared to the amount of electrical power you’d need to replicate the HP 9100’s computational abilities with a minicomputer or a mainframe, so you could consider this a lower-power design for its day. But it’s what happened next that really takes us to the domain of miraculously low power.

Cochran writes: “As soon as the 9100 started showing success in the market place Hewlett started to bug me personally. I know he also talked to Tom Osborne about it, what do you think, and so on. But he would come into the lab and he’d look for me and he’d say, ‘Hey, how are you coming with putting the 9100 in my shirt pocket?’ He said, ‘I want all that computational power in my shirt pocket.’”

OK. From 40 pounds and 70W to something that runs off batteries and fits in your shirt pocket. Now that’s a stretch. Oh, and one thing I forgot to mention. The HP 9100 calculator design had exactly one integrated circuit in it. That IC was used in the magnetic card reader. Osborne didn’t like the primitive ICs offered at the time. He didn’t design the card reader but the rest of the machine was implemented with discrete transistors, magnetic-core RAM, rope memory (go look that one up), and a large capacitive ROM fabricated from a 16-layer printed-circuit board. None of that technology was headed for a shirt pocket. Not in this universe.

Cochran then writes about his shirt-pocket epiphany: “Tom Whitney and I went down to Fairchild Semiconductor on Ellis Street, Mountain View, and they wanted to show us a calculator architecture that they were planning to provide to various companies that wanted to build calculators and semiconductors. So we went down to look at it. And I looked at it, and oh, this could do the algorithms. See, I knew. By this time, I had already fit the algorithms into a small-scale integrated machine, the 9100. So I knew exactly what architecture I needed, the capabilities of the architecture. I didn’t know what it was going to look like, but I knew what its capabilities had to be.

It was I think September of 1970; I saw a design that was different than anything else. It was not your classic computer architecture as taught at the universities. It was all shift register. It was designed for the technology at the time. When talking to the people at Fairchild I meet a fellow, Rich Whicker, who later came to work at HP. I said, “God, this design, did you think of this?” He says, “No. We got it from Sweda, the cash register company.” Sweda at the time was trying to make an electronic cash register or Point-of-Sale products and they were using shift registers.

Shift registers were the densest form of integration of integrated circuits at the time; you had to keep the clock moving and so on. It had no static memory. So here was a design using shift registers a 20-digit chipset that could satisfy anybody making a four-function machine. Add, subtract, multiply, and divide. It could give you the numbers as big as most people wanted, but it was all fixed point. 20 digits should be more than enough for anybody. You could have the decimal point anywhere in that stream.

I got really cranked up about seeing that architecture at Fairchild, I got very excited. And I’m whispering in Tom Whitney’s ear, ‘God, this is great.’ And I’m trying not to be too excited while I’m there. When we drove away from there and, I said, ‘God, that’s exactly—you know, I can tweak that architecture just a little bit. We don’t need the full 20 digits, we can do this and this and this. And gosh, yes, I can do it, I can do it, I can do it.’”

And that’s when the HP 35 Pocket Scientific Calculator crossed over from the realm of the impossible to the realm of the possible. When David Cochran thought that it could. Two years later, in 1972, it became a reality. A pocket scientific calculator that ran on three NiCd batteries in a pack. The rest, as they say, is history.

Be sure to read the whole story at http://www.hpmemory.org/timeline/dave_cochran/a_quarter_century_at_hp_00.htm#chapter_08.

AgigA Tech’s servers sit in a small room in the company’s San Diego corporate headquarters. One recent Saturday, these servers were quietly doing their jobs when an unscheduled power outage occurred. Thanks to uninterruptible power systems backing up the servers, the servers continued to do their jobs. Unfortunately, the air conditioning system at AgigA Tech headquarters has no backup power and did not continue to do its job. The temperature in the server room at AgigA Tech started to climb from the waste heat being thrown off by the equipment.

There was no one in the building to notice.

Eventually, power from the grid came back on and the cooling system restarted. However, by that time the temperature in the server room had climbed high enough to cook six of the hard disk drives in the server room’s RAID arrays. Fortunately, the RAID arrays performed as designed and there was no data loss. But if the power outage had lasted longer and if the servers had continued to run without cooling, eventually all of the RAID drives would have died. Even the best RAID system cannot preserve data when all of its drives fail. As it stands, even the hard disk drives that did not fail are suspect because they’ve been heat stressed. They too need to be replaced quickly before they fail prematurely as well.

After this small weekend near-disaster, my friend Ron Sartore started to ponder the ramifications and lessons of the incident. First, he realized that his servers should be sensing ambient temperature and shutting down gracefully when the server room overheats. In fact, Ron sort of assumed that’s what would happen. Bad assumption, as it turned out. “How often do you test that?” asked Ron. “No one wants to pull the plug on these things to find out” he added. I’ll bet a lot of business owners with little server rooms make precisely the same assumptions that Ron and his IT team did when they designed their server room.

Next, Ron began to think about the uninterruptible power supplies from an engineering perspective, which really calls into question the entire concept of uninterruptible power supplies for servers. There’s clearly no reason to continue to operate servers during a power outage if there’s no cooling available. In fact, there’s a clear reason to shut them down as quickly as possible to prevent overheating and hardware failure due to lack of cooling.

Now large data centers—like the ones operated by Amazon, Microsoft, and Google—have on-site Diesel generators that operate both the servers and the data center’s cooling systems during a power outage. These companies cannot afford to have their servers shut down. Every minute—actually every second—that the servers are down means lost revenue, lost profit, and lost customers. But most companies are not Amazon, Microsoft, or Google.

There are hundreds of thousands of companies in the US and millions in the world that run their servers in small server rooms or even closets where there are uninterruptible power systems designed to keep servers running as long as possible but where there is no backup power for cooling. For these companies, their server-system designs will cook and kill their hard drives rather quickly in the event of a power failure. We design smaller server systems this way almost without thinking. The UPS is a check-box item, meaning we don’t even think critically about including one.

Ron’s story made me think about UPS costs—both the acquisition cost and the operating cost over the life of the UPS. You see most uninterruptible power systems are designed so that they always supply power to the attached servers. Even while a UPS is running from the power grid, it’s still consuming and wasting energy. Today’s best UPS designs are perhaps 95% efficient. That means they consume about 5% of their input energy. All…the…time.

Less expensive UPS designs might be only 90% efficient. They waste about 10% of the energy consumed by the attached server(s).

In reality, a UPS that’s not operating at maximum load is likely to run at somewhat reduced efficiency because UPS manufacturers tend to rate their products’ efficiency at maximum load. In addition, IT managers prefer to overspecify the capacity of a UPS, which is normally good engineering practice but here it pretty much guarantees that the UPS will not operate at maximum efficiency.

Where does that wasted energy from the UPS go? It’s converted to waste heat, of course. Ron told me that his home UPS (purchased from Costco) runs hot even though it is lightly loaded—13W for a small PC appliance. Lightly loaded, a UPS might convert just as much power into waste heat as it delivers to the load.

And where does the waste heat from the UPS go? Right into the cooling system, assuming the cooling system is operational. During a power failure, it probably isn’t.

How much energy is needed for cooling? It depends on the cooling system. I’ve often heard from various presenters talking about energy needs for cooling data centers that it’s a 1:1 ratio—for every Watt of power emitted by equipment as waste heat, you need another Watt to remove the waste heat. Ron tells me that many of today’s cooling systems are actually more efficient than that. Some need only half a Watt to remove a Watt of waste heat from the server room—K factor of 2 for you HVAC engineers. Some high-efficiency cooling systems achieve a K factor of 3. So you can multiply the waste heat generated by a UPS by a factor of between 1.33 and 2 to determine the actual energy cost of UPS inefficiency.

Finally, we get down to computing the actual costs for using a UPS in a server room. Let’s start with a 2700W Dell UPS Short Depth Rack High Efficiency Online power-backup unit. When I looked it up, the purchase cost for the UPS was $1360.99 plus $115.68 for tax for a grand total of $1476.67. Shipping, at least, appears to be free. The UPS has a 3-year expected life and is rated as 95% efficient. About 5% of the energy it consumes is converted to waste heat when it’s fully loaded.

How much will it cost to run this UPS over its three-year expected life? Let’s use northern-California electric rates for commercial/general service from Pacific Gas and Electric where I live. PG&E charges about $0.20/kWh in the summer and about $0.15/kWh in the winter. On average, that’s about $0.175/kWh over the course of a year. Waste heat from the 2700W Dell UPS is about 135W (95% efficient) with another 70W or so needed to remove the waste heat through the air-conditioning system (assuming a K factor of 2). Total power required to have the UPS online all the time in case of a power-grid failure is about 200W continuously because the power for the servers is always flowing through the UPS.

Do the math and it works out to about $0.84 per day just to run the UPS, which is $306.60 per year in electricity cost for power-failure insurance. Over the three year rated life of the UPS, you’ll spend another $919.80 to run this UPS continuously—nearly as much as you spent for the initial UPS purchase. The total cost of adding this UPS to your system is about $2400 every three years using a back-of-the-envelope sort of calculation. If the UPS or air-conditioning systems are less efficient than the ones used in this example calculation, then the energy costs will be higher.

Now let’s be crystal clear here. Ron Sartore isn’t professionally disinterested in this story. He’s not exactly objective. His company, AgigA Tech, makes a line of AGIGARAM DDR2 and DDR3 memory modules that combine DRAM and NAND Flash memory with a controller on board that can independently transfer data back and forth between the module’s DRAM and NAND Flash without going through the server processor. In the case of a power outage, the on-board AGIGARAM controller backs up the data in the DRAM and puts it into the NAND Flash on the memory module using a small amount of standby power supplied by an independent bank of ultra capacitors connected directly to the memory module. Once backed up, the server data is safely stored for 10 years in the NAND Flash even without power. Standard servers and server-management software aren’t designed to exploit the features of this type of server memory that can safely back itself up. So even AgigA Tech’s IT department can’t configure a standard server system using AGIGARAM. At least not yet.

Bottom line, you or your customers may well be spending thousands of dollars for acquiring and powering a server UPS but you will not get the power-failure insurance you expect. You will not get a system that protects your data very well in the event of a power failure, as AgigA Tech has discovered. Instead of reliable backup, equipping a small data center with a UPS can cause hard disk drives to fry should there actually be a power failure—as they did at AgigA Tech—and it costs thousands of dollars extra in UPS costs to allow this to happen. “We paid good money to self-destruct ourselves” said Ron. Even though no data was lost in this example because the drives that failed were installed in RAID arrays, this approach seems like very poor engineering design. Ron has convinced me.

PS: While writing this blog entry, I received a letter from the IEEE. The envelope prominently featured this statement on the outside: “The BEST project plans include dependable backups for ‘out-of-the-blue’ accidents.” Although it might appear that the IEEE was thinking about this very blog entry when it mailed this letter to me, it turns out they’re just trying to sell me accidental death and dismemberment insurance. However, the coincidence is uncanny. In reality, that’s exactly what we’re discussing here—reducing the cost and improving the effectiveness and energy efficiency of accidental death and dismemberment insurance for servers.

Here’s how it works. You start with application code written in C. You add pragmas around known code hotspots to switch on Compaan’s HotSpot Parallelizer and to switch it off. You discover these hotspots using regular code-analysis techniques already used for other sorts of software-specific optimizations. So far, nothing new here.

Then you submit the code to the Compaan HotSpot Parallelizer for analysis. The Parallelizer analyzes the code and creates a Kahn Process Network (KPN, http://en.wikipedia.org/wiki/Kahn_process_networks) that consists of many independently executable processes and the communications linkages needed to pass data between these processes. What you then end up with is several independent C programs that can be compiled and run on one processor, run on several processors, run or on some mix of processors and hardware built using a C-to-hardware compiler. Here’s a picture of the process:

The advantage of this approach is that it’s entirely automatic once you mark the hotspots with pragmas. To use this approach, your design will need to consist of deterministic, independent processes. Parallelization consists of creating a Kahn Process Network and then generating C code for the various independent programs. That generated code must include all of the inter-program communications needed to operate the KPN. Inter-program communications take place through FIFOs, which might be real hardware FIFOs or, more likely, FIFOs implemented in shared memory.

You could do this by hand and in a simple system you can do this by hand. In a complex system, you’ll want all the automation you can muster because otherwise the complexity will kill you, your team, and your project.

Once you have the several C programs that constitute the KPN, you can decide where each will execute. Some might execute on the same processor. That’s convenient because the inter-program communications is simple and takes place in the processor’s memory space. However, you’ll get no acceleration by running everything on one processor. In fact, you’ll likely slow things down with the added inter-program communications overhead. So, you might choose a multicore processor. Compaan’s HotSpot Parallelizer would seem to be a fast way to accelerate code execution for multicore designs. You might also wish to take some of the C programs in the KPN and transform them into hardware to maximize the acceleration potential. It’s your choice, based on cost/performance tradeoffs that are familiar to every System Realization team.

Compaan’s got some case histories that are certain to interest you. Just ask them to share.

Now the reason I’m writing about this product in my Low-PowerDesign blog is because you must use parallelization to drop power consumption. Stacking every possible task on one multi-GHz processor is not going to result in low-power operation and we should all know that by now. However, there are naysayers who tremble and say “we don’t know how to code for parallel execution.” Well, Compaan’s HotSpot Parallelizer apparently does.

The SpiNNaker project is again in the news at EETimes Europe (see “A million ARM cores to host brain simulator”) and the idea of harnessing one million ARM processor cores is certainly a big idea. It excites me. However, we’re still at the humble beginnings of the project.

The SpiNNaker project’s first test chip harnesses 18 ARM9 cores on one 130nm chip manufactured by UMC in Taiwan. This is a 100M-transistor chip and, like most many-processor SoCs, the SpiNNaker SoC mostly consists of memory. The memory needs to be close to the processors for speed and for low-power consumption and there are 55 32Kbyte SRAM blocks on the SpiNNaker die. That’s 14 million bits of SRAM and, frankly speaking, that’s really not very much SRAM. Eighteen processors isn’t really a large number of processors either when your stated goal is one million.

The ARM processors on the SpiNNaker chip use packet communications to emulate the electrical spike communications that occur among the neurons in human and animal brains. From a hardware perspective, I think it’s easy to conceive of a system-level design like this and even conceptually scaling the design to a million connected ARM9 processors isn’t really hard, as long as you don’t try to enumerate all of the processors in your mind. However, with 18 processors per chip, you’ll need approximately 55,600 chips to build an interconnected network of one million processors. That’s still a mighty big box of hardware. More on that in a bit.

The rub is that we really don’t have many good ideas for programming such a massively parallel system. The SpiNNaker project seems to be mostly a hardware endeavor with the explicitly stated intent of developing a hardware testbed for brain researchers who will use SpiNNaker systems for studying various theories of brain function. Presumably, we’ll learn more about massively parallel programming by working with these systems and no doubt we will. As Furber says in a quote published in the EETimes Europe article, “We don’t know how the brain works as an information-processing system, and we do need to find out. We hope that our machine will enable significant progress towards achieving this understanding.”

Each SpiNNaker chip in the current design is bundled with a 166MHz, 1Gbit DDR SDRAM and packaged in a 300-pin BGA package. But we’re not going to be building million-processor testbeds with 18 processors per packaged chip. I’m almost absolutely, positively certain about that. This first SpiNNaker prototype just doesn’t scale to one million processors very easily. So the question is, how to get there?

Well, possible clues to answer that question can be found in two recent blogs that I wrote on the EDA360 Insider blog. First, Samsung has just announced successful tapeout of a 20nm test chip incorporating an ARM Cortex-M0 processor core. (See “Samsung 20nm test chip includes ARM Cortex-M0 processor core. How many will fit on the head of a pin?”) Now an ARM Cortex-M0 processor is not as powerful as an ARM9 processor, but then it’s not supposed to be. It’s designed for control-oriented applications and its 3-stage execution pipeline isn’t designed to get maximum speed from any given process technology. However, we’re building a system that emulates a brain that operates at a few hundred Hertz (that’s Hertz, not kilohertz, megahertz, or gigahertz) so I really don’t think the clock speed is all that critical when you’re talking about a million processors. The ARM Cortex-M0 processor core is still a 32-bit RISC processor and I am guessing with a high degree of confidence that it’s fully up to the task of executing the required electrical-spike calculations, albeit not quite as quickly as an ARM9 processor.

What’s interesting about a 12-to-14Kgate ARM Cortex-M0 processor implemented in 20nm process technology is that my calculations suggest that more than half a million ARM Cortex-M0 processors would fit on a chip the size of an Intel “Tukwila” Itanium processor (OK, that’s a big chip, but it’s a commercial one) and that calculation is based on the published number for the area required by an ARM Cortex-M0 implemented in 90nm process technology, not 20nm. Now there’s a lot of slop in this calculation. First, there’s the disparity of using 90nm numbers instead of 20nm numbers. Then there’s the disparity caused by putting no memory at all into the calculation. I just mentally tiled processors edge to edge. Ditto, there’s no on-chip interconnect.

So you probably won’t get half a million ARM Cortex-M0 processor cores on one 20nm chip. But you might get 100,000 or 200,000 ARM Cortex-M0 processor cores on a chip along with an interesting amount of memory and the required interconnect. Now we’re talking about only a handful of chips to get to one million processors. We’re talking about a tabletop box. Now we’re getting into the realm of the feasible for million-processor systems.

The second related blog entry I recently wrote in EDA360 Insider that also bears on this very interesting endeavor was about an announcement from Imec, a global research company. Just days ago, Imec announced that it and its partners successfully assembled a custom logic chip with two DRAMs in a stacked 3D configuration. (See “3D Thursday: IMEC prototypes 3D chip stack, finds some thermal surprises”.) This 3D stacked-chip prototype allowed Imec to test out some process ideas for manufacturing 3D stacked chip assemblies and to make some critical thermal tests to verify thermal models that will be so necessary when 3D assembly goes mass market. The 3D chip stack uses copper-tin micro-bumps and compression bonding for the electrical and mechanical assembly of the chip stack and you can see photos of the assembled stack below.

Here’s a photo of the overall chip stack:

And here’s a close-up of the edge of the chip stack to show the three stacked die.

The 3D Stack’s base chip is approximately 750µm thick. The two top components in the chip stack are each 25µm thick. There’s more technical info in the referenced EDA360 Insider blog.

I am convinced that 3D stacking of logic and RAM chips will be absolutely essential to developing massively parallel, low-power systems like the ones envisioned by the SpiNNaker project. First, the only way to feed data and instructions to massively parallel processing chips is through large amounts of on-chip memory and through high-bandwidth, low-energy channels connected to large off-chip memories. 3D assembly techniques permit both Wide I/O and high-speed serial I/O channels to work most effectively and at minimal energy levels and I expect to see rapid adoption of 3D assembly—even and perhaps especially in high-volume, cost-sensitive applications such as mobile phone handsets—in the next few years. This is precisely the sort of manufacturing technology we require to think seriously about million-processor systems.

Now all we need to do is figure out how to program them.

Face it, some most IP blocks aren’t that well documented. You may never know enough about the internals of a purchased IP block to fiddle with its clocking. Legacy IP blocks may have been long abandoned by their designers who have gone off to other tasks, other companies, other planes of existence. Even a block you’ve designed yourself may have scrolled off your own internal memory window long ago.

Designers everywhere have a common solution for these sorts of problems. “Give me a tool to do this” they demand from EDA vendors. “I just want to push the button.”

Usually, that’s easier said than done. Calypto’s got a tool you can try however. It’s called PowerPro and comes in two flavors: CG and MG. The CG flavor is based on the company’s SLEC sequential logic equivalency checker. That’s a tool that checks to see if modified IP block “A prime” works the same as original IP block “A.” It’s a general-purpose EDA tool with a variety of uses and one of those uses is for comparing an IP block’s function before and after clock gating.

Calypto’s PowerPro CG encapsulates the SLEC EDA tool to produce a “done for you” tool that can automatically insert clock gating into an IP design. It also checks to make sure the IP block’s behavior doesn’t change as a result of the added clock gating. Usually the insertion process takes 4 to 8 hours according to Calypto CEO Doug Aitelli who spoke to me about the product at DAC 2011 in San Diego. What do you get for this overnight run? Usually 10% to 30% reduction in dynamic power said Aitelli. Sometimes as much as 60%. Not bad for “pushing the button” I’d say.

There’s another flavor of PowerPro called PowerPro MG. Nope, not named for a cute little British sports car, “MG” stands for “memory gating.” We tend to forget that today’s SoCs are more than half memory measured by die area. Usually SRAM. We sort of allude to this fact when we talk about MPSoCs—multiple processor SoCs. With each of those processors comes a boatload of on-chip SRAM for fast execution. However, we don’t seem to explicitly call out the memory. We tend to ignore it. I guess MMSoC—mostly memory SoC—doesn’t have the same cachet as MPSoC in our processor-centric world.

However, if more than half of an SoC is SRAM, it makes sense to pay some attention to reducing the power consumption of an SoC’s on-chip SRAM blocks. That’s what Calypto’s PowerPro MG does. It can automatically add memory gating to an SoC design by evaluating the design’s behavior across many cycles.

It also goes a step further. Many SRAM blocks for SoCs now have a sleep mode where the memory’s operating power can be reduced by shutting down peripheral circuitry such as address decoders and sense amps while keeping the memory storage array alive. According to Calypto’s Aitelli, most SoC designers find these sleep modes too hard to use, so they simply don’t use them. They don’t have the time. But those sleep modes are still there just waiting to be used. PowerPro MG will add the necessary sleep/wake-up state machine to exploit this little-used memory feature. Push the button, save power.

Just a story from a chance meeting at DAC. Par for the course. There’s always something new to learn, something new to try.

To read my blog on the Low-Power Report Card Panel at DAC, click here.

But back to Furber. Today, he’s the ICL Professor of Computer Engineering at the School of Computer Science, Manchester University, UK and his CV sports a long list of impressive achievements. Let’s just say he’s been busy since leaving ARM. These days, he and his group at Manchester University are developing digital ways to emulate organic brain functions. In essence, his group is developing digital analogs of neural networks. Now electronic neural networks aren’t something new. I can remember discussing them when I was a college freshman. That was 1971. Not new. Not recent.

The Manchester University team is developing an SoC with a “massively” parallel network of eighteen ARM 968 RISC processors all mutually interconnected through a Silistix self-clocked network on chip (NoC). Furber had a hand in the early development of this NoC, also at Manchester University. (See, he’s been busy, like I said.) The project is called SpiNNaker. (http://apt.cs.man.ac.uk/projects/SpiNNaker/)

Now there’s a reason for repeatedly emphasizing Furber’s connections to Manchester University and he discussed it in his keynote. Any serious discussion of the history of computing must include the Manchester University Mark I “Baby,” which was the first fully programmable, stored-program digital computer to go online. Baby executed its first program in 1948. ENIAC, developed at the Moore School of Electrical Engineering at the University of Pennsylvania and usually called the first fully electronic computer, was operational two years before the Manchester Baby. But ENIAC was physically programmed with wires—at least initially. Eventually, ENIAC was retrofitted with some programmability but the Manchester Baby was first.

When operational, the Manchester Baby computer executed roughly 800 instructions per second. That was a heck of a lot faster than the mechanical calculators and punched-card equipment of the day but it’s laughably slow when compared to today’s processors. (Even the Intel 4004, the world’s first commercial microprocessor introduced in 1971, executed 108,000 instructions/second.) More to the point for the purposes of this blog, the Manchester Baby consumed approximately 5 Joules of energy to execute each instruction.

Fast forward to today and those ARM 968 microprocessors in the SpiNNaker chip. An ARM 968 processor executes roughly 20 million instructions per second, dissipating 10^-10 Joules per instruction. In other words, the per-instruction energy consumption needed to execute a machine instruction has improved by a factor of about 50 billion in 60 years.

Now the old, worn comparison usually asks you to consider what the world would be like today if automobile manufacturers had improved the energy consumption of their products by a factor of 50 billion in 60 years. That’s not the point here.

Furber’s point is this: if the energy cost per instruction had not improved by such a huge amount since 1948, this world would be a very different place. There would be no cell phones, no iPads, no personal computers, no personal music players, and very few embedded systems of any sort. These would simply be impractical for reasons of all three “P”s: price, performance, and power.

We have relied almost exclusively on Moore’s Law to get to this point.

That ride’s over.

At today’s bleeding-edge IC fabrication process lithographies, 28nm, we’re imaging individual atoms. Layers are a handful of atoms thick. The number of atoms in a transistor is so shockingly few that dopant atoms no longer operate statistically. The resulting on-chip parametric variability is becoming a very real problem that forces physical designers to use bigger and bigger guard bands on design rules. Speed and power gains are slowing from IC generation to generation. We have arrived at the point of rapidly diminishing returns and we’re clearly not getting another factor of 50 billion improvement in the power needed to execute a machine instruction from here on.

Yet the guidepost pointing to lower power operation is frustratingly close and familiar. It sits between your ears. We have chosen to design processors that execute one (or perhaps a few) instructions at one time, but at a very high execution rate. The higher the better. The brain is designed with an entirely different approach. It’s a highly parallel machine where “parallel” means a lot more than 18 processors. The brain contains approximately 10^11 neurons with 10^15 synapses. The neurons are the brain’s processors and the synapse connectivity is the brain’s memory and programming.

Neurons are very simple and very slow processors, but there are a lot of them working in parallel.

The entire brain human operates at roughly 100W—about the power consumption of a PC processor—but the brain runs at 100Hz. Although we can certainly get a lot of processing done with 100W, it’s not a drop in the bucket compared to the brain’s audio and visual processing abilities, let alone its ability for abstract thought. And we can’t get anything done at 100Hz. Our programming models cannot currently accommodate brain-style processing. We do not yet understand parallelism on the brain’s scale.

In addition, our processing systems are remarkably intolerant of failure. Microprocessors represent single-point failure nodes in most embedded designs with a few exceptions such as majority-voting avionics systems where single-point failure usually means death, so we go “massively” parallel with three processors.

The brain however is very tolerant of failure. Our brains lose neurons all the time. In fact, some of us hurry that process a bit by regularly drinking alcohol and killing off a few extra neurons a day. So what? When you’ve got 10^11 neurons, you’re not going to miss a few and of course the brain doesn’t.

The goal of the SpiNNaker project is to create an early parallel platform that will allow brain researchers to study the operation of a machine that can digitally emulate mechanisms that the brain uses to process a wide range of sensory data, to control an incredibly complex system of muscles and organs, to deal with the complex issues of written and spoken language, and to make huge leaps in abstract thought. SpiNNaker will not produce a leap by a factor of 50 billion, but perhaps it will get us going on the right path, now that we’ve managed to come this far.

Schedule:

Thursday, April 7, 2011

• 8:30 AM: Check-ins and On-site Registration
• 9:00 AM: Morning Keynote Speaker
• 10:00 AM: Parallel EDA
• Noon: Keynote Speaker
• 1:30 PM: High-Level Design
• 4:30 PM: Cloud Computing

Friday, April 8, 2011

• 9:00 AM: Low-Power Design
• 11:30 AM: 3D ICs

This is an inexpensive conclave with an emphasis on discussion and networking. It’s only $280 for IEEE Members; $100 for unemployed IEEE members. Save $50 through March 18th.

Full information and registration: www.eda.org/edps