AgigA Tech’s servers sit in a small room in the company’s San Diego corporate headquarters. One recent Saturday, these servers were quietly doing their jobs when an unscheduled power outage occurred. Thanks to uninterruptible power systems backing up the servers, the servers continued to do their jobs. Unfortunately, the air conditioning system at AgigA Tech headquarters has no backup power and did not continue to do its job. The temperature in the server room at AgigA Tech started to climb from the waste heat being thrown off by the equipment.

There was no one in the building to notice.

Eventually, power from the grid came back on and the cooling system restarted. However, by that time the temperature in the server room had climbed high enough to cook six of the hard disk drives in the server room’s RAID arrays. Fortunately, the RAID arrays performed as designed and there was no data loss. But if the power outage had lasted longer and if the servers had continued to run without cooling, eventually all of the RAID drives would have died. Even the best RAID system cannot preserve data when all of its drives fail. As it stands, even the hard disk drives that did not fail are suspect because they’ve been heat stressed. They too need to be replaced quickly before they fail prematurely as well.

After this small weekend near-disaster, my friend Ron Sartore started to ponder the ramifications and lessons of the incident. First, he realized that his servers should be sensing ambient temperature and shutting down gracefully when the server room overheats. In fact, Ron sort of assumed that’s what would happen. Bad assumption, as it turned out. “How often do you test that?” asked Ron. “No one wants to pull the plug on these things to find out” he added. I’ll bet a lot of business owners with little server rooms make precisely the same assumptions that Ron and his IT team did when they designed their server room.

Next, Ron began to think about the uninterruptible power supplies from an engineering perspective, which really calls into question the entire concept of uninterruptible power supplies for servers. There’s clearly no reason to continue to operate servers during a power outage if there’s no cooling available. In fact, there’s a clear reason to shut them down as quickly as possible to prevent overheating and hardware failure due to lack of cooling.

Now large data centers—like the ones operated by Amazon, Microsoft, and Google—have on-site Diesel generators that operate both the servers and the data center’s cooling systems during a power outage. These companies cannot afford to have their servers shut down. Every minute—actually every second—that the servers are down means lost revenue, lost profit, and lost customers. But most companies are not Amazon, Microsoft, or Google.

There are hundreds of thousands of companies in the US and millions in the world that run their servers in small server rooms or even closets where there are uninterruptible power systems designed to keep servers running as long as possible but where there is no backup power for cooling. For these companies, their server-system designs will cook and kill their hard drives rather quickly in the event of a power failure. We design smaller server systems this way almost without thinking. The UPS is a check-box item, meaning we don’t even think critically about including one.

Ron’s story made me think about UPS costs—both the acquisition cost and the operating cost over the life of the UPS. You see most uninterruptible power systems are designed so that they always supply power to the attached servers. Even while a UPS is running from the power grid, it’s still consuming and wasting energy. Today’s best UPS designs are perhaps 95% efficient. That means they consume about 5% of their input energy. All…the…time.

Less expensive UPS designs might be only 90% efficient. They waste about 10% of the energy consumed by the attached server(s).

In reality, a UPS that’s not operating at maximum load is likely to run at somewhat reduced efficiency because UPS manufacturers tend to rate their products’ efficiency at maximum load. In addition, IT managers prefer to overspecify the capacity of a UPS, which is normally good engineering practice but here it pretty much guarantees that the UPS will not operate at maximum efficiency.

Where does that wasted energy from the UPS go? It’s converted to waste heat, of course. Ron told me that his home UPS (purchased from Costco) runs hot even though it is lightly loaded—13W for a small PC appliance. Lightly loaded, a UPS might convert just as much power into waste heat as it delivers to the load.

And where does the waste heat from the UPS go? Right into the cooling system, assuming the cooling system is operational. During a power failure, it probably isn’t.

How much energy is needed for cooling? It depends on the cooling system. I’ve often heard from various presenters talking about energy needs for cooling data centers that it’s a 1:1 ratio—for every Watt of power emitted by equipment as waste heat, you need another Watt to remove the waste heat. Ron tells me that many of today’s cooling systems are actually more efficient than that. Some need only half a Watt to remove a Watt of waste heat from the server room—K factor of 2 for you HVAC engineers. Some high-efficiency cooling systems achieve a K factor of 3. So you can multiply the waste heat generated by a UPS by a factor of between 1.33 and 2 to determine the actual energy cost of UPS inefficiency.

Finally, we get down to computing the actual costs for using a UPS in a server room. Let’s start with a 2700W Dell UPS Short Depth Rack High Efficiency Online power-backup unit. When I looked it up, the purchase cost for the UPS was $1360.99 plus $115.68 for tax for a grand total of $1476.67. Shipping, at least, appears to be free. The UPS has a 3-year expected life and is rated as 95% efficient. About 5% of the energy it consumes is converted to waste heat when it’s fully loaded.

How much will it cost to run this UPS over its three-year expected life? Let’s use northern-California electric rates for commercial/general service from Pacific Gas and Electric where I live. PG&E charges about $0.20/kWh in the summer and about $0.15/kWh in the winter. On average, that’s about $0.175/kWh over the course of a year. Waste heat from the 2700W Dell UPS is about 135W (95% efficient) with another 70W or so needed to remove the waste heat through the air-conditioning system (assuming a K factor of 2). Total power required to have the UPS online all the time in case of a power-grid failure is about 200W continuously because the power for the servers is always flowing through the UPS.

Do the math and it works out to about $0.84 per day just to run the UPS, which is $306.60 per year in electricity cost for power-failure insurance. Over the three year rated life of the UPS, you’ll spend another $919.80 to run this UPS continuously—nearly as much as you spent for the initial UPS purchase. The total cost of adding this UPS to your system is about $2400 every three years using a back-of-the-envelope sort of calculation. If the UPS or air-conditioning systems are less efficient than the ones used in this example calculation, then the energy costs will be higher.

Now let’s be crystal clear here. Ron Sartore isn’t professionally disinterested in this story. He’s not exactly objective. His company, AgigA Tech, makes a line of AGIGARAM DDR2 and DDR3 memory modules that combine DRAM and NAND Flash memory with a controller on board that can independently transfer data back and forth between the module’s DRAM and NAND Flash without going through the server processor. In the case of a power outage, the on-board AGIGARAM controller backs up the data in the DRAM and puts it into the NAND Flash on the memory module using a small amount of standby power supplied by an independent bank of ultra capacitors connected directly to the memory module. Once backed up, the server data is safely stored for 10 years in the NAND Flash even without power. Standard servers and server-management software aren’t designed to exploit the features of this type of server memory that can safely back itself up. So even AgigA Tech’s IT department can’t configure a standard server system using AGIGARAM. At least not yet.

Bottom line, you or your customers may well be spending thousands of dollars for acquiring and powering a server UPS but you will not get the power-failure insurance you expect. You will not get a system that protects your data very well in the event of a power failure, as AgigA Tech has discovered. Instead of reliable backup, equipping a small data center with a UPS can cause hard disk drives to fry should there actually be a power failure—as they did at AgigA Tech—and it costs thousands of dollars extra in UPS costs to allow this to happen. “We paid good money to self-destruct ourselves” said Ron. Even though no data was lost in this example because the drives that failed were installed in RAID arrays, this approach seems like very poor engineering design. Ron has convinced me.

PS: While writing this blog entry, I received a letter from the IEEE. The envelope prominently featured this statement on the outside: “The BEST project plans include dependable backups for ‘out-of-the-blue’ accidents.” Although it might appear that the IEEE was thinking about this very blog entry when it mailed this letter to me, it turns out they’re just trying to sell me accidental death and dismemberment insurance. However, the coincidence is uncanny. In reality, that’s exactly what we’re discussing here—reducing the cost and improving the effectiveness and energy efficiency of accidental death and dismemberment insurance for servers.

The SpiNNaker project is again in the news at EETimes Europe (see “A million ARM cores to host brain simulator”) and the idea of harnessing one million ARM processor cores is certainly a big idea. It excites me. However, we’re still at the humble beginnings of the project.

The SpiNNaker project’s first test chip harnesses 18 ARM9 cores on one 130nm chip manufactured by UMC in Taiwan. This is a 100M-transistor chip and, like most many-processor SoCs, the SpiNNaker SoC mostly consists of memory. The memory needs to be close to the processors for speed and for low-power consumption and there are 55 32Kbyte SRAM blocks on the SpiNNaker die. That’s 14 million bits of SRAM and, frankly speaking, that’s really not very much SRAM. Eighteen processors isn’t really a large number of processors either when your stated goal is one million.

The ARM processors on the SpiNNaker chip use packet communications to emulate the electrical spike communications that occur among the neurons in human and animal brains. From a hardware perspective, I think it’s easy to conceive of a system-level design like this and even conceptually scaling the design to a million connected ARM9 processors isn’t really hard, as long as you don’t try to enumerate all of the processors in your mind. However, with 18 processors per chip, you’ll need approximately 55,600 chips to build an interconnected network of one million processors. That’s still a mighty big box of hardware. More on that in a bit.

The rub is that we really don’t have many good ideas for programming such a massively parallel system. The SpiNNaker project seems to be mostly a hardware endeavor with the explicitly stated intent of developing a hardware testbed for brain researchers who will use SpiNNaker systems for studying various theories of brain function. Presumably, we’ll learn more about massively parallel programming by working with these systems and no doubt we will. As Furber says in a quote published in the EETimes Europe article, “We don’t know how the brain works as an information-processing system, and we do need to find out. We hope that our machine will enable significant progress towards achieving this understanding.”

Each SpiNNaker chip in the current design is bundled with a 166MHz, 1Gbit DDR SDRAM and packaged in a 300-pin BGA package. But we’re not going to be building million-processor testbeds with 18 processors per packaged chip. I’m almost absolutely, positively certain about that. This first SpiNNaker prototype just doesn’t scale to one million processors very easily. So the question is, how to get there?

Well, possible clues to answer that question can be found in two recent blogs that I wrote on the EDA360 Insider blog. First, Samsung has just announced successful tapeout of a 20nm test chip incorporating an ARM Cortex-M0 processor core. (See “Samsung 20nm test chip includes ARM Cortex-M0 processor core. How many will fit on the head of a pin?”) Now an ARM Cortex-M0 processor is not as powerful as an ARM9 processor, but then it’s not supposed to be. It’s designed for control-oriented applications and its 3-stage execution pipeline isn’t designed to get maximum speed from any given process technology. However, we’re building a system that emulates a brain that operates at a few hundred Hertz (that’s Hertz, not kilohertz, megahertz, or gigahertz) so I really don’t think the clock speed is all that critical when you’re talking about a million processors. The ARM Cortex-M0 processor core is still a 32-bit RISC processor and I am guessing with a high degree of confidence that it’s fully up to the task of executing the required electrical-spike calculations, albeit not quite as quickly as an ARM9 processor.

What’s interesting about a 12-to-14Kgate ARM Cortex-M0 processor implemented in 20nm process technology is that my calculations suggest that more than half a million ARM Cortex-M0 processors would fit on a chip the size of an Intel “Tukwila” Itanium processor (OK, that’s a big chip, but it’s a commercial one) and that calculation is based on the published number for the area required by an ARM Cortex-M0 implemented in 90nm process technology, not 20nm. Now there’s a lot of slop in this calculation. First, there’s the disparity of using 90nm numbers instead of 20nm numbers. Then there’s the disparity caused by putting no memory at all into the calculation. I just mentally tiled processors edge to edge. Ditto, there’s no on-chip interconnect.

So you probably won’t get half a million ARM Cortex-M0 processor cores on one 20nm chip. But you might get 100,000 or 200,000 ARM Cortex-M0 processor cores on a chip along with an interesting amount of memory and the required interconnect. Now we’re talking about only a handful of chips to get to one million processors. We’re talking about a tabletop box. Now we’re getting into the realm of the feasible for million-processor systems.

The second related blog entry I recently wrote in EDA360 Insider that also bears on this very interesting endeavor was about an announcement from Imec, a global research company. Just days ago, Imec announced that it and its partners successfully assembled a custom logic chip with two DRAMs in a stacked 3D configuration. (See “3D Thursday: IMEC prototypes 3D chip stack, finds some thermal surprises”.) This 3D stacked-chip prototype allowed Imec to test out some process ideas for manufacturing 3D stacked chip assemblies and to make some critical thermal tests to verify thermal models that will be so necessary when 3D assembly goes mass market. The 3D chip stack uses copper-tin micro-bumps and compression bonding for the electrical and mechanical assembly of the chip stack and you can see photos of the assembled stack below.

Here’s a photo of the overall chip stack:

And here’s a close-up of the edge of the chip stack to show the three stacked die.

The 3D Stack’s base chip is approximately 750µm thick. The two top components in the chip stack are each 25µm thick. There’s more technical info in the referenced EDA360 Insider blog.

I am convinced that 3D stacking of logic and RAM chips will be absolutely essential to developing massively parallel, low-power systems like the ones envisioned by the SpiNNaker project. First, the only way to feed data and instructions to massively parallel processing chips is through large amounts of on-chip memory and through high-bandwidth, low-energy channels connected to large off-chip memories. 3D assembly techniques permit both Wide I/O and high-speed serial I/O channels to work most effectively and at minimal energy levels and I expect to see rapid adoption of 3D assembly—even and perhaps especially in high-volume, cost-sensitive applications such as mobile phone handsets—in the next few years. This is precisely the sort of manufacturing technology we require to think seriously about million-processor systems.

Now all we need to do is figure out how to program them.

At today’s EETimes Virtual SoC event Marc Greenberg from Cadence pulled back the veil on DDR4 a bit more. Here’s what he had to say.

First, even though we don’t have a final specification, some details are public. DDR4 SDRAMs will have double the maximum capacity of DDR3 SDRAMs. They’ll also have twice the maximum clock frequency. Like DDR3 SDRAMs, DDR4 SDRAMs will have an 8n prefetch (important for cache-line-filling operations) but a DDR4 memory controller must alternate or rotate between SDRAM bank groups for maximum SDRAM performance. That’s a new restriction.

The DDR4 I/O voltage has been reduced to 1.2V—DDR3 SDRAMs use 1.5V—so you can expect that the DDR4 SDRAMs will consume less power and energy than DDR3 SDRAMs simply from the lower operating voltage and from the more advanced process technology. However, Greenberg warned that some systems might not realize such savings due to architectural issues. In addition, DDR4 SDRAMs will not use stub-series terminated logic drivers. Instead, they’ll use pseudo-open drain (POD) drivers with Vdd terminations. DDR4 memories also have new features to improve signal integrity. They’ll use data-bit inversion (DBI, more on that below), on-chip parity detection for the command/address bus, and CRC error detection for the data.

Because of the higher maximum clock rate, DDR4 memories may permit a pin-count reduction for some SoC designs. How? At double the clock rate, SoC designs can get the same data bandwidth with 16 data bits clocked at 1600MHz (3.2 Gtransfers/sec) as DDR3 designs get with an 800MHz clock rate. However, there’s a design caveat or two. First, SPB (silicon, package, board) design for DDR4-3200 SDRAM is going to be considerably harder than for DDR3-1600 SDRAM. In addition, most memory experts predict that designs with multiple DDR4 DIMMs on each memory channel will not be able to work reliably (or at all) starting with data transfer rate considerably below the 3.2 Gtransfers/sec maximum. Similarly, DIMMs with multiple memory ranks on the board may also fail before the data transfer rate reaches 3.2 Gtransfers/sec.

There are a couple of possible solutions to these DDR4 signal-integrity challenges. The first and simplest solution is to allow only one DIMM slot per DDR4 memory channel and allow only single-rank DDR4 DIMMs. The problem with this solution is that it increases the number of SoC memory channels for a given memory capacity and thus drives up the SoC’s pin count, cost, and board-level real estate.

No one likes any of those consequences. Not at all. So an alternative solution is the use of load-reduced DIMMs (LRDIMMs) as shown in the following figure.

Now you may be familiar with RDIMMs (registered DIMMs) used in servers. RDIMMs have an extra register chip soldered to the board that stores and buffers the address/control information from the memory controller and distributes that information to the memory chips on the DIMM. LRDIMMs also buffer the data lines to present a single load to the memory controller even when multiple memory ranks are soldered to the DIMM. RDIMMs and LRDIMMs increase memory latency, so the DDR4 controller must be able to understand and accommodate this kind of buffering.

Finally, in the what-we-know category, DDR4 SDRAMs will stay with the 8n prefetch used for DDR3 memories but they will add an extra level of multiplexing so that the memory controller must manage traffic to and from the SDRAM even more carefully than before to extract maximum performance from the device.

Here is where we leave the known DDR4 world and enter into the realm of conjecture.

Although there are no public details on how DDR4 SRAM’s extra multiplexing level works, GDDR5 memory already employs a bank-grouping scheme with an extra level of multiplexing. GDDR5 memory adds new command timings that differ depending on whether successive commands address the same or different bank groups. These extra timings mean that a DDR4-optimized memory controller must be a bit more complex than the controller used for DDR3 memories. The controller needs better command scheduling and it must deal even more efficiently with high-priority memory commands. The Cadence DDR4 memory controller that was just introduced last month has several new features to accommodate the new complexities of the upcoming DDR4 memory protocol said Greenberg.

Here’s a table of enhancements made to the controller’s command queue to accommodate DDR4 requirements and maximize memory-subsystem performance:

One key feature here is a new command-prioritizing scheme that prioritizes DDR4 commands when they enter the command queue (like the DDR3 version of this controller) and then reprioritizes the commands when they’re about to exit from the queue, to be issued to the DDR4 memory. That part’s new. This new feature allows high-priority commands to go straight to the head of the command queue when they’re received, but controller can delay the command’s exit from the queue (and the issue of that command to the memory) until the target DDR4 memory page and bank are ready to accept that command. This capability reduces the impact of high-priority commands and helps to maximize memory bandwidth and throughput.

Another new controller feature is support for DBI. The following figure illustrates the problem:

The left side of this figure shows four consecutive data transfers. In the first transfer, all of the data bits are “1.” In the second transfer, they’re all “0.” As a result, all data bits change state from the first transfer to the next. This is a bad thing, especially at multi-GHz transfer rates. The effects of capacitive charge and discharge for all data lines at high speeds creates a problem called simultaneous switching output (SSO), which stresses the DRAM’s power-distribution system on the chip, in the package, and on the board. The next transfer shows a transition from all zeroes to all ones except for one data bit. Because of capacitive coupling, the data lines making the zero-to-one transition become aggressors that try to induce that lone holdout bit to also make the transition even though it does not want to do so. The fourth transfer exhibits a similar problem. All of the bits make a transition but one bit steadfastly wants to make the transition in the opposite direction. Again, it’s up against a number of aggressors.

The way to solve this problem, implemented in GDDR5, is to add a DBI bit. The right side of the figure shown above illustrates the same state transitions, but with the addition of a DBI bit. When asserted, the DBI bit indicates that the data bus should be inverted. The inversion state can change from transfer to transfer and it is changed to minimize the number of data-bit state changes and thus minimize the I/O switching current and the number of aggressor bits from one transfer to the next. Again, this is how it’s done for GDDR5 memory. The DBI method used for DDR4 SDRAM is not yet public.

With these and other changes to the memory interface specification, SoC designers will need a new tool set to add DDR4 memory interfaces to their designs. That’s why Cadence has introduced a DDR4 IP package and design kit now—because SoC designers preparing early designs that incorporate DDR4 memory need to start now. The Cadence DDR4 offerings include a DDR4-enhanced memory controller (based on the existing, configurable SDRAM controller Cadence obtained when it purchased Denali Software last year), hard and soft DDR4 PHYs, design kits for board- and package-level DDR4 design, and verification IP and memory models for DDR4 memory.

Over the next several years, we will see DDR4 SDRAM gradually enter and then take over the SDRAM memory market. It’s happened with the DDR, DDR2, and DDR3 SDRAM generations and there’s little reason to believe this won’t happen with DDR4 as well. Greenberg said to expect to see designs from early adopters who need maximum memory subsystem possible performance in 2013, early majority adoption for high-performance (but not the most bleeding-edge) designs in 2014, majority adoption in desktop and laptop PCs in 2015, and then pretty much total market penetration all the way down to low-cost devices by 2016.

Time to get going, isn’t it?

Performance isn’t really the engine that drives DDR3 adoption. The real driver is bandwidth and there are two design trends that force the quest for ever-increasing amounts of memory bandwidth. The first such design trend is the wholesale adoption of homogeneous and heterogeneous multicore architectures. As an industry, we’ve embraced the use of multiple processor cores as a solution to the death of Dennard scaling. Although most people attribute the increase in operating frequency and the decrease in per-transistor power consumption through lithographic shrinks to Moore’s Law, which Gordon Moore codified in an article he published in 1965 while working at Fairchild Semiconductor, that attribution is not factually correct. Moore simply predicted that the number of transistors on a chip would grow exponentially over time as lithographies shrank. It was IBM’s Robert Dennard who observed in 1974 that lithographic advances in IC manufacturing also consistently produced faster transistors that consumed less power. For decades, we’ve used Dennard scaling to produce faster and faster processors (while attributing the improvements to Moore’s Law).

The semiconductor industry has poured billions of dollars into keeping Moore’s Law alive but Dennard scaling died at 90nm. We continue to get more transistors on a chip with each advance in IC lithographic scaling, but the transistors no longer get appreciably faster, so the MHz wars have ended. Worse, pushing transistors to their performance limit now produces leaky transistors that dissipate as much power when off as when on. We now recognize that the way to get more performance is to use the transistor bounty to increase the number of processors and to distribute the work load across these processors without striving for multi-GHz clock rates.

With all of these on-chip processors executing code and accessing data on a multicore chip, system designers must find a way to make large amounts of inexpensive memory available to these processors. For the last decade, the most cost effective way to provide a system with large amounts of low-cost memory has been the SDRAM. The classic system design teams a multicore processor or SOC with one or more SDRAM channels. As memory bandwidth needs rise, the SDRAMs’ per-channel transfer rate and the number of SDRAM channels used has increased. DDR transfer rate have now reached and exceeded 1600 Mtransfers/sec and it’s not uncommon to find server processors with three SDRAM channels, for example. Because of the constant thirst for memory bandwidth, DDR3 SDRAM sales exceeded DDR2 SDRAM sales beginning with the first quarter of 2010, according to the leading SDRAM vendor Samsung, and the company expects DDR2’s share of SDRAM market sales to drop below 20% by the end of the year.

When you move that much data between a processor and memory, you’re likely to dissipate a considerable amount of power and indeed, memory power consumption has been on the rise. Lowering memory power consumption can substantially lower system-level power consumption. For example, states Samsung, going to 40nm, 2-Gbit DDR3 SDRAM with a 1.35V power supply can cut a server’s memory power consumption by 80% compared to the equivalent number of storage bits implemented with 60nm, 1-Gbit, DDR2 SDRAMs running at 1.8V and can even cut memory power consumption by 38% compared to equal-sized memory arrays consisting of 60nm, 1-Gbit, DDR2 SDRAMs running at 1.5V.

As a result, according to Samsung’s measurements, 40nm, 2-Gbit DDR3 SDRAMs running at 1.35V can cut power by an astonishing 38% at the system level for servers. To put that into economic perspective, says Samsung, the use of 1.35V DDR3 SDRAMs in a server can save 2564 kilowatt-hours per year. Samsung estimates that there will be 32 million servers operating in data centers worldwide by the end of this year. If they all were equipped with 1.35V DDR3 memory, the annual power consumption would be reduced by 82 terawatt-hours, worth an estimated $28 billion. That kind of money gets any data-center manager’s attention.

The same sort of energy savings apply to any multicore system whether it’s a server, a PC, or an embedded system based on a heterogeneous multicore processor design.

Not coincidentally, that’s the position of the SPMT (Serial Port Memory Technology) Consortium, which has been developing a low-power, high-performance memory interface for mobile and embedded applications. The low-power aspect arises primarily from SPMT’s use of low-voltage differential signaling (LVDS), which transfers information using 150 mV differential signal swings instead of single-ended, ground-referenced signal swings of more than a volt. The high-performance aspect arises from the use of multi-Gbits/sec transfer rates per SPMT data lane.

But there’s been a big, ugly fly in the SPMT ointment. Memory vendors know that more than 80% of all DRAMs go into PCs and servers and they stick with memory designs—and memory interfaces in particular—that best suit the needs of PC and server designers. Today, that means DDR2 memory, which is the mainstream DRAM technology, but the industry is quickly switching to DDR3. DDR4 is yet undefined but it too is a rapidly approaching memory-interface specification that will most assuredly ”fix” the problems we have with DDR3. These PC- and server-centric, high-speed parallel SDRAM interfaces burn a lot of power to deliver high bandwidth, which creates the niche opportunity that the SPMT Consortium has been trying to fill for mobile and embedded designs. Unfortunately, DDR memory has such a huge presence in the DRAM arena that there’s been little chance for any other interface approach to take hold.

Until now.

Today, the SPMT Consortium announced a major revision to the SPMT standard that may well spell the difference between an interesting technical exercise and an immensely successful new memory-interface standard. Previously, the SPMT specification multiplexed read/write commands and the data on the same unidirectional LVDS lanes. Doing so somewhat reduced the throughput on the data lines but it also reduced the memory pin count because SPMT memory didn’t need separate control/address (CA) lines. The reduced pin count was considered a major benefit that reduced the cost of packaged SPMT memory devices. The new SPMT specification, which completely supersedes the prior specification, does away with this control/address/data multiplexing in favor of using the same CA signal and pin definitions that LPDDR2 memory uses to carry control and address signaling.

This is a significant and important change to the SPMT spec because LPDDR2 is already poised to take over the mobile and embedded design spaces. (See LPDDR2: The new mainstream memory for embedded and mobile applications? on Denali Software’s Memory Report blog.) Further, four pairs of unidirectional SPMT data lanes now precisely overlap the 16 bidirectional data lines of a x16 LPDDR2 memory, making it possible to build one memory chip that can support both LPDDR2 and SPMT protocols using the same set of pins. What that means is that with only a few changes to the memory controller and memory PHY, an SOC or embedded processor can accommodate both LPDDR2 and SPMT memory using exactly the same set of interface pins. It also means that SDRAMs designed to the new SPMT specification can be used as LPDDR2 SDRAMs, ensuring a ready market when commercial SPMT SDRAMs first hit the market near the end of 2011—assuming things go according to the SPMT Consortium’s current plans.

So where’s the power advantage? It kicks in after the required SDRAM transfer rate hits a critical level. For example, the SPMT Consortium’s data estimates that a x32 LPDDR2 memory interface operating at 400MHz dissipates about 180mW while providing 3.2 Gbytes/sec of peak data throughput over 32 data lines (800 Gbits/sec/pin) and 360mW at a peak data throughput of 6.4 Gbytes/sec over 64 data lines. (Regular old DDR2 and DDR3 SDRAM interfaces would consume a lot more power than this.) By contrast, the SPMT interface dissipates 180mW while transferring 6.4 Gbytes/sec over eight data lanes (8 Gbits/sec/lane) and 360mW when transferring 12.8 Gbytes/sec over 16 data lanes. So the SPMT interface appears to be about twice as power efficient as the LPDDR2 interface at higher data rates, which LPDDR2 memory can’t attain without resorting to a very wide data bus and using several memory devices in the bargain. However the LPDDR2 parallel interface has a power advantage over the SPMT serial interface at lower transfer rates. So LPDDR2 memory might suffice for today’s embedded and mobile applications and might also suffice for low-activity modes in future applications.

The graph below, supplied by SPMT, tells the story. The graph shows that at low data rates, LPDDR2 memory dissipates less power than SPMT memory—largely because of the DLL integrated into SPMT memory. (DLLs consume non-negligable amounts of power and although DDR2 and DDR3 memories incorporate DLLs, LPDDR2 memory does not.) So the SPMT Consortium has done something very smart and has developed an integrated mode-switching mechanism called SerialSwitch, which allows an SDRAM controller to programmably shift an SPMT memory between its LPDDR2 and SPMT serial interface modes using a control register built into the memory device.

Mobile phone vendors and other embedded/mobile system designers know that video will be heavily used in many future products and they also know that memory transfer-rate and bandwidth requirements will only go up as a result. SPMT’s SerialSwitch mechanism provides a way for one memory device to support both low- and high-bandwidth operating modes with an appropriate level of power consumption depending on a system’s instantaneous bandwidth requirements. By definition, all commercial SPMT memories will incorporate the SerialSwitch feature. The following figure shows how the SPMT SerialSwitch mechanism works.

During Tg, the figure shows SPMT memory operating as a x16 LPDDR2 memory. Note that the data lines (DQ/HS) employ full-voltage, single-ended signaling in this mode. During time Tg, the memory’s DLL is off, which saves power. At the beginning of time Th, the system determines that more bandwidth is or soon will be needed, so it directs the memory controller to send a command to the memory to spin up the DLL in preparation for switching to SPMT serial mode. That process takes 5 to 10 microseconds. During this time, the memory continues to operate as an LPDDR2 memory so the DLL spin-up time is hidden and doesn’t interfere with system operation but power consumption will rise. Once the SPMT memory’s DLL has spun up, at time Ti, the system’s memory controller commands the SPMT memory to switch to serial communications mode. This transition takes a maximum of 10 clock cycles. After that and during time Tj in the figure above, the memory operates in SPMT serial-communications mode. Note that the data lines have switched to LVDS signaling, as shown in the figure. LVDS signaling reduces the memory interface’s power consumption. At some later time depending on system requirements, the memory controller can power down the memory (shown as time Tk) or switch back to LPDDR2 mode (the period following the period that starts at time Tk in the above figure). Don’t be misled by this figure by the way—SPMT memory need not pass through the power-down mode to switch from SMPT-serial communications to LPDDR2 mode.

Systems can use SPMT memory in LPDDR2 mode at boot time and whenever the system is operating in a mode with low memory-bandwidth requirements. The system can quickly switch to the LVDS SPMT-serial mode whenever it requires higher memory data rates—for example when video is activated, when multiple operating modes are in use simultaneously, or when multiple processors are running in a multicore device. The SPMT Consortium estimates that the optimum crossover point between LPDDR2 and SPMT serial interface data rates for a x16/8-lane LPDDR2/SPMT-serial memory device is around 1.6 Gbytes/sec based on energy considerations.

By subsuming the LPDDR2 standard and making SPMT memories wholly superset compatible with LPDDR2 memories, I think the SPMT consortium has significantly raised the likelihood of adoption when commercial SPMT memories finally appear late next year. I also think the likelihood of such memories appearing is pretty high considering that the top two DRAM vendors, Samsung and Hynix, are members of the SPMT Consortium. Together, Samsung and Hynix have a bit more than half of the overall DRAM market according to the latest stats from the DRAMeXchange (http://j.mp/aNaNiY).

On the embedded processor side of the equation, Marvel has announced that it too has joined the consortium, which further improves SPMT’s chances of success. In fact, Marvell supplied a canned quote for the SPMT Consortium’s press release with one of the strongest statements I’ve seen in such press releases, so I am suspending my usual cynicism about such quotes and reproduce it here:

“Today’s mobile DRAM technology is geared to support the bandwidth needs of single core processors. As devices evolve to integrate multi-core CPU, multi shader 3D graphic engines at multi-GigaHertz speeds, it’s clear that DRAM will be the single performance bottleneck, especially for handheld systems where power budget is a major constraint,” said Dr. Sehat Sutardja, chairman, president and chief executive officer at Marvell. “Marvell is joining the SPMT Consortium to actively promote Serial Port Memory Technology as an industry standard and address the immediate needs of the industry. We encourage other companies active in the sector to join us in our mission.”

Strong backing like this from a market maker like Marvell can only help SPMT’s cause. Whether or not SPMT actually reaches critical mass is something that we’ll all be watching as events unfold in the hotly competitive memory arena over the next 18 to 24 months.

 There are two major components to Xilinx’s EPP: a hard-wired, high-end, microcontroller-like block and a connected FPGA fabric based on Xilinx’s 28nm unified FPGA logic-cell design as shown in the diagram below.

Xilinx EPP Block Diagram

First, let’s look at the hard-wired portion. It’s well known that processors don’t run very fast when implemented with FPGAs. The reason mostly revolves around the wiring congestion associated with the large register files of 32-bit RISC processors. Wiring congestion translates into “slow” and you can figure on giving up 50-75% or more of the processor’s maximum clock rate in a given process technology when comparing a synthesized ASIC implementation against a synthesized FPGA implementation. Hand optimization can reclaim some of that speed but if you’re planning on using a standard processor architecture anyway, it makes perfect sense to implement the processor on the FPGA as a hard core using a standard ASIC synthesis flow. That way, you get the full speed of the IC process technology along with the full logic density and therefore a much lower silicon cost.

Xilinx has chosen ARM’s Cortex-A9 32-bit RISC processor core for the EPP but has gone a step farther by implementing a dual-core version of this processor. That choice immediately puts the Xilinx EPP family at the high-end of the microcontroller spectrum. First, there are two 32-bit processor cores. Second, a Cortex-A9 processor can run at 2 GHz in TSMC’s 40nm, high-performance process technology. That’s one fast processor—much faster that many embedded applications require. A dual-core version, as is employed in Xilinx’s EPP family, is faster still.

In choosing a standard processor core from ARM’s extremely successful stable of processors, Xilinx has plugged directly into a broad community of embedded software developers. In other words, choosing the widely used ARM architecture telegraphs Xilinx’s recognition that embedded software development is now the largest and most expensive part of any high-end embedded project. In many such projects, software developers often outnumber hardware developers by 10:1. In announcing the EPP, Xilinx shows that it fully recognizes the need to make the software development team happy first. The company’s selection of an ARM processor core also leverages the associated large and familiar development-tool set, the good selection of operating systems, and the extended ecosystem that goes with the ARM architecture’s large and growing market dominance in the embedded space. All of these factors make the ARM processor very attractive to embedded development teams.

To the dual-core ARM Cortex-A9 processor, Xilinx has added a number of hard-core peripherals including SRAM caches, timers, interrupt controllers, switches, memory controllers, and commonly used I/O peripherals certain to be useful for many high-end embedded designs. Because these additional blocks are all hard-core implementations, they too take little room on the chip and consume much less power than they’d need if implemented in an FPGA fabric. Note that the EPP chips will contain enough SRAM for caches and small scratchpads however bulk memory, generally implemented with DRAM, will be off-chip. Consequently, the EPP architecture includes hard-core DRAM controllers to manage off-chip memory. Ratford’s talk at ESC did not elaborate on the type of memory the on-chip controller can handle however DDR2, DDR3 or both DDR2 and DDR3 would probably be a good guess, considering the high-end nature of the EPP family. The targeted applications will need a lot of memory and DDR2 and DDR3 DRAM are now the best choices in terms of cost/bit.

Key to the software-friendly approach Xilinx is taking with the EPP, the architecture boots code upon power up just like a microcontroller. Only then is the FPGA fabric configured. This approach makes the EPP look very familiar to software developers who are not at all comfortable with writing code for a fluid, amorphous system that’s not well-defined when power comes up. The FPGA vendors spent a lot of money on reconfigurable architectures learning this lesson. In addition, HLL compilers don’t much care for undefined hardware either—undefined hardware just doesn’t fit the standard software-programming models. So the implementation of a complete, hard-wired microcontroller within the EPP cuts out a lot of that old unfamiliar strangeness associated with previous attempts to marry hard processor cores and FPGA fabrics.

Speaking of the FPGA fabric, Xilinx will be using the unified 28nm FPGA fabric in the EPP. Xilinx developed this fabric for its next-generation Spartan and Virtex FPGAs. (If you want more details about this FPGA fabric, take a look at the White Paper here. According to Ratford, Xilinx’s Virtex and Spartan FPGAs will both employ this fabric, which is the first time that Xilinx has used the same FPGA fabric for its high-performance and its low-cost FPGA product families. Using the same fabric for the two Xilinx FPGA product lines and for the EPP means that Xilinx need only develop one set of hardware-design tools for the 28nm node and it also means that hardware designers only need to learn one set of tools as well.

The EPP’s hard-core embedded microcontroller communicates with the on-chip FPGA fabric using ARM’s newly announced AMBA 4/AXI bus. Ratford said at RTECC and repeated again at ESC that Xilinx worked with ARM to develop a version of this new bus specifically for FPGA use but he’s not provided details. The diagram of the EPP Ratford projected (reproduced above) shows multiple buses connecting the EPP’s hard-core embedded microcontroller and the on-chip FPGA fabric. Although Ratford provided no additional details, I plan to write a third blog entry discussing possible ways of optimally connecting the processor cores to the FPGA fabric. In the next installment of this blog, I’ll discuss some specific case studies Ratford covered in his ESC presentation that show how the EPP can reduce the parts count, cost, and the power consumption of high-end embedded systems.

(You can find a White Paper describing the Xilinx EPP here.)

To deliver competitive parts, LSI stays on top of IC design and manufacturing trends. One trend that caught LSI and the semiconductor industry by surprise has been the rapid growth in on-chip memory use. On-chip memory makes sense for two reasons. First and foremost, it provides better performance than off-chip memory because putting memory on the chip along with the logic circuitry eliminates two sets of off-chip drivers and receivers, which reduces power consumption for memory transactions. Second, on-chip logic can communicate with on-chip memory over extremely wide memory interfaces—pin count is not an issue if you stay on the chip. A wide memory interface reduces the number of transfers needed to move a given amount of data and lower transfer rates cut power as well.

However, merging logic and memory on one piece of silicon has always presented design and manufacturing issues. Bulk, high-volume, high-capacity memory manufacturing processes differ from logic manufacturing processes because the two processes must optimize different parameters. Memory processes emphasize low cost manufacturing and tend to have fewer metal layers than logic processes, which emphasize speed and on-chip connectivity. “Frequency, density, and power are always a challenge,” said Madge.

For example:

• Today’s network routers use 400-Mbit buffers. Switches need 512 Mbits of storage or more. In the future, said Madge, these devices will need as much as 1 Gbit of on-chip memory in multiple configurations.

• IP controllers used in network storage applications currently use 60 to 100 Mbits of cache memory. In the future, these devices will need 200 Mbits of memory or more.

• Media processors currently use 60 to 80 Mbits of memory running at 500 MHz. Future needs will be on the order of 100 to 200 Mbits of memory running at 600 to 700 MHz.

All of these examples demonstrate the coming challenges for fast, dense, on-chip memory.

LSI is looking at embedded (on-chip) DRAM and the use of 3D, through-silicon via technology for chip-to-chip stacking as ways of increasing the amount of on-chip memory. The company is doing this because it sees a continued and rapid rise in the amount of on-chip memory needed for its networking and storage chips.

Embedded DRAM cuts power because it uses a 1T (one-transistor) cell, which obviously improves density over a 4T or 6T static RAM cell. However, embedded DRAM also reduces static and dynamic power consumption because the fewer transistors use less power and leak less current than the greater number of transistors required to build the same amount of SRAM memory.

LSI is also investigating other power-saving features that become possible when you move memory onto the logic chip including a sleep mode for the memory, dual power rails, and low-voltage operation. However, said Madge, the biggest benefit appears to be a move to embedded DRAM because of the huge reduction in transistor counts.

I attended Denali’s ninth annual MemCon conference a few days ago. It was three days of intensive discussion about the state of the art in DRAM and Flash memory-the two mainstay memory technologies in use today. Surprisingly, NAND Flash memory is now the low-cost leader in terms of cost per bit, having passed by DRAM a few years ago. However, DRAM remains the mainstay memory for the vast number of designs and DDR SDRAM now rules as it becomes easier and easier to find microcontrollers and FPGAs with direct DDR interfaces and DDR controller and PHY IP for SOCs.

Memory power consumption as a percentage of system power consumption has grown with the rapid growth of memory-array size in all sorts of systems. A real eye opener at MemCon 09 was a chart on the power consumption of memory in server systems, where the large server memory arrays consume as much as 40% of the system power and the processor now consumes a mere 28%. Why is that important? It’s important because big server users like Google pay tens of millions of dollars each year in electrical power costs to run and to cool their server farms and 40% of a few tens of millions of dollars is, well, tens of millions of dollars.

Note that the current share-of-power percentages for servers don’t make processor power consumption unimportant-28% is still a big number-but the clear message is that server designers must now be far more concerned with memory power consumption because it’s a big part of the power puzzle. As embedded designs adopt large DDR memory DIMMs for bulk memory, the same sort of situation applies. Embedded designers must also be aware of the way their DRAM choices affect system power.

Marc Greenberg, Denali’s Director of Technical Marketing, gave a 2-hour tutorial on low-power DDR SDRAM on the first day of MemCon09. He threw up one slide that does a terrific job of putting all of the low-power SDRAM parts in perspective:

Low-Power DDR Selection Criteria

This slide shows the optimum type of SDRAM to use based on your design’s memory-capacity and speed requirements. I like this slide a lot because it helps you to pick from the wide array of DDR types and speeds. However, it seems that your selection job is about to become a lot simpler. Look what happens to the chart when you add in LPDDR2 memory:

Low-Power DDR Selection Criteria with LPDDR2

LPDDR2 memory delivers the low-power goods by operating the SDRAM’s memory core and I/O at 1.2V, which is what you need to do to substantially cut memory power these days. Several manufacturers have announced LPDDR parts with I/O speeds to 400MHz/DDR800 and spec sheets for these parts are beginning to appear on DRAM vendor Web sites. LPDDR2 vendors with announced parts include Elpida, Hynix, Micron, and Nanya. Note that there’s also the possibility for existing LPDDR1 vendors to create parts that operate at 1.2V for similar power savings and that some of the soon-to-be-seen DDR3 parts may operate at 1.35V, which qualify them as low-power DRAMS.

In addition, there’s a spec for LPDDR2 non-volatile memory (LPDDR2-NVM) to allow LPDDR2 DRAM and Flash to be intermixed. The big advantage of Flash LPDDR2 is the very low standby power but Flash memory exhibits both read and write wear-out failure, so DRAM isn’t yet obsolete and you’ll likely need both memory types in your system design. The LPDDR2-NVM spec allows for I/O speeds to 533MHz/DDR1066 operation, but Greenberg says that the initial LPDDR2-NVM parts are likely to be slower than the maximum.