How to Make Smartphone Even Smarter?

The IT industry marvels like augmented reality and artificial intelligence, which marked technological utopianism in the science fiction movies during the 1970s and 1980s, are here now, enabled by a machine-learning technique called deep learning.

Deep learning algorithms—which date back to the 1980s—are now driving Google Now speech recognition, face recognition service on Facebook, and instant language translation on Skype. However, the companies like Facebook and Microsoft are using GPUs to run these algorithms, and they could move to FPGAs in a bid to acquire even more processing speed.

Not surprisingly, therefore, these cutting-edge technology services consume an enormous amount of processing power, which is handily available at the large data centers that these companies have. Now mobile is the next frontier where deep learning can bring unprecedented gains by processing sensor data available from smartphones and tablets and perform tasks like speech and object recognition.

A virtual brain on the phone

And that will inevitably require moving some of the processing power to personal devices like smartphones, tablets and smartwatches. On the other hand, traditional mobile hardware made up of CPU and GPU is computationally constrained due to large processing overhead required to run powerful artificial-intelligence algorithms.

Smartphone's New Smarts

So a new breed of processors is now emerging to bring these services at a much lower power to smartphones and wearable devices. Take CEVA-XM4, for instance, an imaging and computer vision processor IP that allows chips to see by running a deep-learning network trained to recognize gestures, faces and even emotions.

The CEVA-XM4 image processing core takes advantage of pixel overlap by reusing same data to produce multiple outputs. That increases processing capability and reduces power consumption; moreover, it saves external memory bandwidth and frees system buses for other tasks.

It's an intelligent vision processor for cameras, image registration, depth map generation, point cloud processing, 3D scanning and more. The CEVA-XM4 combines depth generation with vision processing and supports applications processing in multiple areas like gesture detection and eye-tracking.

Face recognition: CNN usage flow with Caffe training network

Socionext, a Japanese developer of SoC solutions, is using CEVA's imaging and vision DSP core to power its Milbeaut image processing chip for digital SLR, surveillance, drones and other camera-enabled devices. The first chipset of the Milbeaut image processor family—MB86S27—employs imaging DSP core's powerful vector processing engine and is aimed at next-generation camera applications such as augmented reality and video analytics.

CNN/DNN Deployment Framework

The task of building support for deep learning into chips for smartphones and tablets also requires a new breed of software tools for accelerating deep learning application deployment. And the company supplying XM4 vision processor has acknowledged this by launching the CEVA Deep Neural Network (CDNN), a software framework that provides real-time object recognition and vision analytics to harness the power of imaging DSP core.

CEVA claims that its deep neural network framework for XM4 image processor enables deep learning functions three times faster than the leading GPU-based solutions. Moreover, CDNN enables XM4 vision processor to consume 30x less power while requiring 15x less memory bandwidth. Case in point: a pedestrian detection algorithm running DNN on a 28nm chip requires less than 30mW for a 1080p video stream operating at 30fps.

It's worth noting that deep learning works in two stages. First, companies train a neural network to perform a specific task. Second, another neural network carries out the actual task. Here, CDNN toolset boasts CEVA Network Generator, an automated technology that enables real-time classification with pre-trained networks and automatically converts them into real-time network model.

Real-time CDNN application flow for face recognition

Phi Algorithm Solutions, a supplier of machine learning solutions, has optimized its CNN-based "unique object detection network" algorithm using the CDNN framework alongside CEVA-XM4 vision DSP core. The Toronto, Canada–based firm has been able to make a quick and smooth shift from offline training to real-time detection. Now the company's optimized algorithms are available for applications such as pedestrian detection and face detection.

The CDNN software framework supports complete CNN implementation as well as in specific layers. And it supports various training networks like Caffe, Torch and Theano. Moreover, CDNN includes real-time example models for object and scene recognition, ADAS, artificial intelligence, video analytics, augmented reality, virtual reality and similar computer vision applications.

The availability of intelligent vision processors like CEVA-XM4 and toolsets such as CDNN is a testament that deep learning is no longer an exclusive domain of large, powerful computers. The dramatic advances in deep learning have reached the smartphone doorstep, and smartphone is going to get smarter. The smartphone is now powerful enough to run deep learning.

Do 8 Cores Really Matter in Smartphones?

As the smartphone industry has begun to mature, one-upmanship among smartphone manufacturers and SoC vendors has bred a dangerous trend: ever-increasing processor core counts and the association between increased CPU core count and greater performance. This association originated as SoC vendors and OEMs have tried to find ways to differentiate themselves from one another through core counts. Some vendors are creating confusion, as phones today have core counts from 2 up to 8 and vary wildly in performance and, even more importantly, experience. One reason for this confusion is many users and reviewers have used inappropriate benchmarks to illustrate smartphone user experience and real world performance. As a result, we believe that some consumers are misled in their buying decisions and may end up with the wrong device and the wrong experience.

The 8 Core Myth...
The 8 Core Myth, also known as the Octacore Myth, is the perception that more CPU cores are better and having more cores means higher performance. Today’s smartphones range from 2 cores up to 8 cores, even though performance and user experience are not a function of CPU core count. The myth, however, will not be limited to 8 cores, as there are plans for SoCs with up to 10 cores, and we could even see more in the future.

Not All Cores Are the Same...
In some phones, users are getting Octacore designs with up to 8 ARM Cortex-A53 cores. These 8 cores perform differently than 4 ARM Cortex-A57 cores paired with 4 ARM Cortex-A53 cores in what is called a big.LITTLE configuration. Core designs vary wildly from ARM’s own A53 and A57 64-bit CPUs to Intel’s x86 Atom 4-core processors to Apple’s 2-core A8 ARM processor. All these processors are designed differently and behave differently across application workloads and operating systems. Some cores are specifically designed for high performance, some for low power. Others are designed to balance the two through dynamic clocking and higher IPC (instructions per clock). As a result, no two SoCs necessarily perform the same when you take clock speed and core count into account.

Through the different benchmarks, tools, and applications, we showed that CPU core count in a modern smartphone is not an accurate measurement of performance or experience. More CPU cores are not always better. We do acknowledge that having many smaller cores is one way to simplify power management, but these tests are not focused on power; they are focused on performance and user experience.

CPU core counts are not the way that phone manufacturers or carriers should be promoting their devices. CPU core count is only one factor in Android when the SoC has fewer than 4 cores. The marketing of core counts as a primary driver of performance and experience must end and be replaced with improved benchmarking practices and education

3D Xpoint and the Future of Memory

Existing solutions Logic systems have a memory hierarchy with each tier optimized for a particular task. A typical hierarchy is:

Registers - on chip with the cores. Stores intermediate results during calculations, runs at the core speed, have small capacity (for speed) and wide data paths (for fast loading and unloading of data). Typically kilobytes of SRAM.

Cache – typically on-chip with the cores (used to be off-chip). Stores recently completed results and program instructions and data that are predicted to be needed by the core in the near future, bigger than the registers making access to data somewhat slower but still cache needs to be fast. Typically megabytes of SRAM.

Working memory – typically off-chip, often as multiple memory specific chips. Working memory stores programs that are currently running on the system and stores data currently being worked on by the system, is slower than cache and there is typically a lot more of it so it needs to be less expensive. Typically a few gigabytes of DRAM.

Long term storage – typically off chip in the form of a hard disc drive or in some portable systems as NAND Flash memory. Long term storage is where programs and data are stored even when a system isn’t running, much larger in capacity than the other types of memory outlined above storing more data and programs than could be loaded into main memory at one time, must be non-volatile, typically much slower and much less expensive than main memory. Typically many gigabytes to a terabyte of NAND Flash or a hard drive.

The memory type used for each tier in the hierarchy is determined by a set of tradeoffs in the memory characteristics.

SRAM is the fastest memory and has excellent endurance so it is used for registers and cache even though it is volatile (loses values when the power is turned off), takes up a lot of area and is relatively expensive. DRAM is slower than SRAM and volatile but it is cheaper than SRAM and has excellent endurance so it is used for main memory. NAND is non-volatile and less expensive than DRAM but it is slow and has poor endurance making it only suitable for long term storage.

Memory Evolution All three forms of memory outlined above are facing scaling issues.

SRAM cell sizes continue to scale down as the technologies used to make processors move to smaller and smaller nodes, but the classic 6T SRAM cell is now often supplanted by 8T and 10T cells in the most critical paths.

The key issue in DRAM scaling is the capacitor. A minimum capacitance value is required in order to store values (electrons). In order to shrink the horizontal area of the capacitor very tall structures have been implemented and the dielectric has transitioned to high-k materials. The issue now is twofold, one is that capacitor heights are reaching practical limits, and two, achieving higher dielectric k values is becoming very difficult. There is a variety of high-k materials available with a range of k values, the fundamental problem is that as k values increase, the band gap of the material decreases increasing leakage. The solution to this has been the use of sandwich materials such the current ZAZ where a high band gap aluminum oxide is sandwiched between two layers of high-k zirconium oxide. The problem with a sandwich is the resulting k value is reduced by the aluminum oxide layer. There is a lot of work still being done on capacitor materials with aluminum doped titanium oxide looking promising but DRAM scaling likely only has one or two more nodes left. Current DRAM state-of-the-art is 20nm with possibly a 16nm and maybe a 12nm node in the future.

In NAND scaling the issue has been how to scale down while maintaining good control gate to floating gate coupling within a memory cell - without coupling to adjacent cells. Storing enough electrons is also an issue for further scaling and 2D NAND dimensions have gotten so small that very complex self-aligned quadruple patterning (SAQP) is required on several layers. In the NAND space 3D NAND has been introduced by Samsung as a 2D NAND successor. Samsung’s initial device stacked up 24 memory cell layers with a single bit per cell. Multiple producers are now introducing 48 layers devices with 3 bits per cell. 3D NAND is a new scaling paradigm where a 40nm technology is expected to be used with an increasing number of layers (eventually over 100) until a 1Tb NAND results. There is only one multi-patterning layer required in the device fabrication and the number of lithography layers stays essentially the same even as more memory layers and bits per device are produced. 3D NAND is so promising that Micron has announced 16nm will be their last 2D NAND generation and all their efforts going forward will be on 3D NAND.

3D Xpoint
At the introduction press conference Intel and Micron described their new 3D Xpoint memory as being 1,000 times faster than NAND Flash with 1,000 times the endurance. That is very impressive but still isn’t the speed or endurance required to replace DRAM. The density was described as 10 times as high as DRAM with a cost intermediate between DRAM and NAND.

The exact mechanism behind the 3D Xpoint wasn’t disclosed but each memory cell is a memory element that stores resistance values and a selector (1R1D). Existing memory cells store electrons placing a lower limit on how much the cell can scale, storing values as a resistance should remove those limitations. The memory cells are each located at the cross point between orthogonal bit and word lines. The initial device had two memory layers, further scaling of the device is possible in three ways:

1. Add more memory layers.

2. Scale dimensions in x and y.

3. Go from a single bit per cell to a multi-bit per cell.

Our estimates for the die size suggest a 4F2 memory cell where F is approximately 25nm. Adding more memory layers should be straight forward but will require at least two mask layers for each memory layer and likely more. At a 25nm feature size the mask layers will be relatively complex and expensive multi-patterning layers. Scaling in x and y will require even more complex multi-patterning schemes at least until an alternative such as sufficiently high throughput EUV is available. Multi bit per cell was also mentioned as an option but how easy this is to implement isn’t clear.

Our current calculations find that the bits per mm2 for the initial 3D Xpoint memory is lower than current state-of-the-art 2D or 3D NAND. We expect that with additional layers 3D Xpoint will exceed 2D NAND density but will lag behind 3D NAND. Furthermore, we believe 3D NAND has a cost scaling advantage over 3D Xpoint memory and will continue to be less expensive per bit.

Micron is positioning 3D Xpoint as a Storage Class Memory that sits between main memory and long term storage. We believe this memory will find many interesting applications but will not directly replace DRAM due to speed and endurance issues and won’t replace NAND due to cost.

This discussion of 3D Xpoint is an excerpt from a more detailed document we have produced for our Strategic Cost Model customers.

What’s Next?
3D Xpoint is an interesting memory technology and will likely stake out an applications space between DRAM and NAND, but there is still an issue with DRAM scaling. Micron Technologies’ road map lists a new memory A and new memory B. Memory A is the 3D Xpoint being introduced now with a second generation expected next year, but what is memory B expected in 2017?

There has been a lot of talk for several years about MRAM being the successor to DRAM. MRAM stores memory values as magnetism, not electrons and so can theoretically scale down very small. MRAM also has the potential speed and endurance to be a direct replacement for DRAM. To-date MRAM has been nowhere near the density or cost required to replace DRAM but what if memory B is a multilayer cross point memory with MRAM cells at each cross point? The ability to stack multiple cells up as memory layers might finally get MRAM to a competitive density and ultimately cost to replace DRAM. This is only speculation on my part but a 3D DRAM replacement seems like a logical future direction.

Intel 3D XPoint Storage

The SSD in your computer might seem pretty fast, but it’s about to be left in the dust by a new storage standard from Intel and Micron. After 10 years of research and development, the companies have announced 3D XPoint. It’s the first new type of storage to be created in 25 years and it’s reportedly 1,000 times faster than the NAND flash storage used in SSDs and mobile devices.

Intel is really pushing the speed angle, but 3D XPoint (pronounced cross-point) should also boost storage capacity dramatically. It’s 10 times more dense than the most advanced NAND architectures, meaning you’ll be able to get more bytes in the same physical space. Intel also says XPoint will be affordable, but there’s no telling if you will agree with Intel’s definition of “affordable” when the technology hits the market.

At the heart of XPoint is a new type of data storage mechanism. It doesn’t use transistors or capacitors like traditional flash storage. It’s composed of a lattice of perpendicular conductors stacked on top of each other. The memory cells sit at the intersection of these conductors and can be addressed individually bit-by-bit. The ability to quickly read small data clusters is what makes XPoint so fast.

XPoint is speedy enough that it could replace both non-volatile storage (your SSD) and RAM. In fact, it’s too fast for any current interface technology to keep up with. The first XPoint chips will connect to computers over PCI Express, but even that won’t have enough bandwidth to truly let XPoint shine. New motherboard technology will be needed to take full advantage of the XPoint in the future. Intel and Micron expect to make the memory available sometime next year, but it’s not clear if it will be ready for consumer applications right away.

Synopsys to Acquire Atrenta

There have been lots of rumors about potential suitors for Atrenta, not just recently but over the years. Since they operate at the RTL and IP level there is clearly potential for other companies than the usual suspects of Cadence, Synopsys and Mentor. Two that have been much-rumored were ANSYS (who acquired Apache a few years ago) and Dassault (who have some process management solutions and acquired Tuscany Design Automation [disclosure: I was on the board] right at the end of 2012). I've never heard it mentioned but another possible candidate might have been TSMC who have used Atrenta's SpyGlass solution as their "signoff" tool for IP qualification as part of their OIP ecosystem.

The financial details of the deal were not disclosed. But for sure it was not a fire sale. For a start, the current threshold for HSR filing, the so-called "$50M threshold", is actually $76.3M this year so we know that the price was at least that high.

However, since there were multiple companies rumored to be interested I think it is logical to speculate that the price will be at a reasonable multiple on their revenue, itself rumored to be running in the $60M range. At 3.5X revenue that would be $210M so I'll go with that as my guess. Final answer.

Of course Synopsys plans to integrate the Atrenta technology into their verification continuum, especially SpyGlass which has wide acceptance. Manoj Ghandi, who is the GM of verification, adds a bit of color (not much, to be honest):

Atrenta's demonstrated leadership in static and formal technologies is recognized throughout the EDA industry, and its technology is used by design and verification teams around the world. Synopsys expects to leverage this strong technology to further improve our Verification Continuum platform to address continually increasing verification challenges, and to support our ongoing R&D collaborations with customers in both verification and implementation.

Atrenta will be at DAC on booth #1732. Interestingly they just announced the date of their user conference in October. Of course that may take place, since under the rules for an acquisition like this the two companies are not really able to work together until the deal is closed (on the basis that if the deal is struck down then everything should go back to exactly how it was before). However, I think it is more likely that it will just get folded into SNUG.

The Synopsys press release is here.

Intel plans to buy Altera FPGA for a $16 billion deal

Currently, Intel supplies Atom and Xeon processors for computing solutions.By acquiring Altera, Intel aims to target data-intensive networking centers at major companies such as Google, Facebook, LinkedIn and Ebay. FPGA solutions are desirable because they increase performance and consume less power. For data centers that are built to maximize efficiency, a high performance, low-cost solution is ideal.Intel's second aim is to target the faster growing mobile market. This market includes smartphones, tablets, and wearable technology. The demand and turnover for products in this market is much higher compared to computing devices. Mobile device makers rely on FPGA to power their products. FPGA used in mobile technology devices are available at low per unit costs.