Vision processing unit
A vision processing unit is an emerging class of microprocessor (as of 2016), designed to accelerate machine vision tasks.[1][2]
These are distinct from video processing units (which are specialised for video encoding and decoding) in their suitability for running machine vision algorithms such as convolutional neural networks, SIFT etc.
They may include direct interfaces to take data from cameras (bypassing any off chip buffers), and have a greater emphasis on on-chip dataflow between many parallel execution units with scratchpad memory, like a manycore DSP. But, like video processing units, they may have a focus on low precision fixed point arithmetic for image processing.
They are distinct from GPUs, which contain specialised hardware for rasterization and texture mapping (for 3D graphics), and whose memory architecture is optimised for manipulating bitmap images in off-chip memory ( reading textures, and modifying frame buffers, with random access patterns).
Target markets are robotics, the internet of things, new classes of cameras for virtual reality and augmented reality, and integrating machine vision acceleration into smartphones and other mobile devices.
Examples
- Movidius Myriad 2, which finds use in Google Project Tango.[3]
- Microsoft hololens includes an accelerator referred to as a Holographic Processing Unit (complimentary to its CPU and GPU), aimed at interpreting camera inputs, to accelerate environment tracking & vision for augmented reality applications.[4]
- Eyeriss, a design from MIT intended for running convolutional neural networks.[5]
Similar processors
Some processors are not described as VPUs, but are equally applicable to machine vision tasks such as convolutional neural networks. These may form a broader category of AI accelerators (to which VPUs may also belong), however as of 2016 there is no consensus on the name:
- IBM TrueNorth, a neuromorphic processor aimed at similar sensor data pattern recognition and intelligence tasks, including video/audio.
- Qualcomm Zeroth Neural processing unit, another entry in the emerging class of sensor/AI oriented chips[6]
See also
- Adapteva Epiphany, a manycore processor with similar emphasis on on-chip dataflow, focussed on 32bit floating point performance.
- CELL, a multicore processor with features fairly consistent with vision processing units (SIMD instructions & datatypes suitable for video, and on-chip DMA between scratchpad memories).
- Graphics processing unit, also commonly used to run vision algorithms. NVidia recent Pascal architecture includes FP16 support, as a better precision/speed tradeoff for AI workloads.
- Digital signal processors, which are designed primarily to work with real-time data streams.
- TensorFlow
- OpenCL
- MPSoC
References
- ↑ "A third type of processor for AR/VR".
- ↑ "The rise of VPUs".
- ↑ Weckler, Adrian. "Dublin tech firm Movidius to power Google's new virtual reality headset". Independent.ie. Retrieved 15 March 2016.
- ↑ "microsoft-designed-a-special-processor-to-handle-hololens-data".
- ↑ "MIT eyeriss".
- ↑ qualcomm brain inspired computing
External links
- The rise of VPUs
- Eyeriss architecture
- Holographic processing unit
- NeuFlow: A Runtime Reconfigurable Dataflow Processor for Vision