Several analytics vendors have highlighted graphics processing unit (GPU)-accelerated computing in recent analyst briefings including KineticaMapD, and Jedox. You also tend to hear about GPU when talking about Caffe, Torch and TensorFlow in the machine learning space. Historically we have heard a lot about CPU and in-memory computing in analytics applications. What exactly is GPU acceleration and why should analytics pros care about it?

What is GPU Acceleration

Pioneered ten years ago by Nvidia for visual applications, GPU-accelerated computing is the use of a GPU together with a CPU to accelerate applications. It enables computational scientists and researchers to address some of the world’s most challenging problems up to several orders of magnitude faster than traditional architectures.

Accelerated computing is revolutionizing high performance, exascale computing. According to an old HP paper, in early data science applications of problems that could fit inside GPU’s memory, speeds up to 200-400x faster were realized running on a single CPU core, and about 50x-88x faster than highly optimized CPU apps running on a single cores.

For larger problems where data sets do not fit in GPU memory, GPUs still offered significant performance gains. For example, a very large data set with one hundred million 2-dimensional data points, and two thousand clusters, can be processed by GPU in less than 6 minutes, compared to 450 minutes with highly optimized CPU version on a single core, representing an 80x speed improvement.

Kinetica’s GPU-accelerated, in-memory analytics database delivers real-time response to queries on large, complex, and streaming data sets for 100X query performance improvements with 1/10 the hardware.

GPU Apps

Source: NVIDIA

GPU acceleration has progressed into many different engineering, analytics and business applications across a wide variety of domains.

  • Data Science & Analytics
  • Deep Learning and Machine Learning
  • Computational Finance
  • Engineering
  • Defense and Intelligence
  • Gaming, Media and Entertainment
  • Research
  • Scientific Visualization
  • Supercomputing

Hundreds of GPU-enabled high performance computing applications can be found in the market today. A sampling is listed in NVIDIA’s GPU application catalog.

How it Works

GPU-accelerated computing uses a divide and conquer approach offloading compute-intensive portions of an application to the GPU while the remainder of the code still runs on the CPU. From a user’s perspective, GPU enabled applications merely run a lot faster than CPU apps.

how-gpu-acceleration-works

Source: NVIDIA

A simple way to understand the difference between a GPU and a CPU is to compare how they process tasks. A CPU consists of a few cores optimized for sequential serial processing while a GPU has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously.

cpu-and-gpu

Source: NVIDIA

GPUs are ideal for compute-intensive analytics workloads on large data sets such as aggregate queries that are difficult to index well, and fundamental to data exploration, discovery, and visualization. It is also available on all the major cloud platforms and is key to scaling in a big data world with sensor logs, images, videos, and more.

Below is a fun video demonstration of GPU computing concepts.

Another reason why GPU is becoming more popular is that with many light-weight data-parallel cores, substantial computational power is gained at a much lower capital equipment cost or cloud operational expense. GPU is a “green” tech per se. You can save up to 70% by replacing hundreds of commodity CPU instances with strong nodes having up to 8 GPUs per instance.

GPU Tomorrow

The combination of cloud, artificial intelligence and deep learning algorithms along with high performance computing is expected to profoundly impact every aspect of human life in the digital transformation. GPU is a key part to that innovation. If you have not heard of GPU before…I’m betting that you will start to hear much more about it soon.