What is Simultaneous and Heterogenous Multithreading and Why Should You Care?

Hey Geeks,

It looks like the geniuses at the University of California Riverside are trying to redefine the way our computers work. They've developed a groundbreaking approach called Simultaneous and Heterogeneous Multithreading (SHMT), which aims to boost our computers' efficiency and speed by enabling different types of processors to work together harmoniously.

The idea is to take your computer's GPU, CPU, and TPU, which usually work independently, and team them up to share tasks and tackle them simultaneously. This way, you can use the strengths of each item more efficiently to make a more effective team. It's like when Daredevil teamed up with Spider-Man to beat Kingpin.

The researchers crafted some clever mechanisms, dubbed VOPs (Virtual Operation)and HLOPs (High-Level Operation), that smartly distribute computing chores across various processors. Thanks to a quality-aware mechanism called the Quality-Aware Work-Stealing (QAWS) policy, tasks are completed more swiftly and with increased precision.

The SHMT framework is designed to optimize computing processes by utilizing the different types of processors within a system, such as CPUs, GPUs, and TPUs. It operates using Virtual Operations, which are high-level tasks that don't specify the hardware they should run on. These VOPs are then translated into High-Level Operations (HOPS), which are more specific and can be executed by the appropriate processing units. This system allows for tasks to be distributed across various processors, tapping into each processor's unique strengths to enhance overall computational speed and efficiency. QAWS dynamically adjusts the allocation of tasks among processors to ensure balanced workloads and optimal utilization of resources.

So, SMHT uses QAWS to balance the VOPs and HLOPS of your GPU and TPU coordinated by your CPU. Seems clear enough.

But what do these advancements mean for you and me? Well, for starters, they've accelerated processing speeds by up to 1.95 times while also cutting down energy usage by a whopping 50%.

This leap forward isn't just about faster computing; it's about greener, more energy-efficient technology that benefits our environment and our electricity bills.

To put into perspective, according to the International Energy Agency, global electricity consumption by data centers was estimated to be between 240 and 340 terawatt-hours (TWh) in 2022, accounting for approximately 1-1.3% of global electricity demand. Cutting that in half could be a very big deal.

Because these are huge numbers, I did a little math to gain more perspective. (I'm sure I will hear about it if I am wrong in the comments) A Terrawatt would run the average American household for 93,000 years. (The Average US electricity customer uses 10,715 kilowatt-hours per year, and 1 TWh = 1,000,000,000 kWh.) A nuclear power plant, on average, produces 8-9 Terrawatts per year.

Further, looking at AI, it is estimated that OpenAI has 30,000 a100 GPUs, which is crazy expensive. At over 7,000 per card on Amazon, and I doubt they buy them on Amazon, you are talking about 250 million just in cards. They could cut that cost in half or double the output.


Needless to say, SHMT is a promising tool for turbocharging artificial intelligence and big data analytics, reducing overall costs. A tech that makes digital experiences faster, smarter, and more eco-friendly? It could be an idea full SHMT, but I'm betting this is one to watch.