Hotword detection in FHE on GPU in 350ms

A quick update on our homomorphic encryption compiler HEIR: our partners at Belfort have a hotword detector in FHE that runs in 350ms.

More specifically, they launched a live demo at hotword.belfortlabs.com, that has a little game inspired by the NY Times Connections puzzle. Instead of typing in your guess for the word that connects a group of clues, you say it out loud.

Your audio is encrypted under FHE, transitted to their server, and a hotword detector runs on it to determine which word you said (still encrypted). The client receives the encrypted result, decrypts it, and updates the game state.

Not only was this demo compiled with HEIR to target Belfort’s (proprietary) GPU platform, but the inference step runs in 350ms.

More specfically, the breakdown of runtime is: 15ms for client-side encryption, 350ms for server-side evaluation, 600ms for network latency (my Portland-based laptop to Belfort’s servers which are likely in Belgium) for a total round-trip of less than one second.

The one caveat is that they pre-load the one-time setup encryption keys, which currently take a minute or so to generate in the browser and upload to the server. So if you want to play the game securely, you need that extra step.

Of course, this could all be done locally, but it’s an impressive showcase of how fast FHE has become. And to my delight, how useful our compiler has become for compiling FHE workloads!

There’s still a lot more work to do, but this milestone makes me proud of the community of academics and practitioners we’ve grown around this compiler. Hooray for collaboration!

Belfort also has an early access program for their FHE-based cloud platform which you can sign up for here. You can forgive me for the advertisement, but they intend to use HEIR as a primary tool. So if you use their platform, you’ll be helping find bugs and improvemenets in the HEIR compiler!