Do it yourself again,
at the speed of speech.

For anyone who finds clicking and typing slow, tiring or painful. Hold Ctrl+S while you say what you want, and Surface does the clicking and typing.

Join the waitlist For Windows, and Linux on X11

Four short films of Surface on a Mac

Animated replays on a Mac, with the yellow tags showing the names Surface reads on screen. The Mac version is still in testing.

What the four films show:

  1. Reply to Mom: holding Control-S, the person says “reply to mom, say we'll bring the pie”. Surface opens Mom's email in Mail, presses Reply and writes “Hi Mom! We'll bring the pie.” It does not press Send.
  2. Make the text bigger: the person says “the writing's too small, I can't read it”. Surface presses Command-plus twice in Safari and the page gets bigger.
  3. Play some Sinatra: the person says “put on some sinatra”. Surface goes to a video site in Safari, searches for Frank Sinatra and plays the first video.
  4. Two windows side by side: the person says “put my notes next to the browser”. Surface moves Safari to the left half of the screen and Notes to the right half.

Built to stop before Send.

It writes, then waits. It's built to press Send, Buy or Delete only when you ask, so read it over first.

Right most of the time, not always.

Check anything that matters. Press Ctrl+Alt+Space and it stops after the current step.

Not hands-free yet.

You hold Ctrl+S while you talk, and you type your own passwords. On Linux you can change the shortcut.

Questions

Will I still need my hands?

A little. You hold Ctrl+S while you speak (on Linux the key can be changed), and Surface usually does the clicking, typing and scrolling after that. Some things still need your hands:

  • holding Ctrl+S for as long as you talk (there's no hands-free way to start yet);
  • typing passwords;
  • setting it up: the Windows installer asks you to paste in keys, and on Linux you edit a file and use a terminal;
  • starting Surface after you turn the computer on (an icon on Windows, a terminal on Linux);
  • finishing a job yourself when it can't be sure where to click and stops.

If speaking isn't easy either, you can type the request instead: Ctrl+Alt+Space brings up the bar. While it works, keep your hands off the mouse and keys so a stray click doesn't collide with its own.

Does it always get it right?

No. On tricky requests that other AI models wrote to trip it up, it picked the right steps about nine times in ten, and some of its misses were still reported as done, so check anything that matters. It hasn't yet been tested with wording from people who are new to it. It's built to ask when a detail is missing, but in testing it once made up a zip code instead.

Most apps tell your computer what their buttons are called, and Surface goes by those names. Some apps only draw pictures, like certain office programs and remote desktops. For those, Surface works from a picture of the window and usually clicks only where two different AI models agree on the spot. That's slower, and in testing about 3 clicks in 100 still landed on the wrong thing, and about 1 in 3 when what you asked for wasn't on the screen. On Windows this part hasn't been tried yet.

What leaves my computer?

To plan each request, Surface sends an AI model, through OpenRouter:

  • your words, and the date and time;
  • the titles of your open windows and of the tabs in your browser windows;
  • the names of your installed apps and of your most recent files;
  • your last few requests from the past ten minutes, and how each one ended;
  • the names and text (up to the first 80 characters) of the buttons and boxes in the window you're using. Password boxes are never read.

If the company that usually runs that model is slow to answer, Surface asks a second one as well, through OpenRouter. For windows that don't list their buttons, and sometimes when it can't find a button by name, a picture of the window is sent too, to up to four AI models, with the Surface bar greyed out. While you hold Ctrl+S, your voice is streamed to a speech service to be turned into text. Each request is also saved in a log on your own computer.

Is it listening all the time?

No. The microphone is only open while Ctrl+S is held down, and every press opens it for that moment, even a quick one to save a document (which still saves like it always did). Speaking is new: it has been used with a real microphone on Linux; on Windows and Mac it has so far only been tried with recorded voices.

Which computers does it run on?

Windows 10 and 11, and Linux desktops running X11. On Ubuntu that means choosing “Ubuntu on Xorg” on the login screen; the default (Wayland) isn't supported yet. Windows support is new and so far tested only in a Windows 11 virtual machine at 100% display scaling; most laptops come set to 125% or more, and that hasn't been tried yet. A Mac version is in testing in a macOS virtual machine, not yet on a real Mac.

Tried so far with Edge, Notepad, Calculator, File Explorer and Settings on Windows, and Chrome, Brave, Files, Calculator and Text Editor on Linux.

What do I need to set it up?
  • A key from OpenRouter, the service that connects Surface to the AI models. To speak instead of type, a key from Deepgram too. Both charge your own account for what you use. We haven't measured a typical month's bill yet; OpenRouter lets you set a spending limit.
  • On Windows, the installer asks for the keys and needs no administrator rights. On Linux, you put the keys in a file and start Surface from a terminal for now.
  • Surface types by pasting, so whatever you had copied is replaced.
  • Surface can usually see the buttons right away in Chrome and Brave on Linux, and in Edge on Windows. On Linux, Firefox works from pictures of the window (slower) until you run a one-time setup command, which the terminal shows you.

Your independence,
at the speed of speech.

Surface is still being tested. Leave your email and we'll write to you when it's ready to try.

The test version runs on Windows 10 or 11, or Linux on X11. Setup means pasting in a service key or two (on Linux, a terminal too), and those services bill you for what you use. You can set a spending limit.