Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI. learn more
OpenAI is expanding the number of applications its desktop app can work with, including allowing Advanced Voice Mode to work with other apps, moving it closer to computer-based ChatGPT.
The desktop app introduced integration with the first four applications in November. On day 11 of the “12 Days of OpenAI” event, OpenAI announced several new integrated development environments (IDEs), terminals, and text apps that it supports.
ChatGPT now supports BBEdit, MatLab, Nova, Script Editor, and TextMate as IDEs. VSCode for VSCode Insiders, VSCodium, Cursor, WindSurf, JetBrains IDE family for Android Studio, AppCode, CLion, DataGrip, GoLand, IntelliJ IDEA, PHPStorm, PyCharm, RubyMine, RustRover, WebStorm. We’ve also added the Warp and Prompt terminal apps as integrations. These applications join VS Code, Xcode, Terminal, iTerm 2, and TextEdit as integrated apps.
However, coding applications aren’t the only applications that the ChatGPT desktop app can access. OpenAI also added Apple Notes, Notion, and Quip to its integrations. Advanced voice modes can work with these applications, taking into account the context of the project within the integration.
OpenAI emphasized that users must grant ChatGPT permissions to access these applications.
Make your computer available to ChatGPT
Of course, integrating apps with AI chatbots is nothing new. In October, GitHub Copilot now has coding platform integration. Connecting your application to ChatGPT or Copilot brings context from those platforms into your chat experience. Developers can ask ChatGPT for help coding their projects on VS Code, and the chatbot will understand what they’ve been working on.
Kevin Weil, chief product officer at OpenAI, said in a live stream that improvements to the desktop app will bring ChatGPT closer to a more agent-like user experience.
“We’ve put a lot of effort into our desktop app,” Weil said. “As our model becomes more and more powerful, ChatGPT will become more and more agentic, meaning it will become more than just a question and answer session. ChatGPT will start doing things for you. Masu.”
He added that desktop apps are “a big part” of that transformation.
“Being a desktop app, you can do so much more than just a browser tab,” he said. “This includes being able to see what’s on your screen, with your permission, and being able to automate many of the tasks you do on your desktop. As we head into 2025, There is much more to be said.”
OpenAI allows ChatGPT to see more of your computer, bringing it closer to Anthropic’s Claude Computer Use feature. This allows Claude to click on people’s computers, move around the screen, and type text.
OpenAI has already announced fairly similar functionality for the mobile version of ChatGPT, but chatbots still don’t have access to computers or phones in the same way. Users can share their screen with the chatbot, so the chatbot can “see” what the user is reading and looking at. microsoft and google We have also developed features comparable to Copilot Vision and Project Astra.
How to access
On MacOS, users who wish to open ChatGPT while using other applications can use “Options + Space” to invoke ChatGPT and select the desired application from the buttons on the chat screen. Another shortcut “option + shift + 1” will show you the top level application to use.
From that window, users can also access advanced audio modes in the same way. Voice mode automatically detects the context from the application.
The integration is available to ChatGPT Plus, Pro, Team, Enterprise, and Edu users. However, Enterprise and Edu subscribers must ask their IT administrator to enable this feature.
