How to easily run your favorite local AI models on Linux with this handy app

I have been using AI as one of my go-to tools for research for a while. There are many options that can be accessed via a web browser (Opera’s Aria, the first AI service I’ve ever used), but since I prefer desktop tools, I can install and use a local LLMS, all queries remain private. Desktop tools allow you to easily switch between local LLMs such as Llama, Deepseek R1, Mistral Instruct, ORCA, and GPT4ALL FALCON.

Until now, my default Linux AI GUI was MSTY, but I found a new app that’s just as good. That app is gpt4allavailable on Linux, Macos, and Windows.

Also: My two favorite AI apps on Linux and how to get more done using them

gpt4all can be used to write code and standard queries, as a personal writing assistant. It also has a great UI that is easy to use and fits directly into the look and feel of your desktop.

One aspect of GPT4all that I appreciate is that you can choose a computation device for text generation. For example, my System76 Thelio has an AMD Radeon RX 7600 GPU. GPT4ALL detects the GPU and allows you to use a Vulkan: AMD Radeon RX 7600 (RADV NAVI33) or a Vulkan: AMD Radeon RX 7600 (RADV Raphael_Mendocino).

You can also select a default model, change the suggestion mode (to generate follow-up questions), configure the number of CPU threads, and enable the System Tray app. You can enable the local API server to access gpt4all http:// localhost:4891. The drawback of using server mode is that it consumes more system resources.

Let’s try installing gpt4all on Linux.

Install GPT4ALL on Ubuntu-based distribution

What you need: Currently, the only supported Linux distribution is based on Ubuntu, so you need a running instance of an Ubuntu-based distribution. We’ll demonstrate this in pop! _os linux.

chmod u+x gpt4all-installer-linux.run

./GPT4ALL-INSTALLER-LINUX.RUN

Please note that you do not need to use sudo for your installation. When I try to install gpt4all with sudo, I get an error. Once the installation wizard opens, click on the steps to complete the installation.

Also: how to send files to local AI for better, more relevant responses

The installation must be completed intermittently.

Use gpt4all

When you first open GPT4All, you must opt-in or opt-out of anonymous usage analytics and anonymous sharing of chat. I recommend opting out. Once you have completed it, you will see yourself in the main GPT4ALL window and you will need to install the local model.

To install the first model,[モデル]Click and[モデルの追加]Click. In the resulting window, scroll down to find a model such as llama 3.2 3b instructe. If you find it,[ダウンロード]Click to start the process. Remember that these models can get quite large (minimum is the llama 3.2 3B direction at 1.79 GB).

Once the model is downloaded and installed, you will see it is listed in the Models section.

Also: How to run deepseek ai locally to protect your privacy – two easy ways

Click the chat icon in the left sidebar. If you installed only a single model, it will be listed in the top center. If you have installed multiple models, select the model you want to use from the dropdown.

Enter the query in the Send Message field and press Enter on your keyboard. gpt4all gets busy answering queries. If you enable follow-up suggestions, they will be listed under the answer. I like follow-up suggestions because follow-up suggestions help me dive into a seriously insightful (and sometimes fun) rabbit hole. In my System76 Thelio, I barely noticed the additional computational overhead required for the proposed follow-up, so it’s worth enabling.

And it’s just my friend, and there’s the basics to install GPT4All on Linux and use this useful tool.

Want more stories about AI? Sign up for Innovationour weekly newsletter.