Running Deepseek R1 locally: my first impressions

Running DeepSeek Locally on My Modest Machine: Impressions and Setup**

In the realm of artificial intelligence, few experiences are as exhilarating as running a model like DeepSeek-R1 locally. For someone with a modest yet capable setup—a NVIDIA GeForce GTX 4090 GPU paired with an AMD Ryzen 7900X3D CPU—this journey has been nothing short of awe-inspiring.

Installation and Configuration

The process began with downloading Ollama, an open-source tool that simplifies running AI models locally. Using pip, I installed Ollama effortlessly:

pip install ollama

Once installed, both the command line interface and a graphical interface (Ollama Studio) were available for use. I chose to stick with the command line for simplicity.

Running DeepSeek-R1 was as straightforward as typing the following command:

ollama run deepseek-r1:8b

This command not only initiated the model but also demonstrated the ease of execution, with minimal setup required beyond basic hardware specifications. I eventually moved to the 32b parameter model which was slower on my build, but still faster than I could read as deepseek generated replies.

The cutest part of deepseek by far is that it reveals its internal "thoughts" as it works, you get a thought process enclosed in <thoughts> brackets in realtime. Very cute.

Performance and Functionality

The performance of DeepSeek-R1 was a pleasant surprise. Despite its designation as an 8B parameter model, it handled complex tasks such as generating coherent text and engaging in conversations without lag or slowness. While not always perfect, the output was consistently close to what I expected.

For instance, when posing questions ranging from creative writing prompts to logical puzzles, DeepSeek-R1 provided responses that were both meaningful and occasionally insightful. This functionality made it a versatile tool for various tasks, including generating short stories and solving simple logic problems.

Ease of Use

What truly stood out was the user-friendly nature of Ollama. The plug-and-play experience allowed me to experiment with different models without delving into low-level configurations or complex frameworks. This ease of use is particularly beneficial for someone with a technical background but not an expert in AI development.

Trying to run DeepSeek-R1 on a MacBook Pro

Just for fun, I wanted to try running the 7B parameter model on my first-gen M1 MacBook Pro, and the thing struggled to even output anything—it basically just hung there.

My go-to question for new models is:

"Who is your daddy, and what does he do?"

This harks back to Kindergarten Cop and lets me quickly eyeball if I'm not on the right hardware for the model.

In the MacBook's case, it essentially failed the task at 7B parameters and didn't even reply at 1.5B—it just kept repeating its documentation link.

Tips for Getting Started

For those considering running DeepSeek-R1 or other AI models, here are some recommendations:

  1. Assess Hardware Needs: While powerful machines can enhance performance, even lower-end systems can run basic AI tasks effectively.

  2. Install Ollama: Use pip to download and install the tool, making it accessible for both command line and graphical interface use.

  3. Run Commands: Utilize the ollama run... command to initiate models like DeepSeek-R1, allowing for interactive sessions through either CLI or GUI.

  4. Experiment with Inputs: Adjust prompts and inputs to see how they affect outputs, providing valuable feedback for iterative improvement.

  5. Explore Models: Ollama offers a variety of models beyond DeepSeek-R1, each with unique strengths that can be tailored to specific tasks. You can always find more models or list all installed models with ollama list.

Conclusion: A New Era in AI Accessibility

Running DeepSeek-R1 locally signifies a significant milestone in AI accessibility. By democratizing access to powerful tools, it empowers users without extensive technical expertise to engage with intelligent systems.

This experience has not only been personally enriching but also reinforces optimism about the future of AI. As more user-friendly interfaces emerge, the potential for creating and interacting with intelligent systems becomes increasingly accessible, heralding a new era in technological innovation.

For someone who craves exploration and innovation, this journey into the world of AI has been nothing short of transformative—a testament to the possibilities that lie ahead.

As for me? I'm hitting the gym.