Getting Started with Ollama and LLMs in Python

Introduction

Large Language Models (LLMs) have revolutionized AI applications, enabling chatbots, code generation, and intelligent assistants. Ollama is a powerful framework that makes it easy to run open-source LLMs locally on your machine with a simple interface.

In this post, we’ll explore what Ollama is, what LLMs are, and how you can get started with Ollama in Python using the langchain-ollama library.

What is Ollama?

Ollama is an open-source tool that lets you run LLMs on your local workstation. It provides a lightweight and easy-to-use API for interacting with various models without relying on cloud services.

Key features of Ollama:

• Runs Locally: No external API calls, everything is processed on your device.
• Supports Multiple Models: Use open-source models like Llama, Mistral, Gemma, and more.
• Efficient and Fast: Optimized for low-latency inference.

What is an LLM (Large Language Model)

LLMs are AI models trained on vast amounts of text data to understand and generate human like language. These models power applications like ChatGPT, Google Gemini, Claude AI, etc.

Popular open-source LLMs include:

• Llama 3: A state-of-the-art model from Meta.
• Mistral 7B: Known for efficiency and quality.
• Gemma: A new AI model from Google.

With Ollama, you can run these models locally without needing a Cloud API.

Setting Up Ollama in Python

Before we can use Ollama with Python, we first need to install Ollama, you can consult the documentation for Ollama Installations for your Operating System of choice.

After the installation we can verify if everything is working by running:

ollama list

The command should finish successfully, but you should have no open-source models listed, we can run llama3.2 by running:

ollama run llama3.2

Installing the Python Libraries

I am making use of virtualenv to install my packages in a isolated environment:

python -m virtualenv .venv

Then we can actiave the environment:

source .venv/bin/activate

Then we can install the required package:

pip install langchain-ollama

Use Ollama with LangChain in Python

Let's use the langchain-ollama package to interact with Ollama:

from langchain_ollama import OllamaLLM

# We will load the llama3.2 model
llm = OllamaLLM(model="llama3.2")

# Generate a response
response = llm.invoke("What is the capital of Japan?")
print(response)

# response
# The capital of Japan is Tokyo.

Thank You

Ollama provides an easy way to run powerful LLMs locally, giving you full control over AI chatbots, code assistants, and other NLP applications. With langchain-ollama, you can integrate these models into your Python applications effortlessly.

Thanks for reading, if you like my content, feel free to check out my website, and subscribe to my newsletter or follow me at @ruanbekker on Twitter.

• Linktree: https://go.ruan.dev/links
• Patreon: https://go.ruan.dev/patreon