Lesson 1: Welcome to Loki's Lab 101: Your First Local Model!

Prerequisites

You need a device that can run Linux, macOS, or Windows 10/11 and an internet connection for the initial setup. No credit card is needed because we are using open-source software! A modern browser and some curiosity are all you really need.

Steps

1. Head over to the official Ollama website and download the installer for your specific operating system. This gives you access to powerful AI models running directly on your own hardware.
2. Once installed, open your terminal or command prompt in a new window. If you are on Mac or Linux, ensure the terminal has the necessary tools installed. On Windows, the installer usually handles this automatically.
3. Type the simple command ollama serve in your terminal to start the server. You should see a message confirming it is listening on port 11434.
4. Pull your first model by typing ollama pull llama3. This downloads a popular open-weight model optimized for local performance. Wait until the progress bar hits 100%.
5. Chat with your new assistant by simply typing ollama run llama3 followed by any question you like, such as "Write a poem about a brave robot".

Expected Output

Your terminal window will display a series of logs showing the model downloading chunks into your system memory. Once finished, the prompt will return, and when you type your request, you will see the text generated by the AI appear immediately in that same window without leaving your machine. You should not see any cloud connection bars loading during generation.

Troubleshooting

• Port Blocked: If you get an error saying the server failed to start, check your firewall settings on Windows or macOS to allow traffic on port 11434. Sometimes local firewalls treat new applications suspiciously at first.
• Out of Memory: If the generation stops abruptly, you might not have enough RAM allocated for this specific model version. Consider trying a smaller model like mistral instead until your system cools down.
• Slow Connections: Ollama uses a public repository. If you are in an area with slow internet, simply ensure you are on Wi-Fi and avoid opening heavy browser tabs while the download runs to save bandwidth for the AI package.

Links to relevant benchmark/tool pages

• Ollama Official Website
• Open Source Model Repository
• Community Modelfile Examples

Lesson 2: Let Agents Work for You (Safely!)

Prerequisites

You must have successfully installed Ollama and pulled at least one model from Lesson 1. It is helpful to be comfortable typing commands in a terminal window, but you can click through the steps if you prefer a graphical interface on Windows.

Steps

1. Open your code editor or text file and create a new document to define your agent rules. This acts as a safety map for what your AI is allowed to do.
2. Enter a System Instruction that says: "You are an autonomous helper. You may search the web if you have permission, but never run system commands on the host machine unless explicitly verified by me first."
3. Define simple tools like calculator or file_search in your configuration file to give it specific jobs without giving it full admin access keys.
4. Start the agent workflow using a command like ollama run llama3:7b -p 'use_tool' (or your preferred command syntax depending on your setup) and input a request that requires multiple steps, like "Summarize these two notes for me".
5. Observe the agent working. It should pause to think before answering. If it tries to execute dangerous commands, you will see a refusal based on your safety instructions.

Expected Output

Your terminal or application window will show the step-by-step breakdown of what the agent is doing. It might say "Calling Calculator Tool" or "Searching Notes". The final output should be a clean answer that solves your specific problem without attempting to alter your system settings or delete files.

Troubleshooting

• Looping Errors: If the agent gets stuck repeating the same sentence, stop the session by typing ctrl+c. You may have defined a prompt that triggers an infinite loop, so edit your instruction file to break the cycle.
• Hallucinated Tools: Sometimes models invent tools they don't have. This is common in local runs because they lack real-time data. If you see an error about a missing function, update your model or switch to a smaller, more deterministic model for this task.
• Memory Limits: Large agents can run out of space if they process too many files. Close unused applications on your computer to free up virtual memory before starting another complex agent task.

Links to relevant benchmark/tool pages

• Agent Safety Best Practices
• Tool Calling Examples in Ollama
• Local AI Security Guide

Lesson 3: Decoding the Leaderboards: Are We Being Fooled?

Prerequisites

You need a basic understanding of how local models run from Lesson 1. You should also have finished configuring an agent safely in Lesson 2 to understand why raw scores might differ from real-life usage.

Steps

1. Go to the open source leaderboard hosted on Hugging Face or the Ollama community library to find current ranking charts. Look for metrics like MMLU (General Knowledge) and HumanEval (Code Tasks).
2. Open a model you have installed locally and run the specific questions used in those benchmarks. Run 10 questions of each type to get an accurate average for your hardware.
3. Compare your local results with the published global scores. You will likely see that local versions score lower due to lack of specialized cloud training, but accuracy is what matters most to you.
4. Read the notes on how benchmarks penalize certain behaviors. Some leaderboards favor speed over safety, which is why trusting the numbers alone can sometimes mislead us about a model's true reliability.

Expected Output

You will arrive with a clearer picture of your model's strengths and weaknesses. Instead of blindly trusting a 95% accuracy score from another server, you understand that performance varies based on how well the AI handles context locally. You now know to interpret leaderboard results as suggestions rather than absolute rules for your specific setup.

Troubleshooting

• Conflicting Data: Different leaderboards use different question sets. If one chart says your model is amazing and another says it is poor, check if they are measuring different types of reasoning like coding vs writing essays.
• Version Differences: Models update frequently. A score from last year might not apply today because the developers have retrained the weights since then. Always verify you are looking at the version number that matches your installed local copy.
• Bias Concerns: Notice that some benchmarks only measure English fluency. If you want to try other languages, you must look for different leaderboards that test specific language capabilities so you don't get false confidence in your model's multilingual abilities.

Links to relevant benchmark/tool pages

• Open LLM Leaderboard
• MMLU Benchmark Explanation
• Llama Evaluation Page