{"id":1797,"date":"2024-09-27T10:42:08","date_gmt":"2024-09-27T10:42:08","guid":{"rendered":"http:\/\/3.10.207.114\/?p=1797"},"modified":"2025-02-07T12:02:16","modified_gmt":"2025-02-07T12:02:16","slug":"running-you-own-llm-using-ollama","status":"publish","type":"post","link":"https:\/\/tech-musing.com\/2024\/09\/27\/running-you-own-llm-using-ollama\/","title":{"rendered":"Running you own LLM using Ollama"},"content":{"rendered":"\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"1024\" src=\"https:\/\/18.132.103.31\/wp-content\/uploads\/2024\/09\/image-2.png\" alt=\"\" class=\"wp-image-1798\" srcset=\"https:\/\/tech-musing.com\/wp-content\/uploads\/2024\/09\/image-2.png 1024w, https:\/\/tech-musing.com\/wp-content\/uploads\/2024\/09\/image-2-300x300.png 300w, https:\/\/tech-musing.com\/wp-content\/uploads\/2024\/09\/image-2-150x150.png 150w, https:\/\/tech-musing.com\/wp-content\/uploads\/2024\/09\/image-2-768x768.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<div class=\"wp-block-aioseo-table-of-contents\"><ul><li><a href=\"#Highlevel\">Highlevel<\/a><\/li><li><a href=\"#Sites\">Sites<\/a><\/li><li><a href=\"#aioseo-videos\">Videos<\/a><\/li><li><a href=\"#Run-Through\">Run Through<\/a><ul><li><a href=\"#Bundled-install\">Bundled install<\/a><\/li><li><a href=\"#Install-Ollama---Shell\">Install Ollama &#8211; Shell<\/a><\/li><li><a href=\"#Install-Ollama---Pip\">Install Ollama &#8211; Pip<\/a><\/li><li><a href=\"#Install-a-LLM-Model\">Install a LLM Model<\/a><\/li><li><a href=\"#To-check-what-LLM-models-you-have-and-other-stuff-on-Ollama\">To check what LLM models you have and other stuff on Ollama<\/a><\/li><li><a href=\"#Passing-Input-files---Bash\">Passing Input files &#8211; Bash<\/a><\/li><li><a href=\"#Passing-Input-files---Python\">Passing Input files &#8211; Python<\/a><\/li><li><a href=\"#Quick-Chat\">Quick Chat<\/a><\/li><\/ul><\/li><\/ul><\/div>\n\n\n<h1 class=\"wp-block-heading\" id=\"Highlevel\">Highlevel<\/h1>\n\n\n<p>Olama hosts LLM models and allows you to interact with them all locally<\/p>\n\n\n\n<p>openwebui is a nice gui front end for Ollama and models<\/p>\n\n\n<h1 class=\"wp-block-heading\" id=\"Sites\">Sites<\/h1>\n\n\n<p><a href=\"https:\/\/ollama.com\/download\/linux\">Download Ollama on Linux<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/ollama.com\/library\">library<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/openwebui.com\/\">Open WebUI<\/a><\/p>\n\n\n<h1 class=\"wp-block-heading\" id=\"aioseo-videos\">Videos<\/h1>\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Getting Started with Ollama and Web UI\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/BzFafshQkWw?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"python ollama read local file (EASY)\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/IsEYXyMkRF8?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n<h1 class=\"wp-block-heading\" id=\"Run-Through\">Run Through<\/h1>\n\n\n<p>Ran from an Aws Vm, the basic Micro doesn\u2019t have enough \/tmp space and you have to fudge around with things.<\/p>\n\n\n\n<p>The quickest solution is uping the instance type to something with more power a t2.xlarge seems to work well<\/p>\n\n\n<h2 class=\"wp-block-heading\" id=\"Bundled-install\">Bundled install<\/h2>\n\n\n<pre class=\"wp-block-code\"><code>curl -fsSL https:\/\/ollama.com\/install.sh | sh\nyum install pip -y \npip install ollama\nollama run llama3.1<\/code><\/pre>\n\n\n<h2 class=\"wp-block-heading\" id=\"Install-Ollama---Shell\">Install Ollama &#8211; Shell<\/h2>\n\n\n<p><code>curl -fsSL https:\/\/ollama.com\/install.sh | sh<\/code><\/p>\n\n\n\n<p>by following instructions on <a href=\"https:\/\/ollama.com\/download\/linux\">Download Ollama on Linux<\/a><\/p>\n\n\n<h2 class=\"wp-block-heading\" id=\"Install-Ollama---Pip\">Install Ollama &#8211; Pip<\/h2>\n\n\n<p><code>yum install pip pip install ollama<\/code><\/p>\n\n\n<h2 class=\"wp-block-heading\" id=\"Install-a-LLM-Model\">Install a LLM Model<\/h2>\n\n\n<p><code>ollama run llama3.1<\/code><\/p>\n\n\n\n<p>find model in <a href=\"https:\/\/ollama.com\/library\">library<\/a> and copy command<\/p>\n\n\n<h2 class=\"wp-block-heading\" id=\"To-check-what-LLM-models-you-have-and-other-stuff-on-Ollama\">To check what LLM models you have and other stuff on Ollama<\/h2>\n\n\n<pre class=\"wp-block-code\"><code>$ ollama list\nNAME            ID              SIZE    MODIFIED\nllama3.1:latest 42182419e950    4.7 GB  38 minutes ago\ngemma2:2b       8ccf136fdd52    1.6 GB  2 hours ago\n$ ollama\nUsage:\n  ollama &#91;flags]\n  ollama &#91;command]\nAvailable Commands:\n  serve       Start ollama\n  create      Create a model from a Modelfile\n  show        Show information for a model\n  run         Run a model\n  pull        Pull a model from a registry\n  push        Push a model to a registry\n  list        List models\n  ps          List running models\n  cp          Copy a model\n  rm          Remove a model\n  help        Help about any command\nFlags:\n  -h, --help      help for ollama\n  -v, --version   Show version information\nUse \"ollama &#91;command] --help\" for more information about a command.<\/code><\/pre>\n\n\n<h2 class=\"wp-block-heading\" id=\"Passing-Input-files---Bash\">Passing Input files &#8211; Bash<\/h2>\n\n\n<pre class=\"wp-block-code\"><code>$ cat \/home\/ollama_files\/helloworld_testfile       i have 5 oranges and 2 apples\nif i eat 4 oranges and 1 apple\nhow much is left?\n$ cat \/home\/ollama_files\/helloworld_testfile | ollama run gemma2:2b  \"prompt\"\nHere's how to figure out the remaining fruit:\nOranges Left: You started with 5 oranges, and you ate 4, so you have\n5 - 4 = 1 orange left.\nApples Left:  You started with 2 apples, and you ate 1, leaving you\nwith 2 - 1 = 1 apple.\nAnswer: You have 1 orange and 1 apple left. \ud83c\udf4a\ud83c\udf4e<\/code><\/pre>\n\n\n<h2 class=\"wp-block-heading\" id=\"Passing-Input-files---Python\">Passing Input files &#8211; Python<\/h2>\n\n\n<pre class=\"wp-block-code\"><code># cat .\/llm_test.py\n#!\/usr\/bin\/python3.9\nimport ollama\nnotes = \"helloworld_testfile\"\nwith open(notes,'r') as file:\n    content= file.read()\nmy_prompt = f\"give me the answer {content}\"\nresponse = ollama.generate(model=\"gemma2:2b\", prompt=my_prompt)\nactual_response = response&#91;\"response\"]\nprint(actual_response)\n#  .\/llm_test.py\nHere's how to solve that:\nOranges: You started with 5, and you ate 4, so you have 5 - 4 = 1 orange left.\nApples: You started with 2, and you ate 1, so you have 2 - 1 = 1 apple left.\nAnswer: You have 1 orange and 1 apple left.<\/code><\/pre>\n\n\n<h2 class=\"wp-block-heading\" id=\"Quick-Chat\">Quick Chat<\/h2>\n\n\n<pre class=\"wp-block-code\"><code>$ ollama run gemma2:2b\n>>> tell me a joke\nWhy don't scientists trust atoms?\nBecause they make up everything! \ud83d\ude04\nLet me know if you want to hear another one! \ud83d\ude0a\n>>> Send a message (\/? for help)<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Highlevel Olama hosts LLM models and allows you to interact with them all locally openwebui is a nice gui front end for Ollama and models Sites Download Ollama on Linux library Open WebUI Videos Run Through Ran from an Aws Vm, the basic Micro doesn\u2019t have enough \/tmp space and you have to fudge around [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-1797","post","type-post","status-publish","format-standard","hentry","category-ai"],"_links":{"self":[{"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/posts\/1797","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/comments?post=1797"}],"version-history":[{"count":1,"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/posts\/1797\/revisions"}],"predecessor-version":[{"id":1857,"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/posts\/1797\/revisions\/1857"}],"wp:attachment":[{"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/media?parent=1797"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/categories?post=1797"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tech-musing.com\/wp-json\/wp\/v2\/tags?post=1797"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}