• The Juicer
  • Posts
  • How to (Easily) Run Local LLMs on Your Computer šŸ’»

How to (Easily) Run Local LLMs on Your Computer šŸ’»

(And keep your privacy)

5-min Read | Advertise here

In partnership with

šŸ’„ Use AI to 10X your productivity & efficiency at work (free bonus) šŸ¤Æ

Still struggling to achieve work-life balance and manage your time efficiently?

Join this 3 hour Intensive Workshop on AI & ChatGPT tools (usually $399) but FREE for first 100 readers.

Save your free spot here (seats are filling fast!) ā°

An AI-powered professional will earn 10x more. šŸ’°

An AI-powered founder will build & scale his company 10x faster šŸš€

An AI-first company will grow 50x more! šŸ“Š

Want to be one of these people & be a smart worker?
Free up 3 hours of your time to learn AI strategies & hacks that less than 1% people know! 

šŸ—“ļø Tomorrow | ā±ļø 10 AM EST

In this workshop, you will learn how to:

āœ… Make smarter decisions based on data in seconds using AI 
āœ… Automate daily tasks and increase productivity & creativity
āœ… Skyrocket your business growth by leveraging the power of AI
āœ… Save 1000s of dollars by using ChatGPT to simplify complex problems 

Privacy is important in AI.

Thatā€™s why I found a way to run an LLM (Large Language Model) on my local computer.

For free.

Hereā€™s how to do it:

Iā€™ve always been cautious about uploading private information to ChatGPT, Gemini, or any AI model online.

Thereā€™s one big concern: data privacy.

These big tech companies use the data we input to train their models, so I never upload any sensitive content.

But there has been a compelling solution: locally run LLMs.

In this segment, Iā€™ll explore a new open-source interface: LM Studio.

From its beautiful interface, you can discover open-source models, download them, and run them locally on your computer or a localhost.

But does LM Studio collect any data?

No. One of the main reasons for using a local LLM is privacy, and LM Studio is designed for that. Your data remains private and local to your machine.

Let me show you how.

Step 1: Download LM Studio

The first thing you need to do is head to LM Studioā€™s website and download it.

Itā€™s open-source, which means itā€™s free.

Step 2: Search and Download an LLM

In this tutorial, Iā€™ll just search for ā€œMistralā€.

It will bring up a list of Model Cards with Mistral in it, which are like repositories for LLMs from an open-source ML sharing website called Hugging Face.

I picked this one by TheBloke.

On the right, youā€™ll see a bunch of files displayed on the Model Card.

You can also view the Model Card in your browser (Huggin Face) by clicking on a button at the top right.

These files differ in their quantization, which determines the fidelity of the results.

LM Studio will make a compatibility guess based on your machineā€™s specs.

Any LLMs with the words ā€œFULL GPU OFFLOAD POSSIBLEā€ in green mean that you can probably run them.

I downloaded this one: mistral-7b-instruct-v0.1.Q3_K_S.gguf

As the file sizes are huge, give it some time to download.

Step 3: Initialize Model and Configure Settings

Click on the AI Chat bubble on the left panel.

Select a model to load at the top. Iā€™ll select the model I just downloaded.

The model will take a while to load.

Before we start, please click on Markdown at the top so that you can get pretty text responses.

Next, on the right, you can set a system prompt.

You can do some advanced configuration for these parameters:

  • Context length

  • Temperature

  • Tokens to generate

and other settings.

Iā€™ll leave it as default, but you can play around with them to get your desired result.

Step 4: Chat with LLMs Offline

Letā€™s get chatting!

I entered this prompt and hit enter:

ā€œyou are an expert content strategist and email marketers with 5 years of experience. I want you to give me a 2 week newsletter content strategy for one month. my newsletter is about AI and it's called The Juicer.ā€

Hereā€™s its response:

Seems like the responses arenā€™t amazing (compared to GPT-4o), but it works great!

What Iā€™d do next is try out some newer open-source LLMs by Meta (LLaMa 3) or choose a model with a higher quantization number.

Also, note that your chats will be saved on the left side, similar to ChatGPTā€™s interface!

Have fun running this model for your private use!

Feel free also to explore using your models on a local host or a multi-model session in the Playground tab.

This tutorial was adapted from this YouTube video:

Donā€™t know where to start with AI?

ā†³ Hereā€™s where to start: The AI Starter Kit.

I spent 100s of hours crafting it, with ALL my AI knowledge inside.

Inside, youā€™ll get lifetime access to:

  • How To Start Using AI Guide

  • The Complete ChatGPT Beginnerā€™s Guide

  • My 10,000 prompts

  • AI Learning Library of AI Resources

  • Bonus project: Create Your Own Assistant With ChatGPT

Ready to finally learn AI and apply it to your business/work?

(Youā€™ll also help keep The Juicer completely free - thank you! šŸ™šŸ»)

Thatā€™s all for this week.

See you next time!

Justin Chia

Find me on LinkedIn or shoot me an email at [email protected]

My Complete AI Tech Stack āœšŸ»

I use these tools to automate my daily business processes.

  • ChatGPT Plus - Powerful AI chatbot with GPT4o model that powers my writing process.

  • Perplexity - Amazing AI chatbot tool for research.

  • TubeonAI - AI-powered YouTube video summarizer.

  • Canva - An easy-to-use design tool with AI built-in.

  • Beehiiv - Beautiful email marketing platform built for growth (Get a 30-day trial + 20% OFF for 3 months!)

  • Supergrow - Powerful AI LinkedIn scheduling tool.

  • Namecheap - Cheap domain name registrar.

  • Google Workspace - Workspace to store all business documents.

  • Notion - Flexible workspace to track and build business processes.

  • Stripe - Trusted automatic billing platform.

When Youā€™re Readyā€¦

Letā€™s work together ā†’ here are 2 ways I can help you:

  1. The AI Starter Kit (50% off): My complete beginner resource pack for learning about AI in 2024. I share beginner resources to get started, prompts I use, an AI learning library, and other bonus content.

  2. Promote your company to 7,000+ AI & tech lovers by sponsoring this newsletter.

If you have any questions or feedback, reply to this email. (I'll personally reply! :)

Reply

or to participate.