NO Code Galaga game in 10 prompts using Llama 3.1
After initial prompt it’s already working, quite impressive After 10th prompt it look like a game
The cheapest way to host a local LLM – with reasonable speed
That’s probably the trash can Mac Pro from 2013. In December 2013, Apple released a new cylindrical Mac Pro (colloquially called the “trash can Mac Pro). Apple said it offered twice the overall performance of the first generation while taking…
ML Workstation upgrade – 2024 Summer
I’ve decided to upgrade my ML workstation, I was especially interested to improve the CPU core count. Unfortunately the original Asus ROG Zenith Extreme only supports Threadripper Gen2 processors and as 2990WX is more expensive than a 3970X which is…
The Ridiculous Accelerator setup
Since getting the Orange Pi 5 SBC, I wanted to try connecting a GPU via the M.2 M key NVMe port to create a compact inference server. I expected this wouldn’t be easy, but a post and YouTube video initially…
Inference on a $200 SBC
I’ve really wanted a server that I can use to host a quantized model for personal purposes, such as demoing a model, using it in a Hugging Face Space, etc. There are a huge variety of options to do this.…
LLama3 generation on Huggingface CPU Basic Space Free Tier Hardware
Huggingface Spaces are amazing tools to demo a model you’ve built, the only problem the Free tier the CPU Basic with 2 vCPUs and 16GB Ram is not really capable to run any bigger model. Here I want to show…
Measure and improve text diversity in generated content
There are certain type of tasks that requires to generate variates of outputs for the same or very similar prompts. Example of these task is content generation. In this article I’ve looked for a way to measure the diversity of…
Learning of an unsupervised Fine-Tuning 2
2 phase fine tuning Here’s another version of the fine tuned Phi-2 on the NextStep 3.3 Network System Admin manual data. The idea I’ve tested here is to do a 2 phased fine tuning.1) At the first phase I’ve “injected”…
Learning of an unsupervised Fine-Tuning
My goal with this experiment was to add new knowledge to a model with LoRA, and make it in an automated way. For the topic I’ve chosen a niche area of vintage computers so I’ve chosen to teach a model…
AI rig upgrades
Since I have the 2nd 3090 I’m working on niche Fine-tuned model. As I’m generating the data or do training the machine is running all the time. I’m sharing 2 set of monitors between that and other computers, and it’s…