I wanted to experiment with combining image generation and vision models to see how images evolve when each description is used as the next prompt for the generator. My goal was to observe whether this feedback loop causes images to…
Sample code to generate the gif
Elmo chat is Lepton AI’s little fun chrome extension that lines in your chrome browser and offering summarization question answering and even TTS (Text To Speech) do chat with the webpage you’re browsing. Slowly but surely changing the way we’re…
I’ve rerun the Speculative Decoding experiment with some larger models where I’ve paired a Llama 3 70B primary model with the 8B draft model, to see if a larger model Llama 3 70B can benefit more from a draft model…
And there I’ve crunched the numbers on GPU. The configuration is a 3GPU system 1x RTX4080 + 2x RTX3090. Baseline has been set by the following llama.cpp generation command (same generation config and prompts has been used in the Speculative…
Here’s a followup on the Speculative decoding on CPU tradeoffs now performing the same test on a Threadripper 3970x, and finally I see some benefit. Acceptance rate Generation speed Summary The results seems very similar to the Xeon E5 test.…
I’ve played around with the llama.cpp speculative decoding on CPU (Mac Pro 2013 – Xeon E5 12core 2.7GHz) and wanted to share my experience. First of all I’ve struggled to find models where the vocab size difference is less than…
After initial prompt it’s already working, quite impressive After 10th prompt it look like a game
That’s probably the trash can Mac Pro from 2013. In December 2013, Apple released a new cylindrical Mac Pro (colloquially called the “trash can Mac Pro). Apple said it offered twice the overall performance of the first generation while taking…
I’ve decided to upgrade my ML workstation, I was especially interested to improve the CPU core count. Unfortunately the original Asus ROG Zenith Extreme only supports Threadripper Gen2 processors and as 2990WX is more expensive than a 3970X which is…
