Local LLM Chat
Welcome to the Local LLM Chat!
This chat interface allows you to interact with a local Large Language Model (LLM) deployed on your machine. You can ask the model anything, and it will respond with generated text based on your prompt.
To use the chat, simply type your question or statement in the input box below and click “Send”. The model will generate a response, which will appear in the response area.
This site is powered by a Flask server running locally that acts as a bridge between your browser and the model.
The model is running on your local machine at localhost:11434
.
Response:
Debug Log:
Flask Server Example:
To make the HTML interface work, you need to set up a local Flask server that acts as a proxy between the browser and your local LLM. Below is the code for the Flask server:
from flask import Flask, request, jsonify
from flask_cors import CORS
import requests
app = Flask(__name__)
# Enable CORS for all routes
CORS(app)
# LLM server URL (this is your local LLM server)
LLM_SERVER_URL = "http://localhost:11434/api/generate"
# Define the endpoint that the HTML will call
@app.route('/api/generate', methods=['POST'])
def generate():
# Get the JSON payload from the HTML request
payload = request.json
# Make a request to the local LLM server with the same payload
response = requests.post(LLM_SERVER_URL, json=payload)
# Return the LLM response to the HTML page
return jsonify(response.json()), response.status_code
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000) # Running on local machine, port 5000