Alex Shroyer hoosierEE

tensors over stressors

hoosierEE / local_llama.py

Created August 9, 2024 17:24

local llama in your terminal

	#!/usr/local/bin/env python3
	import openai
	import sys

	# inspired by https://two-wrongs.com/q

	# first download llamafile (https://github.com/Mozilla-Ocho/llamafile)
	# then run it in another terminal:
	# sh -c ./llava-v1.5-7b-q4.llamafile
	#