vulkan support for typescript bindings, gguf support

[jacoobes]

Describe your changes

Issue ticket number and link

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• I have added thorough documentation for my code.
• I have tagged PR with relevant project labels. I acknowledge that a PR without labels may be dismissed.
• If this PR addresses a bug, I have provided both a screenshot/video of the original bug and the working solution.

Demo

Steps to Reproduce

Notes

[iimez]

Did some testing today.

Found some typing issues:

• model.llm.availableGpus is not a function - but its defined in types. probably should be listGpu?
• DEFAULT_PROMPT_CONTEXT is typed as DEFAULT_PROMT_CONTEXT (note missing P in PROMT)
• loadModel takes EmbeddingOptions | InferenceOptions, should be EmbeddingModelOptions | InferenceModelOptions

On GPU support:
Its working and I'm able to do inference on 13B models with about double to triple the speed. Amazing!
Interestingly its consistently not utilizing GPU for the first model I try to load with device=gpu, but it does for the second. For example:

const firstModel = await loadModel('llama-2-7b-chat.ggmlv3.q4_0', {
	type: 'inference',
	device: 'gpu',
})

// this will work, but using cpu
const firstCompletion = await firstModel.generate('Hello, my name is')

const secondModel = await loadModel('llama-2-7b-chat.ggmlv3.q4_0', {
	type: 'inference',
	device: 'gpu',
})

// this will use gpu
const secondCompletion = await secondModel.generate('Hello, my name is')

Not sure if this is an issue in bindings code, gonna try again after we have synced main.

Whats the best way to check if a model uses GPU (in code?) I noted that I get a llama.cpp: using Vulkan on AMD Radeon RX 6750 XT (RADV NAVI22) when GPU is being utilized but can't find a way to check for it in code. hasGpuDevice only reports whether my machine has a GPU available.

[iimez]

gpu-2023-10-31-01-16-03.md

Switching models works flawlessly now. Looking great, lets get this out!