-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding llama models support #33
Comments
Hi @petric3, thanks for the issue! I think the problem you're seeing is caused by a couple of things. The first is that meta-llama/Meta-Llama-3-8B is unfortunately not available for free (serverless) access via the Inference API. From the model page:
The second issue is that the Thanks for letting me know that the example hangs indefinitely when given a |
As for why # 1) Does not work
curl https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-8B-Instruct \
-X POST \
-d '{"inputs": ["The quick brown fox", "jumps over the lazy dog"]}' \
-H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
-H 'Content-Type: application/json'
# 2) Works
curl https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-8B-Instruct \
-X POST \
-d '{"inputs": "The quick brown fox"}' \
-H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
-H 'Content-Type: application/json'
# 3) Works
curl https://api-inference.huggingface.co/models/gpt2 \
-X POST \
-d '{"inputs": ["The quick brown fox", "jumps over the lazy dog"]}' \
-H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
-H 'Content-Type: application/json' Currently,
Clearly that isn't the case for all models though. I've opened #35 to track that issue. |
@Kardbord Thank you for the explanation, it makes sense. Also thanks for the links to the dedicated endpoints, may come handy. Best of wishes with your house move 🌞 |
Thank you for your program.
I was trying to do some sentiment analysis. I took your example and tried to switch the models, simply from
hfapigo.RecommendedTextClassificationModel
tometa-llama/Meta-Llama-3-8B
, but the response is not returned (waiting for it indefinitely). I also tried to make it work with thellama
models on other examples, but no response. Could you add an example on How to use thellama
models via the HugFace API/Interface?The text was updated successfully, but these errors were encountered: