What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks
-
Updated
Jul 26, 2024 - Jupyter Notebook
#
llms-benchmarking
Here are 55 public repositories matching this topic...
A comprehensive set of LLM benchmark scores and provider prices.
-
Updated
Feb 25, 2025 - JavaScript
BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models
-
Updated
Feb 21, 2025 - Python
Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
metrics good-first-issue llm prompt-engineering generative-ai llmops llm-eval llm-tools llm-evaluation llm-evaluation-toolkit llms-benchmarking llm-evaluation-framework
-
Updated
Feb 13, 2025 - Python
How good are LLMs at chemistry?
-
Updated
Mar 3, 2025 - Python
Language Model for Mainframe Modernization
-
Updated
Aug 23, 2024 - Python
Thematic Generalization Benchmark: measures how effectively various LLMs can infer a narrow or specific "theme" (category/rule) from a small set of examples and anti-examples, then detect which item truly fits that theme among a collection of misleading candidates.
-
Updated
Feb 27, 2025
Benchmark that evaluates LLMs using 601 NYT Connections puzzles extended with extra trick words
-
Updated
Feb 27, 2025 - Python
CompBench evaluates the comparative reasoning of multimodal large language models (MLLMs) with 40K image pairs and questions across 8 dimensions of relative comparison: visual attribute, existence, state, emotion, temporality, spatiality, quantity, and quality. CompBench covers diverse visual domains, including animals, fashion, sports, and scenes.
benchmark reasoning vision-and-language multimodal-deep-learning human-annotation foundation-models large-language-models llms vision-language-model multimodal-large-language-models evaluation-llms llms-benchmarking
-
Updated
Aug 6, 2024 - Jupyter Notebook
Training and Benchmarking LLMs for Code Preference.
-
Updated
Nov 15, 2024 - Python
Develop reliable AI apps
-
Updated
Feb 20, 2025 - Svelte
The data and implementation for the experiments in the paper "Flows: Building Blocks of Reasoning and Collaborating AI".
ai competitive-programming agents competitive-programming-contests competitive-coding llms llms-reasoning llms-benchmarking aiflows
-
Updated
Feb 12, 2024 - Python
Restore safety in fine-tuned language models through task arithmetic
-
Updated
Mar 28, 2024 - Python
A minimalist benchmarking tool designed to test the routine-generation capabilities of LLMs.
-
Updated
Nov 28, 2024 - Python
-
Updated
Jan 2, 2025 - Python
Code and data for Koo et al's ACL 2024 paper "Benchmarking Cognitive Biases in Large Language Models as Evaluators"
nlp evaluation bias bias-detection llm llms llm-evaluation llms-benchmarking llm-as-judge llm-as-a-judge llm-as-evaluator
-
Updated
Feb 16, 2024 - Jupyter Notebook
Join 15k builders to the Real-World ML Newsletter ⬇️⬇️⬇️
-
Updated
Apr 19, 2024 - Python
FaceXBench: Evaluating Multimodal LLMs on Face Understanding
attributes face face-recognition gender-classification facial-expression-recognition race-detection age-estimation crowd-counting face-antispoofing multimodal face-segmentation face-perception deepfake-detection headpose-estimation llms llms-benchmarking multimodal-llms
-
Updated
Feb 4, 2025 - Python
Awesome Mixture of Experts (MoE): A Curated List of Mixture of Experts (MoE) and Mixture of Multimodal Experts (MoME)
artificial-intelligence sparse moe load-balancing multimodal-learning mixture-of-experts mome gating-network foundation-models large-language-models llms large-language-model large-vision-language-models expert-network llms-reasoning llms-benchmarking mixtrure-of-multimodal-experts sparse-moe sparse-mixture-of-experts sparse-mixture-of-multimodal-experts
-
Updated
Jan 10, 2025
Improve this page
Add a description, image, and links to the llms-benchmarking topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llms-benchmarking topic, visit your repo's landing page and select "manage topics."