Enhancement: Custom Token Rates for Endpoints (including Custom)

[danny-avila]

What features would you like to see added?

Define own token rates via librechat.yaml

More details

Configuration from YAML file to affect and add to these values

// api/models/tx.js
/**
 * Mapping of model token sizes to their respective multipliers for prompt and completion.
 * @type {Object.<string, {prompt: number, completion: number}>}
 */
const tokenValues = {
  '8k': { prompt: 30, completion: 60 },
  '32k': { prompt: 60, completion: 120 },
  '4k': { prompt: 1.5, completion: 2 },
  '16k': { prompt: 3, completion: 4 },
  'gpt-3.5-turbo-1106': { prompt: 1, completion: 2 },
  'gpt-4-1106': { prompt: 10, completion: 30 },
};

// api/utils/tokens.js
const openAIModels = {
  'gpt-4': 8187, // -5 from max
  'gpt-4-0613': 8187, // -5 from max
  'gpt-4-32k': 32758, // -10 from max
  'gpt-4-32k-0314': 32758, // -10 from max
  'gpt-4-32k-0613': 32758, // -10 from max
  'gpt-3.5-turbo': 4092, // -5 from max
  'gpt-3.5-turbo-0613': 4092, // -5 from max
  'gpt-3.5-turbo-0301': 4092, // -5 from max
  'gpt-3.5-turbo-16k': 16375, // -10 from max
  'gpt-3.5-turbo-16k-0613': 16375, // -10 from max
  'gpt-3.5-turbo-1106': 16375, // -10 from max
  'gpt-4-1106': 127990, // -10 from max
  'mistral-': 31990, // -10 from max
};

// Order is important here: by model series and context size (gpt-4 then gpt-3, ascending)
const maxTokensMap = {
  [EModelEndpoint.openAI]: openAIModels,
  [EModelEndpoint.custom]: openAIModels,
  [EModelEndpoint.google]: {
    /* etc. */

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct