Code

[Feat] Add streaming to chatbot#1272

Code

Merged

rahul-tuli merged 11 commits intomainneuralmagic/deepsparse:mainfrom add-streaming-to-chatbotneuralmagic/deepsparse:add-streaming-to-chatbotCopy head branch name to clipboardOct 16, 2023

+65-36

Commits

Commits on Oct 11, 2023

• Add changes needed on parent class side

  

  rahul-tuli

  authored and

  mgoin

  committed
  1d3c54f

  Copy full SHA for 1d3c54f

• Add stream mode to chatbot

  Show description for 3b9a00a
  

  rahul-tuli

  authored and

  mgoin

  committed
  3b9a00a

  Copy full SHA for 3b9a00a

• Update sampling temperature to temperature

  

  rahul-tuli

  authored and

  mgoin

  committed
  e4fb27c

  Copy full SHA for e4fb27c

• Update src/deepsparse/transformers/infer.py

  mgoin

  committed
  9886574

  Copy full SHA for 9886574

• Rebase and fix streaming

  mgoin

  committed
  8c2e74f

  Copy full SHA for 8c2e74f

• Fix broken data pathway

  rahul-tuli

  committed
  a29478f

  Copy full SHA for a29478f

• Add stream to data mode

  Show description for 86379d1
  rahul-tuli

  committed
  86379d1

  Copy full SHA for 86379d1

• Merge branch 'main' into add-streaming-to-chatbot

  rahul-tuli

  authored
  f92c313

  Copy full SHA for f92c313

Commits on Oct 12, 2023

• Update prompt prefill tok/s

  mgoin

  committed
  68d7c59

  Copy full SHA for 68d7c59

• Merge branch 'main' into add-streaming-to-chatbot

  rahul-tuli

  authored
  c59607d

  Copy full SHA for c59607d

Commits on Oct 16, 2023

• Merge branch 'main' into add-streaming-to-chatbot

  rahul-tuli

  authored
  2e12bac

  Copy full SHA for 2e12bac