Move to better local-search based on Pagefind

[yujonglee]

I couldn't get related pages with the current search bar of docs.mistral.ai, even for simple queries.

The problem is the underlying local search index called Lunr. I agree that local search can be enough for the current size of docs, but Lunr hasn't been updated for the last 4 years and there's a better alternative called Pagefind.

After the switch from Lunr to Pagefind, I found the search results got way better.

To try locally:

npx pnpm@8 i && npx docusaurus build && npx docusaurus serve

All source code related to search index and UI is MIT licensed. For UI customization, there's documentation too.

[socket-security]

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/@getcanary/web@0.0.64	None	0	367 kB	yujonglee

View full report↗︎

[fracapuano]

Hey @yujonglee 👋 thank you very much for this PR! I see there are some conflicts preventing merging.

• Would love to get a look at this PR more in detail after those have been resolved. - - Would you mind rebasing and adding the necessary commits on this PR when you can?
• Thank you 🙏

[yujonglee]

@fracapuano Should be fixed now!

There's big lockfile change because previous lockfile was v6, although v9 is specified.

platform-docs-public/package.json

Line 52 in cc1dbe1

"packageManager": "pnpm@9.8.0+sha256.56a9e76b51796ca7f73b85e44cf83712862091f4d498c0ce4d5b7ecdc6ba18f7"

Can confirm search is working.
Note that it require local search index to be created at least once. (to get search working).

So pnpm build is needed at least once. After that, Search will work locally even with pnpm start.