Natural Language Processing (NLP) with Hugging Face

This repository provides a comprehensive guide to using 🤗 Transformers.

Modules

Module 1: Fundamentals of 🤗 Transformers: From Basics to Fine-Tuning

Learn the fundamental concepts of the 🤗 Transformers library, including how Transformer models function. By the end of this section, you’ll know how to utilize a model from the Hugging Face Hub, fine-tune it on a dataset, and share your results.

• 01. Introduction to Transformers Pipeline

• 02. Transformer Pipelines: Behind the Scenes

• 03. Models and Tokenizers

• 04. Handling Multiple Sequences

Module 2: Fundamentals of 🤗 Datasets and Tokenizers for NLP

This module covers the foundational concepts of working with 🤗 Datasets and 🤗 Tokenizers, preparing to independently solve common NLP tasks.

• 01. Processing Large Data

• 02. Full Training with GPU and Accelerator

• 03. Datasets in HuggingFace

• 04. Semantic Search with FAISS

  Section	Description	Links
  Using embeddings for semantic search	Introduction to building a semantic search engine using embeddings.	Transformers Documentation
  Loading and Preparing Dataset	Loading the GitHub Issues dataset and filtering out pull requests to focus on issues with comments.	GitHub Issues Dataset
  Creating Text Embeddings	Using the sentence-transformers library to create embeddings for text data, with a focus on pooling techniques.	Sentence-Transformers Documentation
  Using FAISS for Efficient Similarity Search	Implementing FAISS to create an index for fast similarity searches on the embeddings and conducting nearest neighbor searches.	FAISS Documentation

• 05. Training New Tokenizer