This repository contains a highly scalable course platform that leverages modern ๐ง technologies to provide video streaming in HLS format. The platform includes both an ๐จ admin panel for course management and a user-facing interface for consuming content.
- Frontend: Next.js
- Backend: Nest.js
- Storage: ๐ฅ Amazon S3 (for file storage)
- Queue Service: ๐ข Amazon SQS (for asynchronous task management)
- Database: ๐ MySQL (for data storage and retrieval)
- ๐พ Upload files in chunks using a multipart upload mechanism.
- ๐ฎ Create and manage courses.
- โจ Automatic transcoding of uploaded videos to HLS format.
- ๐ Browse available courses.
- ๐ฅ Stream in HLS format for a smooth viewing experience.
- ๐พ Handles uploads to Amazon S3 using multipart upload for efficient handling of large files.
- ๐ข Publishes SQS events upon successful upload.
- โณ Listens to notifications for triggering the transcoding process.
- โ๏ธ Transcodes into HLS format for adaptive streaming.
- ๐ Scalable and fault-tolerant architecture.
-
Frontend: The Next.js frontend allows admins to upload in chunks using multipart upload. Regular users can browse and view courses.
-
Backend:
- The Nest.js backend processes uploads and communicates with Amazon S3.
- After a upload, it sends a notification to an Amazon SQS queue.
-
SQS Worker:
- A separate service listens to the queue for new events.
- This worker processes the uploaded and transcodes it into multiple HLS formats for adaptive streaming.
-
Storage:
- ๐ฅ Amazon S3 is used for storing and HLS transcoded files.
-
Database:
- ๐ MySQL is used to store information about courses and user data.
- ๐ ๏ธ Node.js (โฅ 16.x)
- ๐ MySQL (โฅ 8.x)
- ๐ AWS account with access to S3 and SQS
-
๐ Clone the repository:
git clone git@github.com:sahariardev/eduScope.git cd eduScope
-
๐ Install dependencies for both frontend and backend:
cd edu-scope-backend npm install cd ../edu-scope-client npm install
-
๐ง Configure environment variables:
-
Create
.env
files in bothfrontend
andbackend
directories. -
Add the following variables:
Frontend
.env
:NEXT_PUBLIC_API_URL=<backend-api-url>
Backend
.env
:DATABASE_URL=<data_url> JWT_TOKEN=<jwt_sign_key> AWS_ACCESS_KEY_ID=<aws-access_key_id> AWS_SECRET_ACCESS_KEY=<aws_secret_key> AWS_VIDEO_UPLOAD_BUCKET=<video_upload_bucket_name> AWS_VIDEO_HLS_BUCKET=<hls_bucket_name> AWS_S3_REGION=<aws_region> AWS_SQS_URL=<sqs_url>
-
๐ง Set up the database:
cd edu-scope-backend npm run migrate
-
๐ Start the development servers:
# Start backend cd edu-scope-backend npm run start:dev # Start frontend cd ../edu-scope-client npm run dev
- Use ๐ ๏ธ Docker or deploy directly to a cloud environment.
- Ensure the
.env
variables are properly configured in the production environment.
- ๐ Build the Next.js app:
npm run build
- Serve using a ๐ hosting service like Vercel or deploy on your own server.
- Deploy the ๐ข SQS worker service alongside the backend or as a standalone service.
- Ensure it has access to the ๐ข queue and ๐ฅ S3 bucket.
- ๐ฅ Upload: Admins upload files in chunks. The frontend communicates with the backend to manage multipart uploads to S3.
- ๐ข Notification: Once the upload is complete, the backend sends an event to the SQS queue.
- โ๏ธ Transcoding: The worker service picks up the event from and transcodes the into multiple HLS formats.
- ๐ฅ Streaming: Users stream the directly from S3 in HLS format.
We welcome ๐ contributions! Please submit a pull request or open an issue for discussion.
This project is licensed under the MIT License.
For any inquiries, please contact: rifatsahariar@gmail.com