Outfit Recommendation System Documentation

1. Title

Integration of Multi-Modal Framework for Enhanced Fashion Recommendation Systems

2. Abstract

The ChicMate Outfit Recommendation System focuses on improving personalized fashion recommendations through a multi-modal framework. The project integrates user preferences, image-based analysis, and an intuitive UI to enhance fashion e-commerce experiences. Current progress includes dataset preparation, preprocessing, feature extraction, and foundational UI development.

3. Introduction

Fashion e-commerce demands sophisticated systems to understand user preferences and item features. ChicMate leverages deep learning techniques to provide personalized outfit recommendations based on wardrobe data uploaded by users. Initial work involves managing wardrobes and filtering recommendations by weather and occasion.

Completed --

• Image preprocessing: Ensures consistency in neural network inputs.
• Feature extraction: Uses pre-trained CNNs for embedding generation.
• User Interface Development: Enables wardrobe management and interactive recommendations.

4. Dataset Loading and Preprocessing

Dataset Details

• Dataset Used: In-shop Clothes Retrieval Dataset
• Extracted Folders:
  • img_highres: High-resolution images of clothing items.
  • Benchmark folders: Metadata for classification tasks.
• Preprocessing Workflow
  • Objective: Resize images to a uniform 224x224 resolution for consistent neural network input.
  • Tools Used: Python, OpenCV.
  • Techniques:
    • Handled .DS_Store files to avoid non-image errors.
    • Applied batch preprocessing for scalability.
  • Output: Preprocessed image directory.

5. Feature Extraction

Purpose

Extract visual features for similarity-based recommendations.

Methodology

Pre-trained Model: Feature embeddings extracted using CNN models like ResNet.

• Approach: Images passed through the model to obtain high-dimensional embeddings. Embeddings stored for downstream tasks, including similarity matching.

• Future Alignment: Integrate Siamese Neural Networks for precise similarity analysis as proposed.

6. User Interface

Framework:

React Native with Expo

Screens Developed:

• Home Screen:

  Central navigation to Wardrobe and Recommendation screens.

• Wardrobe Management:

  Add/Delete clothing items using Expo Image Picker. Display wardrobe in a grid view.

• Recommendation Screen:

  Apply filters (e.g., weather, occasion) for personalized suggestions. Button to fetch recommendations. Design Highlights:

• Clean and responsive layout using react-native-paper.

  Dynamic navigation enabled with Expo Router.

7. Pending Components:

• Textual data analysis (e.g., descriptions, user reviews).
• Hybrid model development (Siamese + Multi-Scale CNN).
• Integration of user preferences for personalized scoring.

8. Future Work

Model Development:

• Implementation of Siamese Neural Networks for image similarity.

• Integration of Multi-Scale CNNs for text and visual feature fusion.

• System Refinement:

  • Extend filters (e.g., seasonality, color preferences).
  • Optimize feature storage for performance.

• Testing and Deployment:

  • Evaluation with precision, recall, and F1 metrics.
  • Conducting A/B testing against baseline VGG19 models.

9. Technologies Used

• Programming Language: Python
• Deep Learning Framework: TensorFlow/PyTorch
• Frontend: React Native with Expo
• Libraries:
  • Image Preprocessing: OpenCV
  • UI Development: react-native-paper, expo-image-picker
• Storage: Preprocessed datasets and embeddings saved locally.

10. Conclusion

The ChicMate Outfit Recommendation System is advancing towards integrating a multi-modal framework for fashion recommendations. Current progress demonstrates effective preprocessing, foundational UI, and feature extraction. The next steps involve implementing advanced neural network architectures and optimizing the recommendation system for deployment.