-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
17 changed files
with
988 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,204 @@ | ||
|
||
# FMP (Fragmentation and Multipath Protocol) | ||
|
||
**FMP** is a lightweight data transmission protocol designed for secure, efficient, and decentralized communication. It fragments data into dynamically sized pieces, encrypts them with robust cryptographic algorithms, and routes them through optimized multi-path channels. The protocol ensures low latency, robust tamper resistance, and scalability, making it ideal for applications requiring cutting-edge security and performance. | ||
|
||
--- | ||
|
||
## Table of Contents | ||
|
||
- [Features](#features) | ||
- [Installation](#installation) | ||
- [Usage](#usage) | ||
- [Sending Data](#sending-data) | ||
- [Receiving Data](#receiving-data) | ||
- [File Structure](#file-structure) | ||
- [Testing](#testing) | ||
- [Benchmarking](#benchmarking) | ||
- [Contributing](#contributing) | ||
- [License](#license) | ||
|
||
--- | ||
|
||
## Features | ||
|
||
- **Data Fragmentation:** Splits data into manageable fragments with metadata for reassembly. | ||
- **Encryption:** Secures each fragment using authenticated encryption (AES-GCM) with unique nonces. | ||
- **Adaptive Multi-Path Routing:** Routes fragments through dynamically scored paths for optimal delivery. | ||
- **Reassembly:** Collects and reassembles fragments securely at the destination. | ||
- **Error Handling:** Validates fragment integrity and handles missing fragments with high reliability. | ||
- **Logging:** Provides detailed logs for monitoring and debugging. | ||
- **Scalability:** Efficiently handles large data payloads and high-throughput scenarios. | ||
|
||
--- | ||
|
||
## Installation | ||
|
||
### Prerequisites | ||
|
||
- **Python 3.7 or higher**: Ensure Python is installed. You can download it from [python.org](https://www.python.org/downloads/). | ||
- **pip**: Python package installer. It usually comes with Python. Verify by running `pip --version`. | ||
|
||
### Steps | ||
|
||
1. **Clone the Repository:** | ||
|
||
```bash | ||
git clone https://github.com/yourusername/fmp.git | ||
cd fmp | ||
``` | ||
|
||
2. **Create a Virtual Environment:** | ||
|
||
```bash | ||
python3 -m venv venv | ||
source venv/bin/activate # On Windows use `venv\Scripts\activate` | ||
``` | ||
|
||
3. **Install Dependencies:** | ||
|
||
```bash | ||
pip install --upgrade pip | ||
pip install -r requirements.txt | ||
``` | ||
|
||
--- | ||
|
||
## Usage | ||
|
||
### Sending Data | ||
|
||
1. **Start the Receiver:** | ||
|
||
On the destination machine or terminal, run the receiver script: | ||
|
||
```bash | ||
python scripts/receiver.py | ||
``` | ||
|
||
2. **Send Data from Sender:** | ||
|
||
On the source machine or another terminal, run the sender script: | ||
|
||
```bash | ||
python scripts/sender.py | ||
``` | ||
|
||
**Example Code:** | ||
|
||
```python | ||
from fmp.protocol import FMPProtocol | ||
# Define available paths (IP, port) | ||
paths = [('localhost', 8001), ('localhost', 8002)] | ||
# Initialize protocol | ||
protocol = FMPProtocol(fragment_size=1024, paths=paths) | ||
# Data to send | ||
data = b"Your data here..." | ||
# Send data | ||
protocol.send_data(data) | ||
``` | ||
|
||
### Receiving Data | ||
|
||
**Receiver Script Example (`scripts/receiver.py`):** | ||
|
||
```python | ||
from fmp.protocol import FMPProtocol | ||
# Define available paths (IP, port) | ||
paths = [] # Not used for receiving | ||
# Initialize protocol | ||
protocol = FMPProtocol(fragment_size=1024, paths=paths) | ||
# Function to handle incoming fragments (to be integrated with actual network code) | ||
def handle_incoming_fragment(encrypted_fragment): | ||
data = protocol.receive_data([encrypted_fragment]) | ||
if data: | ||
print("Received Data:", data) | ||
# Note: Integrate this with actual network listening code. | ||
``` | ||
|
||
--- | ||
|
||
## File Structure | ||
|
||
``` | ||
FMP/ | ||
├── fmp/ | ||
│ ├── __init__.py | ||
│ ├── core.py # Unified fragmentation, encryption, and reassembly | ||
│ ├── routing.py # Adaptive routing and path scoring | ||
│ └── protocol.py # Main protocol logic | ||
├── tests/ | ||
│ ├── __init__.py | ||
│ ├── test_core.py | ||
│ ├── test_protocol.py | ||
│ └── test_routing.py | ||
├── benchmarks/ | ||
│ ├── benchmark_latency.py | ||
│ └── benchmark_throughput.py | ||
├── scripts/ | ||
│ ├── sender.py # Example sender script | ||
│ └── receiver.py # Example receiver script | ||
├── README.md | ||
├── requirements.txt | ||
├── setup.py | ||
├── LICENSE | ||
└── .gitignore | ||
``` | ||
--- | ||
## Testing | ||
Run unit tests using `unittest`: | ||
```bash | ||
python -m unittest discover tests | ||
``` | ||
|
||
--- | ||
|
||
## Benchmarking | ||
|
||
### Latency Benchmark | ||
|
||
Measure fragmentation and encryption latency: | ||
|
||
```bash | ||
python benchmarks/benchmark_latency.py | ||
``` | ||
|
||
### Throughput Benchmark | ||
|
||
Measure data throughput: | ||
|
||
```bash | ||
python benchmarks/benchmark_throughput.py | ||
``` | ||
|
||
--- | ||
|
||
## Contributing | ||
|
||
Contributions are welcome! Please follow these steps: | ||
|
||
1. Fork the repository. | ||
2. Create a new branch: `git checkout -b feature/YourFeature` | ||
3. Commit your changes: `git commit -m 'Add some feature'` | ||
4. Push to the branch: `git push origin feature/YourFeature` | ||
5. Open a pull request. | ||
|
||
Please ensure your code follows the project's coding standards and includes relevant tests. | ||
|
||
--- | ||
|
||
## License | ||
|
||
This project is licensed under the MIT License. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
# benchmarks/benchmark_latency.py | ||
|
||
import time | ||
import secrets | ||
from fmp.core import FMPCore | ||
from fmp.protocol import FMPProtocol | ||
import msgpack | ||
|
||
def unique_nonce(): | ||
return secrets.token_bytes(12) # AES-GCM requires 96-bit (12-byte) nonces | ||
|
||
def benchmark_latency(): | ||
# Initialize FMPCore with adequate fragment_size | ||
fragment_size = 100 # 100 bytes data per fragment | ||
master_key = b'0' * 32 # Ensure this is consistent between sender and receiver | ||
nonce_gen = (unique_nonce() for _ in range(10000)) # Generator for unique nonces | ||
|
||
core = FMPCore( | ||
fragment_size=fragment_size, | ||
master_key=master_key, | ||
nonce_generator=lambda: next(nonce_gen) | ||
) | ||
protocol = FMPProtocol( | ||
fragment_size=fragment_size, | ||
paths=[('localhost', 8001), ('localhost', 8002)], | ||
master_key=master_key, | ||
nonce_generator=lambda: next(nonce_gen) | ||
) | ||
|
||
# Prepare data to send | ||
data = b'B' * (fragment_size * 1000) # 1000 fragments, total 100,000 bytes | ||
|
||
# Fragment and encrypt | ||
start_fragment_time = time.time() | ||
encrypted_fragments = core.fragment_and_encrypt(data) | ||
fragmentation_time = time.time() - start_fragment_time | ||
print(f"Fragmented data into {len(encrypted_fragments)} fragments in {fragmentation_time:.4f} seconds.") | ||
|
||
# Simulate sending fragments | ||
# In a real scenario, you'd send the fragment over the network | ||
# For benchmarking, we'll skip actual sending | ||
|
||
# Simulate receiving fragments (for benchmarking, you might reassemble them) | ||
start_decrypt_time = time.time() | ||
try: | ||
decrypted_data = core.decrypt_and_reassemble(encrypted_fragments) | ||
decrypt_time = time.time() - start_decrypt_time | ||
print(f"Decrypted data length: {len(decrypted_data)} in {decrypt_time:.4f} seconds.") | ||
assert decrypted_data == data, "Decrypted data does not match original data." | ||
print("Data integrity verified: Decrypted data matches original data.") | ||
except ValueError as ve: | ||
decrypt_time = time.time() - start_decrypt_time | ||
print(f"Decryption failed after {decrypt_time:.4f} seconds: {ve}") | ||
except AssertionError as ae: | ||
decrypt_time = time.time() - start_decrypt_time | ||
print(f"Assertion Error after {decrypt_time:.4f} seconds: {ae}") | ||
|
||
total_time = fragmentation_time + decrypt_time | ||
print(f"Latency Benchmark Completed in {total_time:.2f} seconds.") | ||
|
||
if __name__ == "__main__": | ||
benchmark_latency() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
|
||
import os | ||
import time | ||
import threading | ||
from fmp.protocol import FMPProtocol | ||
|
||
# Initialize a global list to store results | ||
results = [] | ||
|
||
def scalability_test(): | ||
print("\n--- Running Scalability Test ---\n") | ||
payload_sizes = [100_000, 1_000_000, 10_000_000] # 100 KB, 1 MB, 10 MB | ||
fragment_sizes = [512, 1024, 2048] # Fragment sizes in bytes | ||
|
||
for payload_size in payload_sizes: | ||
for fragment_size in fragment_sizes: | ||
try: | ||
data = b"A" * payload_size | ||
protocol = FMPProtocol(fragment_size=fragment_size, paths=[('localhost', 8001)]) | ||
|
||
# Measure time taken for encryption and decryption | ||
start_time = time.time() | ||
encrypted_fragments = protocol.core.fragment_and_encrypt(data) | ||
protocol.core.decrypt_and_reassemble(encrypted_fragments) | ||
end_time = time.time() | ||
|
||
elapsed_time = end_time - start_time | ||
# Append result | ||
results.append({ | ||
"test": "Scalability", | ||
"payload_size": payload_size, | ||
"fragment_size": fragment_size, | ||
"time_taken": elapsed_time | ||
}) | ||
except Exception as e: | ||
print(f"Error during scalability test for payload={payload_size}, fragment_size={fragment_size}: {e}") | ||
|
||
def concurrency_test(): | ||
print("\n--- Running Concurrency Test ---\n") | ||
|
||
def sender_task(data, fragment_size): | ||
protocol = FMPProtocol(fragment_size=fragment_size, paths=[('localhost', 8001)]) | ||
protocol.send_data(data) | ||
|
||
payload_size = 1_000_000 # 1 MB | ||
fragment_size = 1024 | ||
threads = [] | ||
|
||
try: | ||
start_time = time.time() | ||
|
||
# Create and start multiple threads | ||
for _ in range(10): # Simulate 10 concurrent senders | ||
thread = threading.Thread(target=sender_task, args=(b"A" * payload_size, fragment_size)) | ||
threads.append(thread) | ||
thread.start() | ||
|
||
# Wait for all threads to complete | ||
for thread in threads: | ||
thread.join() | ||
|
||
end_time = time.time() | ||
elapsed_time = end_time - start_time | ||
|
||
# Append result | ||
results.append({ | ||
"test": "Concurrency", | ||
"payload_size": payload_size, | ||
"threads": 10, | ||
"fragment_size": fragment_size, | ||
"time_taken": elapsed_time | ||
}) | ||
except Exception as e: | ||
print(f"Error during concurrency test: {e}") | ||
|
||
def save_summary_to_file(): | ||
# File path for results | ||
output_file = os.path.join(os.getcwd(), "benchmark_results.txt") | ||
|
||
with open(output_file, "w") as file: | ||
file.write("--- Benchmark Results Summary ---\n\n") | ||
if not results: | ||
file.write("No results to display. Ensure tests have been executed.\n") | ||
else: | ||
for result in results: | ||
if result["test"] == "Scalability": | ||
file.write(f"[Scalability Test] Payload: {result['payload_size']} bytes | " | ||
f"Fragment Size: {result['fragment_size']} bytes | " | ||
f"Time: {result['time_taken']:.4f} seconds\n") | ||
elif result["test"] == "Concurrency": | ||
file.write(f"[Concurrency Test] Payload: {result['payload_size']} bytes | " | ||
f"Threads: {result['threads']} | " | ||
f"Fragment Size: {result['fragment_size']} bytes | " | ||
f"Time: {result['time_taken']:.4f} seconds\n") | ||
file.write("\n--- End of Summary ---\n") | ||
print(f"\nSummary saved to {output_file}") | ||
|
||
if __name__ == "__main__": | ||
try: | ||
print("Starting benchmark tests...") | ||
scalability_test() | ||
print("\nScalability test completed.\n") | ||
except Exception as e: | ||
print(f"Error during scalability test: {e}") | ||
|
||
try: | ||
concurrency_test() | ||
print("\nConcurrency test completed.\n") | ||
except Exception as e: | ||
print(f"Error during concurrency test: {e}") | ||
|
||
print("\nSaving summary to file...\n") | ||
save_summary_to_file() |
Oops, something went wrong.