Pub-sub Distributed System Part2

This project is an extention on Pub-sub distributed system part1.

You can find part 1 here: https://github.com/CS682-S22/distributed-system-part1-Jennytang1224

I'm adding two additional features in this project:

1. Bully Election

2. Heartbeat message

3. Membership table

4. Replication: both asynchronous and synchronous replications

5. Push-based broker

6. Performance analysis

Current System:

The existing program is a simple version of Kafka pub-sub system:

• Producers: will send data to all brokers through load balancer.
• Load Balancer: will calculate the partition number based the "key" inside of data and pass data to the brokers.
• Brokers: maintain their own topic map that stores data based on assigned partition number.
• consumers: requesting data from brokers given the topic and starting position.

Push-based Broker

Currently I have pull-based broker with partition implemented.

Performance Analysis

Variables:

• pull-based broker and push-based broker
• number of partitions
• number of producers
• number of brokers
• number of topics
• randomly-generate partitionID VS. mod-generated partitionID

Measurement:
will use runtime as measurement to compare result by testing on different variables

Paper:
will producer a short paper to descirbe the result and analyze the data with graphs

Milestones:

By May 10

• implement push-based brokers
• set breakpoints for runtime measurement
• experiment on pull-based broker and push-based broker

By May 12

• experiment on number of data
• experiment on number of partitions
• experiment on number of brokers
• experiment on randomly-generate partitionID VS. mod-generated partitionID
• experiment on consumer runtime
• experiment on loadbalancer runtime

By May 15

• generate graphs
• write short paper