Skip to content

A barebone / prototype of a B2B messaging system using Kafka, Logstash, Kibana, and Spring boot

License

Notifications You must be signed in to change notification settings

wenchien/B2B-Messaging-System-Architecture

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Java Versions Maven Kafka JUnit Log4j2 LogStash Kibana ElasticSearch

Table of Contents
  1. About The Project
  2. System Diagram
  3. Roadmap
  4. Contributing
  5. License

About The Project

The project aims to migrate monolithic B2B messaging system to a new and more modern system that utizilies Spring Boot, Docker, Microservices and Monitoring (logstash and ElasticSearch) that will resolve a lot of the maintenance costs / time that backend developers need to spend.

Do note this is a barebone / prototype / proof of concept. There will be flaws / ideas that haven't been considered / evaluated thoroughly but it's generally better to have something down (something is better than nothing afterall).

By utilizing such design, the frontend (i.e. enterprise portal) can easily be migrated to newer technology (React, AngularJS). It should speed up development time comparing to a direct client-to-microservice communication architecture.

(back to top)

Built With

  • Java Versions
  • Maven
  • Kafka
  • JUnit
  • Log4j2
  • LogStash
  • Kibana
  • ElasticSearch
  • See the pom.xml for all relevant dependencies

(back to top)

System Diagram

Main B2B System



This system aims to solve:

  1. Single service / monolithic service deployment -> since we constantly receive messages sent from our trading partners / customers, there's no reason code re-deployment / features changes should cause disturbance in message processing.
  2. Features deployment time -> resolves similar to the issue mentioned in #1 thanks to kafka's partitioning feature.
  3. Long inheritance list from template pattern -> the original system's class inheritance list is way too long that utlimately creates a tightly coupled relationship. Any additional feature that wish to be added will generally result in some form of modification and in SOLID design principle, it is not wise.
  4. Message validation -> the new system allows custom validation done at various event stages. I've also wrote a small library that goes well with this design. See StringFirewall
  5. Maintenance of LOGS -> the original system has logs everywhere / under applications' log folders (Tomcat, Kafka, Airflow, ETL tools...etc). An implementation of LogStash + ElasticSearch should resolve this issue and hopefully reduce maintenance cost.
  6. EDI / X12 Message processing time
  7. Message monitoring
  8. System component can be moved easily -> many components can then be moved to internal docker containers or cloud services (AWS..etc.)
  9. And many many more...

Sub-modules / Microservices:

Each sub-module utilizes chain of responsibility design pattern. It is done this way to ensure better code readability and scalability. In addition, each sub-module will implement event validation to ensure consistencies.


Improved message marshalling / unmarshalling process



Improved file processing (data population from static files)



Improved dynamic data source population (connecting to / getting data from database / REST API...etc.)

(back to top)

Roadmap

  • Version 1.0.0 release

(back to top)

License

MIT License

Copyright 2022 Wen Chien Chang

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

(back to top)

About

A barebone / prototype of a B2B messaging system using Kafka, Logstash, Kibana, and Spring boot

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published