🎉Mayhem server fully operational!

⚒️demo client now supports custom thread count

CMakeLists.txt

@@ -22,8 +22,8 @@ if (CMAKE_BUILD_TYPE STREQUAL "Debug")
     # ASan and TSan are incompatible, so we can't use them together.
     # And because we use std::shared_ptr, it is most likely that there is no need for ASan.
     # However, TSan is somehow too strict, enable it only temporarily.
-    # set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=undefined,thread")
-    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address,undefined")
+    # set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=thread,undefined")
+    # set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address,undefined")
 endif()
 
 # strict warnings

README.md

@@ -15,6 +15,31 @@
 > [!IMPORTANT]
 > Currently, this library only targets Linux platforms. It is not the library you are looking for if you want cross-platform support.
 
+## Features
+
+Although **minet core** is simple, it indeed has some cool features.😎
+
+- **Easy to Use**: It is super easy to create a server with just a few lines of code.
+- **Flexible Configuration**: You can configure the server with a JSON file.
+- **Dependency Injection**: It uses IoC to manage components, and you can replace them with your own.
+- **Multiple Servers**: It provides multiple server implementations, good for study and comparison.
+- **Extensible**: It is designed to be extensible, and you can build your framework on top of it.
+
+## Purpose
+
+The main purpose for **minet core** is to learn the mechanism behind an HTTP server. The key is to understand how the server accepts and handles requests, and how it sends responses back. It is also a good practice to learn how to design a library that is easy to use and flexible to extend.
+
+## Key Points
+
+The most important part of **minet core** is the design of the server. The server is responsible for accepting requests. As my knowledge expands, I have implemented three different servers.
+
+The first is `Basic` server, accepting requests in one thread. It is simple and easy to understand, but it is not efficient.
+
+The second is `Threaded` server, which handles requests in multiple threads with a thread pool.
+It is way more efficient than the `Basic` server, but it is not the best.
+
+The third is `Mayhem` server, based on `Threaded` server, but also utilizes epoll for asynchronous I/O. It is the most efficient server in **minet core**.
+
 ---
 
 # Getting Started
@@ -47,6 +72,29 @@ If you want to update the submodules, you can run the following command, or `./s
 git submodule update --recursive
 ```
 
+## Build the Library
+
+> [!INFO]
+> By default, the sanitizer options are commented out in `CMakeLists.txt`, as they are not compatible with debugger. If you want to use them, uncomment the corresponding lines.
+
+### Have a Try
+
+Before you decide to use **minet-core** in your project, you can build the demo server to see how it works. The following commands will build libray and the demo server. You can also jump to the following section to see the bundled demo.
+
+```bash
+./script/build.sh debug   # build the debug version
+./script/build.sh release # build the release version
+```
+
+### Use it in Your Project
+
+To use **minet core** in your project, simply add it as a subdirectory in your CMake project. Then link your target with `minetcore`.
+
+```cmake
+add_subdirectory("minet-core")
+target_link_libraries(your-app-name PRIVATE minetcore)
+```
+
 ## Your First Server
 
 It is super easy to create a server with **minet-core**, just a few lines of code and you are ready to go.😆
@@ -86,6 +134,7 @@ The example above might be a little too simple. For a more comprehensive demonst
 # in another terminal
 ./script/demo.sh client        # 4 processes, each sending 10 requests
 ./script/demo.sh client N      # 4 processes, each sending N requests
+./script/demo.sh client N M    # 2^M processes, each sending N requests
 ```
 
 Basic server handles requests in one thread, so you'll see the client return one response at a time. Mayhem server handles requests asynchronously, so you'll see a significant speedup in the client.

demo/appsettings.mayhem.json

@@ -3,7 +3,7 @@
         "name": "Mayhem",
         "threads": 4,
         "capacity": 1024,
-        "port": 5001
+        "port": 5000
     },
     "logging": {
         "level": "Debug",

demo/appsettings.threaded.json

@@ -1,10 +1,8 @@
-// All values have default value marked with *
-// i.e. This file can be as simple as {}
 {
     "server": {
-        "name": "Mayhem",
-        "threads": 4, // by default 0, using all cores
-        "capacity": 1024, // by default 1024
+        "name": "Threaded",
+        "threads": 4,
+        "capacity": 1024,
         "port": 5000
     },
     "logging": {
@@ -34,4 +32,4 @@
             }
         }
     }
-}
+}

demo/demo.cpp

@@ -7,14 +7,14 @@ Ref<Logger> logger;
 static void ping(const TextRequest& request, TextResponse& response)
 {
     logger->Info("Ping request received:\n-----\n{}\n-----", request.Request().ToString());
-    std::this_thread::sleep_for(std::chrono::milliseconds(200));
+    std::this_thread::sleep_for(std::chrono::milliseconds(40));
     response.Text().append("pong");
 }
 
 static void echo(const TextRequest& request, JsonResponse& response)
 {
     logger->Info("Echo request received:\n-----\n{}\n-----", request.Request().ToString());
-    std::this_thread::sleep_for(std::chrono::milliseconds(500));
+    std::this_thread::sleep_for(std::chrono::milliseconds(60));
     response.Json()["status"] = "ok";
     response.Json()["message"] = request.Text();
 }

include/minet/common/Http.h

@@ -7,6 +7,8 @@
 
 #include "minet/common/Base.h"
 
+#include <string>
+
 MINET_BEGIN
 
 namespace http
@@ -48,14 +50,28 @@ enum class HttpMethod : uint8_t
     HEAD,
     OPTIONS,
     TRACE,
-    PATCH
+    PATCH,
+    INVALID
 };
 
 /**
  * @brief Get corresponding string representation of HTTP method.
  */
 const char* HttpMethodToString(HttpMethod method);
 
+/**
+ * @brief Get corresponding HTTP method from string.
+ */
+HttpMethod HttpMethodFromString(const std::string& method);
+
+enum class HttpVersion : uint8_t
+{
+    HTTP_1_0,
+    HTTP_1_1,   // Currently only support HTTP/1.1
+    HTTP_2_0,
+    INVALID
+};
+
 /**
  * @brief Get corresponding description of HTTP status codes.
  * @param statusCode HTTP status code.

include/minet/core/HttpContext.h

@@ -7,6 +7,10 @@
 
 #include "minet/common/Http.h"
 
+#include "minet/utils/Parser.h"
+
+#include "io/StreamReader.h"
+
 #include <string>
 #include <unordered_map>
 
@@ -29,6 +33,7 @@ using HeaderCollection = std::unordered_map<std::string, std::string>;
 struct HttpRequest
 {
     http::HttpMethod Method;
+    http::HttpVersion Version;
 
     /**
      * @brief The content length.
@@ -142,4 +147,37 @@ int CreateHttpContext(int fd, Ref<HttpContext>* context);
  */
 int DestroyHttpContext(const Ref<HttpContext>& context);
 
+/**
+ * @brief Asynchronously parse HTTP context.
+ */
+class AsyncHttpContextBuilder
+{
+public:
+    AsyncHttpContextBuilder(const network::AcceptData& data);
+
+    /**
+     * @brief Parse the HTTP context.
+     * @note This may only be able to parse part of the request.
+     * @return
+     *      1: Parse completed, call GetContext to get the result.
+     *      0: Parse not completed, call again.
+     *    < 0: Error occurred.
+     */
+    int Parse();
+
+    /**
+     * @brief Get the parsed HTTP context.
+     * @return The parsed HTTP context.
+     */
+    const Ref<HttpContext>& GetContext()
+    {
+        return _context;
+    }
+
+private:
+    Ref<HttpContext> _context;
+    Ref<io::StreamReader> _reader;
+    http::AsyncHttpRequestParser _parser;
+};
+
 MINET_END

include/minet/utils/Parser.h

@@ -0,0 +1,103 @@
+/**
+ * @author Tony S.
+ * @details A tiny syntax parser for HTTP request.
+ */
+
+#pragma once
+
+#include "minet/common/Base.h"
+
+#include <string>
+
+MINET_BEGIN
+
+struct HttpRequest;
+
+namespace http
+{
+
+/**
+ * @brief Clean up the URL path.
+ * If the URL path does not start with '/', add one, and remove trailing '/'.
+ * e.g. "abc" -> "/abc", "abc/" -> "/abc", "/abc" -> "/abc", "/abc/" -> "/abc"
+ */
+std::string CleanPath(const std::string& path);
+
+/**
+ * @brief Parse HTTP request.
+ * @param request The output request.
+ * @return 0 on success, error codes are as follows:
+ *   1: Failed to read from the stream.
+ *   2: Failed to parse the prologue.
+ *   3: Failed to parse one of the headers.
+ */
+int ParseHttpRequest(HttpRequest* request);
+
+/**
+ * @brief Asynchronously parse HTTP request.
+ */
+class AsyncHttpRequestParser final
+{
+public:
+    AsyncHttpRequestParser(HttpRequest* request);
+
+    /**
+     * @brief Feed a character to parse the request.
+     * @return
+     *      1: The request is done.
+     *      0: The request is still in progress.
+     *    < 0: Error occurred, value is the error state.
+     */
+    int Feed(char ch);
+
+private:
+    enum class State
+    {
+        Start = 10, // avoid conflict with io::StreamStatus
+        Method,
+        Space1,
+        Path,
+        Space2,
+        Version,
+        NewLine,
+        HeaderKey,
+        Colon,
+        Space3,
+        HeaderValue,
+        HeaderNewLine,
+        EndLine,
+        Body,
+        Done,
+        Error
+    };
+
+    int _ParseStart(char ch);
+    int _ParseMethod(char ch);
+    int _ParseSpace1(char ch);
+    int _ParsePath(char ch);
+    int _ParseSpace2(char ch);
+    int _ParseVersion(char ch);
+    int _ParseNewLine(char ch);
+    int _ParseHeaderKey(char ch);
+    int _ParseColon(char ch);
+    int _ParseSpace3(char ch);
+    int _ParseHeaderValue(char ch);
+    int _ParseHeaderNewLine(char ch);
+    int _ParseEndLine(char ch);
+    int _ParseBody(char ch);
+
+    int _ParseDone(char ch);
+    int _ParseError(char ch);
+
+    using ParseFunc = int (AsyncHttpRequestParser::*)(char);
+    ParseFunc _handler;
+    std::string _token;
+    std::string _key;
+
+    HttpRequest* _request;
+    int _error;
+};
+
+} // namespace http
+
+MINET_END

scripts/demo.sh

@@ -66,14 +66,15 @@ function run_client() {
             send_requests $ROUND
         fi
     else
-        ROUND=${1:-10}
+        M_ROUND=${1:-10}
+        M_LEVEL=${2:-2}
         start=`date +%s`
 
-        bash $0 client _ 2 $ROUND
+        bash $0 client _ $M_LEVEL $M_ROUND
         wait $!
 
         end=`date +%s`
-        echo "Sent $((ROUND*4)) ($ROUND * 4) requests in $((end-start)) seconds"
+        echo "Sent $((M_ROUND*(2**M_LEVEL))) ($M_ROUND * $((2**$M_LEVEL))) requests in $((end-start)) seconds"
     fi
 }
 

src/common/Http.cpp

@@ -25,11 +25,51 @@ const char* HttpMethodToString(HttpMethod method)
         return "TRACE";
     case HttpMethod::PATCH:
         return "PATCH";
+    case HttpMethod::INVALID:
+        return "INVALID";
     }
 
     return "UNKNOWN";
 }
 
+HttpMethod HttpMethodFromString(const std::string& method)
+{
+    if (method == "GET")
+    {
+        return HttpMethod::GET;
+    }
+    if (method == "POST")
+    {
+        return HttpMethod::POST;
+    }
+    if (method == "PUT")
+    {
+        return HttpMethod::PUT;
+    }
+    if (method == "DELETE")
+    {
+        return HttpMethod::DELETE;
+    }
+    if (method == "HEAD")
+    {
+        return HttpMethod::HEAD;
+    }
+    if (method == "OPTIONS")
+    {
+        return HttpMethod::OPTIONS;
+    }
+    if (method == "TRACE")
+    {
+        return HttpMethod::TRACE;
+    }
+    if (method == "PATCH")
+    {
+        return HttpMethod::PATCH;
+    }
+
+    return HttpMethod::INVALID;
+}
+
 const char* StatusCodeToDescription(int statusCode)
 {
     using namespace status;