Rust, The new chapter

[MrAliSalehi]

1.Introduction

This proposal aims to outline the envision for Re-implementation of the entire The Open Network using RUST.

At first glance, this proposal appears to be a bold move, considering the pros and cons, it'll also appear as an actual requirement.

the backstory

After working with rust for a few years, I came into the conclusion that it might actually be THE language for the future.
Squeezing every bit of performance while offering security and stability is definitely the most valuable feature of this language.
Every developer who uses an unsafe language will quickly experience the constant stress of accidentally blowing something up and causing harm. There are plenty of examples of these accidents happening, I'm sure you are already familiar with them, from rockets blowing up in the sky, to memory vulnerabilities causing platforms to lose millions of dollars.
These incidents motivated people to move their codebase to safer languages, and rewrite everything with rust :D.

Technical Points

the point of RIIR is not just for the sake of the meme, there are several benefits which are mentioned below.

Security

it is obvious that rust's main advantage (to cpp/zig..etc) is security and memory safety.
But security is even more important in platforms where there are financial transactions, such as TON. Currently, the ton network seems to be operating without any security vulnerabilities or bugs, but the important note to consider is the fact that you won't need security until you DO. taking actions after problems has accrued is good, but the best thing to do is prevention. Even kernels and fundamental software packages are not safe from these issues. So this can be a excellent point to consider this re-implementation.

Code Design & Quality

The Rust language is a modern and fast-moving tool. It is constantly improving and adapting to the feedbacks and the community needs.

The building, packaging and managing tools are straightforward, productive and unified. Which can be considered "lacking" in c++.

Rust enforces/prefers quality code over fast-written prototype code, which is a suitable for sensitive platforms (not so great for other things) which require stable and reusable code.

Contribution & Community

there is no doubt that an old language which has been around for over 40~ years has a massive community and support.
But the problem is that a gigantic (and sensitive) platform like TON is scary even as an imagination, let alone trying to contribute to it in a scarier language like C++.
Rich documentation and tutorials solves the first part, but switching to rust can solve (or at least reduces it) the second part of that problem.

The Future

Currently, Rust can be considered the language of the future.

As its community grow and mature, more companies will adopt rust. This can be a huge reason to consider switching to rust, since it can be useful to have more contributors and curious people digging into ton platform to learn and experience.

The Path

since this is a big step forward, the actions need to be taken very carefully with precise planning.

Since it would be a great waste of time to analyze and explain the deep details of The path if it were to be rejected. We'll discuss and update this proposal for more details upon basic approval of the idea. So I'll just quickly go over them:

• Investigation & Study

  before doing anything, it is essential to talk to the TON developers about the known issues, pitfalls and ongoing problems with the current implementations.

  The source code of the monorepo needs to be studied and analyzed for finding best designs and getting a deeper understanding of its goals and intentions.

• Design & Architecture

  as you might already know, rust is a "different" language, simply taking designs and architectures from other languages/codebases simply and applying them in rust usually almost always is not going to work. So every aspect of architecture needs to be reconsidered for this project and be planned.

• Phase 1: Basic Implementation

  Writing the core fundamentals of the platform, this is the part that is probably going to take a long time compared to the other phases.

• Phase 2: Testing & Pipelines

  setup test environment. Write integrations/unit tests and ensure 70% (at least) code coverage in early phases.
  Setup CI/CD Pipelines for automated tests (security vulnerabilities and integration tests).

  Continuously check for differences in outcomes of the rust version and the OG version of TON to avoid any breaking changes and unwanted behaviors.

• Phase 3: Ton Core

  implement the core of the open network.

• Phase 4: Validators

  Migrate validators into the rust version.
  Run an actual validator and test for safety features of the core.

To Be Continued

I'll continue with the planning and phases once we get some feedback and positive responses :D.

Timing

since this is just an idea of a possible action, it is not possible to determine the exact deadlines and springs. But it will need a huge amount of time and effort, and it is not a one-night job.
It should replace the core slowly and gradually; no rushes!.

Requirements

• since we need a deep & precise analyses of the ton source, we'll need contact with the ton core developers for guidance and helps (currently the ton documentations are not that good to replace the developer's advice!).

• since this is a complicated task, it is vital to have multiple people working and it and save possibly save some time and effort.

• funds: unspecified

[emiliana32]

I don't want to undermine this project, but I think Rust is over hyped compared to its real potential. Without doubt, when a programmer encounters Rust for the first time, he considers it the holy grail of languages. It has performance comparable to compiled languages such as C/C++ and the security of languages with garbage collectors such as Java/C#. The main disadvantage is its ecosystem (libraries, documentation, development tools, community, etc.) that is still behind and immature. In particular, Rust has lower performance than C/C++ in the compilation phase (official reason and a number of interesting articles on various optimisations to improve it).
According to Google, between 50-80% of the bugs (depending on the years) on Chrome are related to memory management (use-after-free and other problems) and the same applies to other software. These bugs are related both to C++ and to its incorrect use, which is why alternatives are being sought, and Rust seems to be the best available (Chrome, Firefox, Linux kernel, Android, Microsoft and others).
However, in practice the situation is slightly different and Rust only performs comparable to C/C++ in unsafe mode and suffers from several problems as this article shows, but especially this one where we see that C++ is not only faster in execution and compilation, but also more secure.
Regarding C++ and its memory management, an interesting opinion, obviously biased, by Stroustrup who points out that many commonplaces are based on old versions of the language.
Finally, a discussion on the possibility of converting the Linux kernel from C to modern C++, which would solve several problems without the need for Rust.
In conclusion, it does not seem to me that Rust is able to solve all the problems of C++, and that often the problems are not related to the language, but to its use. Why should the TON Foundation invest resources in rewriting the entire project in Rust if there are no real evident advantages? Just to keep up with the fashion of the time?

[ProgramCrafter]

@emiliana32 Sometimes, with introduction of New Technologies, the right call to do is really to abandon old ones as much as possible. At least fearless concurrency (without pretty much ANY reasoning what fields may change in what threads) is worth it. (And shall I remind that Rust is cross-platform, and can execute even in browser with a limited amount of changes?)

TON sources carry signs that some parts were rewritten from Rust into C++. Whether that happened because of lack of developers, miscompilations, non-production-readiness, low number of crates available or something else, I cannot know; however, now we have an existence proof that TVM-based network may be written in Rust - namely, Everscale. Its code is indeed orders of magnitude clearer than TON's.

@MrAliSalehi I don't think this grant will be given; after all, Everscale is out there.

[emiliana32]

@emiliana32 Sometimes, with introduction of New Technologies, the right call to do is really to abandon old ones as much as possible. At least fearless concurrency (without pretty much ANY reasoning what fields may change in what threads) is worth it. (And shall I remind that Rust is cross-platform, and can execute even in browser with a limited amount of changes?)

Do you have proof of your claims? The evidence shows otherwise, as you can see in my previous post.

TON sources carry signs that some parts were rewritten from Rust into C++. Whether that happened because of lack of developers, miscompilations, non-production-readiness, low number of crates available or something else, I cannot know; however, now we have an existence proof that TVM-based network may be written in Rust - namely, Everscale. Its code is indeed orders of magnitude clearer than TON's.

As far as I know, Everscale, formerly freeTON, forked TON back in 2020. However, it is now (almost) a completely different project:

• TON uses an infinite sharding paradigm, while Everscale uses threading (workchains are present in both).
• TON uses the BFT (catchain) consensus algorithm with standard threshold (liveness=33.33%, safety=66.66%) while Everscale uses the SMFT consensus algorithm with custom threshold (usually liveness=safety=50%).
• TON uses the TON Virtual Machine (TVM) that supports different languages (Func, Tact and Tolk) while Everscale Threaded Virtual Machine (TVM) supports (Threaded Solidity).

There are probably other important differences that I left behind.

[ProgramCrafter]

@emiliana32 you'd better read the articles you cite (https://pvs-studio.com/fr/blog/posts/0733/ in particular) since that one is focused on debunking the myths.

If we are trying, collaboratively, to determine what's wrong with Rust and if it can actually be used, you're welcome to open an issue on my personal repo https://github.com/ProgramCrafter/ProgramCrafter/issues.

If we are determining if TON Society (not TON Foundation, mind you) would issue the grant for rewriting blockchain in Rust, then it most probably won't. Putting off the roadmap items by a couple of months isn't nice when the big changes are already planned.