Skip to content

Commit

Permalink
Cleanup files and add documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
TheOnlyZac authored Nov 25, 2024
1 parent ab0df3c commit 7ac7395
Show file tree
Hide file tree
Showing 13 changed files with 96 additions and 88 deletions.
63 changes: 0 additions & 63 deletions .gitattributes

This file was deleted.

File renamed without changes
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
<!-- Shields -->
[![Build][build-badge]][build-url] ![code-progress-badge] <!--![data-progress-badge]--> [![Contributors][contributors-badge]][contributors-url] [![Discord Channel][discord-badge]][discord-url] [![Docs][docs-badge]][docs-url] [![Wiki][wiki-badge]][wiki-url]

[<img src="logo.png" style="margin:7px" align="right" width="33%" alt="Sly 1 Decompilation Logo by Cooper941">][docs-url]
[<img src=".github/logo.png" style="margin:7px" align="right" width="33%" alt="Sly 1 Decompilation Logo by Cooper941">][docs-url]

This is a work-in-progress decompilation of [*Sly Cooper and the Thievius Raccoonus*](https://en.wikipedia.org/wiki/Sly_Cooper_and_the_Thievius_Raccoonus) for the PlayStation 2. It builds the NTSC-U version of the game, `SCUS_971.98` (SHA1: `57dc305d`).

Expand Down Expand Up @@ -63,7 +63,7 @@ Copy the file `SCUS_971.98` from your Sly 1 game disc to the `disc` directory of

If it works, you will see this:
```
[XXX/XXX] sha1sum checksum.sha1
[XXX/XXX] sha1sum config/checksum.sha1
out/SCUS_971.98: OK
```

Expand Down
File renamed without changes.
21 changes: 21 additions & 0 deletions config/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Splat Config Files

This directory contains config files for splat, the tool we use to split and disassemble the game binary. The config files are used to tell splat how to split the binary and what to name the output files.

## Files in this directory
- `sly1.yaml` - Main config file for splat. Contains the offsets of each file split in the game binary. Creating new `.c` files involves modifying this file to create a `c` split and then running `python configure.py` to generate the new files.
- `symbol_addrs.txt` - Contains the addresses of symbols in the game binary. Used by splat to properly reference functions and variables in the game code when splitting the binary.
- `checksum.sha1` - SHA1 checksum of the original game binary. Used to verify the integrity of the binary after compilation.

## Adding new splits
To add a new source code file to the project, you must add the split to `sly1.yaml`. Here is an example of some file splits:
```yaml
- [0x1f560, c, P2/brx]
- [0x1fe70, asm, P2/ac]
#- [0x, asm, P2/act]
```
- The first line tells splat to split the binary at offset `0x1f560` and create a new file `P2/brx.c`. It will contain the data from the translation unit starting at `0x1f560` and ending at the next split offset. The `c` tells splat to create a `.c` file in `src/P2/`.
- The second line tells splat to split that the functions starting from offset `0x1fe70` are in the translation unit `P2/ac`, but not to create a new `.c` file.
- The third line is commented out and does nothing, because we know the next translation unit is `P2/act`, but we don't yet know the offset of the first function in that TU.

When splat is run, it will create `.s` assembly files for each non-matching functions in each split in the `asm/nonmatchings/` directory.
1 change: 0 additions & 1 deletion config/readme.txt

This file was deleted.

2 changes: 1 addition & 1 deletion config/sly1.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ segments:

- [0x529e0, c, P2/difficulty]
- [0x53438, c, P2/dl]
- [0x5380C, asm, dl_padding] # Needed to align dmas
- [0x5380C, asm, P2/dl_padding] # Needed to align dmas
- [0x53810, c, P2/dmas]
- [0x53E28, asm, P2/dsp]
#- [0x, asm, P2/dysh]
Expand Down
5 changes: 3 additions & 2 deletions configure.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""
Configure the project for building.
Configures the project for building. Invokes splat to split the binary and
creates build files for ninja.
"""
#! /usr/bin/env python3
import argparse
Expand Down Expand Up @@ -200,7 +201,7 @@ def build(
ninja.build(
ELF_PATH + ".ok",
"sha1sum",
"checksum.sha1",
"config/checksum.sha1",
implicit=[ELF_PATH],
)
else:
Expand Down
6 changes: 6 additions & 0 deletions disc/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Disc Files

This directory is intentionally left empty. It is where you should place files you extract from your own copy of the game. The files you will need are:
- `SCUS_971.98` - The original executable of the game. This file is required to build the project.

Extract your original ELF file and place it in this directory before building the project.
1 change: 0 additions & 1 deletion disc/readme.txt

This file was deleted.

32 changes: 17 additions & 15 deletions docs/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# So you want to decompile Sly 1...
# Contributing Guide

## So you want to decompile Sly 1...

You've come to the right place! We welcome new contributors of all skill levels. Even if you've never tried game hacking, modding, or reverse-engineering, we have a dedicated community of modders and reverse-engineers to help you get started. Feel feel to join our [Discord server](https://discord.gg/2GSXcEzPJA) and ask for help in #sly-research.

Expand All @@ -12,17 +14,18 @@ By following this guide, you will learn how to fork the repo on GitHub, choose a
2. [Find a function to match](#find-a-function-to-match)
3. [Match the function using Decomp.me](#match-the-function-using-decompme)
4. [Integrate the matched code](#integrate-the-matched-code)
5. [Make a Pull Request](#make-a-pull-request)
5. [Conclusion](#conclusion)


## Getting Started
## Getting started

If you haven't used Git before, follow the [Beginner's Guide](/docs/BEGINNERSGUIDE.md) to learn how to fork the repo, make changes, and submit a pull request. If you are already familiar with Git, continue on with this guide.
If you haven't used Git before, follow the [Beginner's Guide](/docs/BEGINNERSGUIDE.md) to learn how to fork the repo, make changes, and submit a pull request when you're done. If you're already familiar with Git, continue on with this guide.

Start by following the instructions in the [README](/README.md) to build the project. If you see the following in the terminal, you're ready to start contributing:
First, follow the instructions in the [README](/README.md) to build the project. If you see the following in the terminal, you're ready to start contributing:

```
[XXX/XXX] sha1sum checksum.sha1
[XXX/XXX] sha1sum config/checksum.sha1
out/SCUS_971.98: OK
```

Expand All @@ -41,14 +44,14 @@ First, find a function in the game that you want to match. There are a few ways

Once you have chosen a function, you can use the website [decomp.me](https://decomp.me/) to match it.
1. Go the website and click "Start decomping".
2. Under "Preset", select "Sly Cooper and the Thievius Raccoonus".
2. Click "PS2", then under "Preset", select "Sly Cooper and the Thievius Raccoonus".
3. Under "Diff label", enter the name of the function.
4. Find the `.s` file corresponding to the function you want to match in the `asm/nonmatchings` directory. Copy the contents of the file into the "Target assembly" box.
5. Copy any functions, global variables, or data types used by the function from the headers in the `include` directory into the "Context" box.
* E.g. For the function `void ResetClock(struct CLOCK* pclock, float t)`, you will need to include the definition of `CLOCK` struct, and the `TICK` data type since it is used in the CLOCK struct.
* You should almost always copy the entirety of "types.h" to ensure you have all the necessary primitive types.
6. Click "Create scratch".
7. (Optional) Go to the "Options" tab and under "Debug information" select `-g3 (Macro expansions)`. This will show you which line numbers the source code coreespond to each line in the assembly.
7. **(Recommended)** Go to the "Options" tab and under "Debug information" select `-g3 (Macro expansions)`. This will show you which line numbers the source code correspond to each line in the assembly.

Then start writing your code under the "Source code" tab. It will tell you what percent of the code matches the original. Tweak the code until it matches 100%. An example scratch can be found here: https://decomp.me/scratch/hmmyP

Expand All @@ -60,12 +63,12 @@ Once the function matches 100%, follow these steps to integrate it into the proj
1. Replace the `INCLUDE_ASM` macro in the `.c` file with the matched function.
* If the function is in a new file, **do not create a new file**. Creating new `.c` files is done by editing the `config/sly1.yaml` file to change the file split from `asm` to `c`, then running `python configure.py` to generate the new file. If you don't know how to do this, feel free to ask for help in the Discord server.
2. Check `config/symbol_addrs.txt` to see if the mangled name of the function is present.
* If it is not, add the **mangled name** of the function with it's address. The mangled name of the function can be found by searching the strings in the May 19 prototype elf, if it's the same as the release version. If you don't know how to do this, ask for help in the Discord server; Someone will be able to find it for you easily.
* The symbol_addrs.txt file is sorted alphabetically by filename, then by address within the file.
* If it is not, add the **mangled name** of the function with it's address. The mangled name of the function can be found in the debug symbols for the May 2002 prototype. If you don't know how to find it, ask for help in the Discord server; someone will be able to find it for you easily.
* The symbol_addrs.txt file is grouped alphabetically by filename, then sorted by address within the file.

The project should build and match. Here are some tips for troubleshooting common issues:
* Undefined reference errors usually mean the entry in the symbol_addrs.txt is wrong. Make sure the function name is mangled in symbol_addrs.txt and normal/un-mangled in the source code, and the mangled version matches the signature of the function. Also ensure that the address is correct in symbol_addrs.txt.
* Checksum failed means the compiled elf doesn't match the original. If it matches on decomp.me, it might be an issue with the compiler, so open an issue on GitHub or let someone know on Discord.
The project should build and match. Here are some common troubleshooting tips:
* `undefined reference error` usually means the entry in the symbol_addrs.txt is wrong. Make sure the function name is mangled in symbol_addrs.txt and unmangled in the source code, and the mangled version matches the signature of the function. Also ensure that the address is correct in symbol_addrs.txt.
* `checksum failed` means the compiled elf with your added code doesn't match the original. If it matches on decomp.me but not in the project, it might be an issue with the compiler, so open an issue on GitHub or let someone know on Discord.

<!--### CodeMatcher
Expand All @@ -74,16 +77,15 @@ You can use [CodeMatcher](https://github.com/felinis/CodeMatcher) to help match
If you are adding new code, it is strongly recommended that you run CodeMatcher before submitting a pull request. We will accept pull requests that don't match as long as the code is clean and readable, but in the future we may require that your code matches before merging it into the main branch.-->


## Code Review Process
## Make a pull request

Once you add your code to the project and it builds + matches, you can open a pull request to merge your fork into the main branch of the project. A code reviewer will need to approve it before it can be merged into the main branch.

We are a volunteer-driven project, so please be patient while we review your code. These are the main things we will look for in your code:

* It compiles without any errors.
* The compiled elf matches the original elf.

If everything looks good, we will merge your pull request as soon as possible. If anything needs to be fixed, we will let you know.
If everything looks good, we will merge your pull request as soon as possible. If anything needs to be addressd, we will let you know.


## Conclusion
Expand Down
11 changes: 8 additions & 3 deletions reference/readme.txt → reference/readme.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
This directory contains reference files you can use to help with decompiling game functions. They are all source code files we wrote by hand before we used Splat to decompile and reassemble the game to a byte-matching executable. Because of this, none of the functions in this directory match, but we tried to make them as close as possible to the original game functions.
# Reference files

The include directory contains header files that are used by the source files in the src dir. Many of the short data structures/enums are correct and can be copied directly into the actual decomp. The longer ones are probably not correct (missing fields, wrong field types, etc), but they can be used as a reference.
This directory contains reference files you can use to help with decompiling game functions. They are all source code files we wrote by hand before we used Splat to decompile and reassemble the game to a byte-matching executable.

The src/P2 directory contains the source code for the game engine. The following is a list of all known source code file names from a prototype build of the game. Some of them are not present in the final game.
It is organized as follows:
- `include/` contains header files for the game engine Many of the very short structs/enums are correct. The longer ones are probably not correct (missing fields, wrong field types, etc), but they can be used as a reference.
- `src/P2/` contains source code files for the game engine. None of the code is matching, though at the time we tried to make them resemble the original code as closely as possible, so they may help you understand the logic and control flow of certain functions.

## Source files
The following is a list of all known source code file names from a **prototype build** of the game. Some of them are not present in the final game, and the final game has some that are not present here.
ac.c
act.c
actseg.c
Expand Down
38 changes: 38 additions & 0 deletions scripts/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Scripts

This directory contains the following scripts used to setup and build the project.
- `build.sh`
- `check_progress.sh`
- `quickstart.sh`
- `setup_progd_linux.sh`

There are a few others, but they are not used in the normal contributing workflow.

## build.sh

Runs a clean reconfigure (deletes build files and splits the binary), then builds the project. Will warn you if `disc/SCUS_971.98` is not present.

## check_progress.sh

Checks the progress of the project by comparing the number of bytes in the original game binary to the number of bytes that have been matched. The output will be in this format:

```
Category : DecompedSize / Total OfFolder% ( OfTotal%)
all : 10516 / 1143680 0.9195% ( 0.9195% / 100.0000%)
sce : 0 / 116164 0.0000% ( 0.0000% / 10.1570%)
P2 : 10516 / 1005644 1.0457% ( 0.9195% / 87.9305%)
data : 0 / 21872 0.0000% ( 0.0000% / 1.9124%)
```

The value that we consider the progress of the project is row "`P2`", column "`OfFolder%`". This is the percentage of the game engine code that has been matched.

## quickstart.sh

Installs the necessary dependencies using pip/apt and sets up the build environment. You only need to run this script once.

## setup_progd_linux.sh

Installs the compiler needed to build the project on Linux. `quickstart.h` will run this script for you, so you don't need to run both.

There is a similar script for Windows, but the compiler

0 comments on commit 7ac7395

Please sign in to comment.