Skip to content

Commit

Permalink
Merge pull request #54 from cytechmobile/feature/pkAuth
Browse files Browse the repository at this point in the history
feat(backend): Replace Ssh Agent with Keys Handling
  • Loading branch information
tdakanalis authored Oct 24, 2023
2 parents bd060dd + 07bbf17 commit ea721c7
Showing 16 changed files with 153 additions and 261 deletions.
13 changes: 2 additions & 11 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -23,18 +23,9 @@ jobs:
java-version: '17'
distribution: 'corretto'

# Start SSH Agent and add the private key
- name: Start SSH Agent and add the private key
env:
SSH_PASSPHRASE: ${{secrets.SSH_PASSPHRASE}}
SSH_PRIVATE_KEY: ${{secrets.SSH_PRIVATE_KEY}}
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
# Setup any environment variable in this step
- name: Setup environment
run: |
ssh-agent -a $SSH_AUTH_SOCK > /dev/null
echo 'echo $SSH_PASSPHRASE' > ~/.ssh_askpass && chmod +x ~/.ssh_askpass
echo "$SSH_PRIVATE_KEY" | tr -d '\r' | DISPLAY=None SSH_ASKPASS=~/.ssh_askpass ssh-add - >/dev/null
echo "SSH_AUTH_SOCK=$SSH_AUTH_SOCK" >> $GITHUB_ENV
echo "MVN_OPTS=-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true --batch-mode --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true" >> $GITHUB_ENV
# Run tests and build the application
- name: build
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -3,6 +3,7 @@
## 0.4.0 - 2023-10-19
### Features
* Added support for migrating the inline assets/files discovered within GitHub issues and comments as Radicle embeds.
* Removed dependency on the ssh agent by using the private key to sign the session.

## 0.3.1 - 2023-08-23
### Features
49 changes: 23 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -54,7 +54,7 @@ This tool is available under [Apache License, Version 2.0](https://www.apache.or
The tool offers several important features, including:
* It enables the migration of all GitHub issues from the source repository in a single run.
* It migrates essential information such as the `Title`, `Description`, `Status`, `Labels`, `Comments`, `Events`, and `Milestone` details.
* It supports the migration of inline assets/files discovered within GitHub issues and comments as Radicle embeds. A GitHub `user_session` cookie must be provided when migrating assets/files from a private GitHub repository by using either the `--github-session` CLI parameter or the `GITHUB_SESSION` environment variable. Login to GitHub via your browser and copy the value of the `user_session` cookie.
* It supports the migration of inline assets/files discovered within GitHub issues and comments as Radicle embeds. A GitHub `user_session` cookie must be provided when migrating assets/files from a private GitHub repository by using either the `--gh-session` CLI parameter or the `GH_SESSION` environment variable. Login to GitHub via your browser and copy the value of the `user_session` cookie.
* Any additional information that doesn't fit within the Issue model is preserved in a dedicated `GitHub Metadata` section, along with references to the original repository
* It supports incremental migration, allowing you to rerun the tool (e.g., on a schedule) and create only the newest issues that haven't been previously migrated.
* It offers a range of filtering options to streamline the issue migration process, including issues created after a specified time, issues with specific labels, issues in a particular state, issues belonging to a given milestone number, issues created by a specific user, and issues assigned to a particular user.
@@ -64,19 +64,20 @@ The tool offers several important features, including:

### Command-line interface
```bash
Usage: radicle-github-migrate issues [-gv=<gVersion>] [-gu=<gUrl>] -gr=<gRepo> -go=<gOwner> -gt [-rv=<rVersion>] [-ru=<rUrl>] -rp=<rProject> [-fs=<fSince>] [-fl=<fLabels>] [-ft=<fState>] [-fm=<fMilestone>] [-fa=<fAssignee>] [-fc=<fCreator>]
Usage: radicle-github-migrate issues [-gv=<gVersion>] [-gu=<gUrl>] -gr=<gRepo> -go=<gOwner> -gt [-gs=<gSession>] [-rv=<rVersion>] [-ru=<rUrl>] -rp=<rProject> -rh [-fs=<fSince>] [-fl=<fLabels>] [-ft=<fState>] [-fm=<fMilestone>] [-fa=<fAssignee>] [-fc=<fCreator>] [-dr]

Migrate issues from a GitHub repository to a Radicle project.

-gv, --github-api-version=<gVersion> The version of the GitHub REST API (default: 2022-11-28).
-gu, --github-api-url=<gUrl> The base url of the GitHub REST API (default: https://api.github.com).
-gr, --github-repo=<gRepo> The source GitHub repo.
-go, --github-repo-owner=<gOwner> The owner of the source GitHub repo.
-gt, --github-token Your GitHub personal access token (with repo scope or read-only access granted).
-gs, --github-session The value of the user_session cookie. It is utilized for migrating assets and files from a private GitHub repository.
-rv, --radicle-api-version=<rVersion> The version of the Radicle HTTP API (default: v1).
-ru, --radicle-api-url=<rUrl> The base url of Radicle HTTP API (default: http://localhost:8080/api).
-rp, --radicle-project=<rProject> The target Radicle project.
-gv, --gh-api-version=<gVersion> The version of the GitHub REST API (default: 2022-11-28).
-gu, --gh-api-url=<gUrl> The base url of the GitHub REST API (default: https://api.github.com).
-gr, --gh-repo=<gRepo> The source GitHub repo.
-go, --gh-repo-owner=<gOwner> The owner of the source GitHub repo.
-gt, --gh-token Your GitHub personal access token (with repo scope or read-only access granted).
-gs, --gh-session The value of the user_session cookie. It is utilized for migrating assets and files from a private GitHub repository.
-rv, --rad-api-version=<rVersion> The version of the Radicle HTTP API (default: v1).
-ru, --rad-api-url=<rUrl> The base url of Radicle HTTP API (default: http://localhost:8080/api).
-rp, --rad-project=<rProject> The target Radicle project.
-rh, --rad-passphrase=<rPassphrase> Your radicle passphrase.
-fs, --filter-since=<fSince> Migrate issues created after the given time (default: lastRun in store.properties file, example: 2023-01-01T10:15:30+01:00).
-fl, --filter-labels=<fLabels> Migrate issues with the given labels given in a csv format (example: bug,ui,@high).
-ft, --filter-state=<fState> Migrate issues in this state (default: all, can be one of: open, closed, all).
@@ -104,15 +105,16 @@ Since the Issue model and HTTP API in Radicle are currently simpler compared to

### Environment Variables
You can pass any of the command line options via environment variables. Here is the complete list of the supported environment variables:
* GITHUB_API_VERSION: The version of the GitHub REST API (default 2022-11-28)
* GITHUB_API_URL: The base url of the GitHub REST API (default https://api.github.com)
* GITHUB_REPO: The source GitHub repo
* GITHUB_OWNER: The owner of the source GitHub repo
* GITHUB_TOKEN: Your GitHub personal access token (with `repo` scope or `read-only access` granted).
* GITHUB_SESSION: The value of the user_session cookie. It is utilized for migrating assets and files from a private GitHub repository.
* RADICLE_API_VERSION: The version of the Radicle HTTP API (default v1)
* RADICLE_API_URL: The base url of Radicle HTTP API (default http://localhost:8080/api)
* RADICLE_PROJECT: The target Radicle project
* GH_API_VERSION: The version of the GitHub REST API (default 2022-11-28)
* GH_API_URL: The base url of the GitHub REST API (default https://api.github.com)
* GH_REPO: The source GitHub repo
* GH_OWNER: The owner of the source GitHub repo
* GH_TOKEN: Your GitHub personal access token (with `repo` scope or `read-only access` granted).
* GH_SESSION: The value of the user_session cookie. It is utilized for migrating assets and files from a private GitHub repository.
* RAD_API_VERSION: The version of the Radicle HTTP API (default v1)
* RAD_API_URL: The base url of Radicle HTTP API (default http://localhost:8080/api)
* RAD_PROJECT: The target Radicle project
* RAD_PASSPHRASE: Your radicle passphrase
* FILTER_SINCE: Migrate issues created after the given time (default: lastRun in store.properties file, example: 2023-01-01T10:15:30+01:00).
* FILTER_LABELS: Migrate issues with the given labels given in a csv format (example: bug,ui,@high).
* FILTER_STATE: Migrate issues in this state (default: all, can be one of: open, closed, all).
@@ -152,17 +154,12 @@ docker pull ghcr.io/cytechmobile/radicle-github-migrate:latest
# Tag the docker image in your local docker registry
docker tag ghcr.io/cytechmobile/radicle-github-migrate:latest radicle-github-migrate

# Run the migration ...
# ... either by mounting the SSH_AUTH_SOCK (Option 1)
docker run -it -v .:/root/config -v ~/.radicle:/root/.radicle -v $SSH_AUTH_SOCK:/ssh-agent radicle-github-migrate issues

# ... or by passing the RAD_PASSPHRASE environment variable (Option 2)
# Run the migration
docker run -it -v .:/root/config -v ~/.radicle:/root/.radicle -e RAD_PASSPHRASE=<YOUR_PASSPHRASE> radicle-github-migrate issues
```
To ensure that the `docker run` command executes successfully, the following volumes are required:
* `.:/root/config`: This allows the tool to write a `store.properties` file in your current directory, which helps maintain its state across subsequent runs. IMPORTANT: Please ensure that the folder from which you run the tool has the appropriate write permissions.
* `~/.radicle:/root/.radicle`: This enables the tool to access your Radicle path. If the `rad path` command returns a different path, please update the volume accordingly.
* `$SSH_AUTH_SOCK:/ssh-agent`: This allows the application to access your SSH agent for session authorization. Alternatively, the RAD_PASSPHRASE environment variable can be set.

The image assumes that your `radicle-httpd` service runs by default at `http://172.17.0.1:8080/api`, where `172.17.0.1` represents the IP address of the host from inside the Docker container. If you need to change this default configuration, you can utilize the available environment variables or CLI options provided.

11 changes: 8 additions & 3 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -78,8 +78,12 @@
</dependency>
<dependency>
<groupId>com.sshtools</groupId>
<artifactId>maverick-sshagent</artifactId>
<version>3.0.11</version>
<artifactId>maverick-synergy-client</artifactId>
<version>3.0.10</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk18on</artifactId>
</dependency>
<dependency>
<groupId>com.vladsch.flexmark</groupId>
@@ -201,7 +205,8 @@
</executions>
<configuration>
<environmentVariables>
<GITHUB_TOKEN>testGitHubToken</GITHUB_TOKEN>
<GH_TOKEN>test</GH_TOKEN>
<RAD_PASSPHRASE>test</RAD_PASSPHRASE>
</environmentVariables>
</configuration>
</plugin>
3 changes: 1 addition & 2 deletions src/main/docker/Dockerfile.jvm
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
FROM amazoncorretto:17-alpine
ENV STORAGE_FILE_PATH="/root/config/store.properties"
ENV SSH_AUTH_SOCK="/ssh-agent"
ENV RADICLE_API_URL="http://172.17.0.1:8080/api"
ENV RAD_API_URL="http://172.17.0.1:8080/api"

RUN apk --no-cache add wget curl
RUN wget https://radicle.xyz/install && chmod +x install && ./install && cp ~/.radicle/bin/rad /bin/
2 changes: 1 addition & 1 deletion src/main/java/network/radicle/tools/github/Config.java
Original file line number Diff line number Diff line change
@@ -53,7 +53,7 @@ public String toString() {
'}';
}
}
public record RadicleConfig(URL url, String version, String project, Boolean dryRun) { }
public record RadicleConfig(URL url, String version, String project, String passphrase, Boolean dryRun) { }

public record Filters(Instant since, String labels, State state, Integer milestone, String assignee,
String creator) {
Original file line number Diff line number Diff line change
@@ -13,8 +13,8 @@
import network.radicle.tools.github.core.radicle.Session;
import network.radicle.tools.github.core.radicle.actions.Action;
import network.radicle.tools.github.handlers.ResponseHandler;
import network.radicle.tools.github.services.AuthService;
import network.radicle.tools.github.services.CliService;
import network.radicle.tools.github.services.SshAgentService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@@ -27,7 +27,7 @@ public class RadicleClient implements IRadicleClient {
@Inject Client client;
@Inject ObjectMapper mapper;
@Inject Config config;
@Inject SshAgentService agent;
@Inject AuthService authService;
@Inject CliService cli;

@Override
@@ -40,7 +40,7 @@ public Session createSession() throws Exception {
session = mapper.readValue(json, Session.class);
}

session.signature = agent.sign(session);
session.signature = authService.sign(session);

// in case the signing session failed (e.g. due to ssh agent not being accessible)
// fallback to creating session via the cli
46 changes: 27 additions & 19 deletions src/main/java/network/radicle/tools/github/commands/Command.java
Original file line number Diff line number Diff line change
@@ -19,64 +19,72 @@ public class Command implements Runnable {
public static final int PAGE_SIZE = 100;

@CommandLine.Option(
names = {"-gv", "--github-api-version"},
defaultValue = "${GITHUB_API_VERSION:-2022-11-28}",
names = {"-gv", "--gh-api-version"},
defaultValue = "${GH_API_VERSION:-2022-11-28}",
description = "The version of the GitHub REST API (default: 2022-11-28).")
String gVersion;

@CommandLine.Option(
names = {"-gu", "--github-api-url"},
defaultValue = "${GITHUB_API_URL:-https://api.github.com}",
names = {"-gu", "--gh-api-url"},
defaultValue = "${GH_API_URL:-https://api.github.com}",
description = "The base url of the GitHub REST API (default: https://api.github.com).")
URL gUrl;

@CommandLine.Option(
names = {"-gr", "--github-repo"},
names = {"-gr", "--gh-repo"},
required = true,
defaultValue = "${GITHUB_REPO}",
defaultValue = "${GH_REPO}",
description = "The source GitHub repo.")
String gRepo;

@CommandLine.Option(
names = {"-go", "--github-repo-owner"},
names = {"-go", "--gh-repo-owner"},
required = true,
defaultValue = "${GITHUB_OWNER}",
defaultValue = "${GH_OWNER}",
description = "The owner of the source GitHub repo.")
String gOwner;

@CommandLine.Option(
names = {"-gt", "--github-token"},
names = {"-gt", "--gh-token"},
required = true,
interactive = true,
defaultValue = "${GITHUB_TOKEN}",
defaultValue = "${GH_TOKEN}",
description = "Your GitHub personal access token (with `repo` scope or `read-only access` granted).")
String gToken;

@CommandLine.Option(
names = {"-gs", "--github-session"},
defaultValue = "${GITHUB_SESSION}",
names = {"-gs", "--gh-session"},
defaultValue = "${GH_SESSION}",
description = "The value of the user_session cookie. It is utilized for migrating assets and files from a private GitHub repository.")
String gSession;

@CommandLine.Option(
names = {"-rv", "--radicle-api-version"},
defaultValue = "${RADICLE_API_VERSION:-v1}",
names = {"-rv", "--rad-api-version"},
defaultValue = "${RAD_API_VERSION:-v1}",
description = "The version of the Radicle HTTP API (default: v1).")
String rVersion;

@CommandLine.Option(
names = {"-ru", "--radicle-api-url"},
defaultValue = "${RADICLE_API_URL:-http://localhost:8080/api}",
names = {"-ru", "--rad-api-url"},
defaultValue = "${RAD_API_URL:-http://localhost:8080/api}",
description = "The base url of Radicle HTTP API (default: http://localhost:8080/api).")
URL rUrl;

@CommandLine.Option(
names = {"-rp", "--radicle-project"},
names = {"-rp", "--rad-project"},
required = true,
defaultValue = "${RADICLE_PROJECT}",
defaultValue = "${RAD_PROJECT}",
description = "The target Radicle project.")
String rProject;

@CommandLine.Option(
names = {"-rh", "--rad-passphrase"},
required = true,
interactive = true,
defaultValue = "${RAD_PASSPHRASE}",
description = "Your radicle passphrase.")
String rPassphrase;

@CommandLine.Option(
converter = InstantConverter.class,
names = {"-fs", "--filter-since"},
@@ -133,7 +141,7 @@ public void run() {
private void loadConfiguration() {
var filters = new Filters(fSince, fLabels, fState, fMilestone, fAssignee, fCreator);
config.setGithub(new GitHubConfig(gSession, gToken, gUrl, gVersion, gOwner, gRepo, filters, PAGE_SIZE));
config.setRadicle(new RadicleConfig(rUrl, rVersion, rProject, dryRun));
config.setRadicle(new RadicleConfig(rUrl, rVersion, rProject, rPassphrase, dryRun));
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
package network.radicle.tools.github.services;

import com.sshtools.common.publickey.SshKeyUtils;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import network.radicle.tools.github.Config;
import network.radicle.tools.github.core.radicle.Session;
import network.radicle.tools.github.utils.Multibase;
import org.eclipse.microprofile.config.inject.ConfigProperty;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.PrintStream;
import java.nio.charset.StandardCharsets;
import java.util.Optional;

@ApplicationScoped
public class AuthService {
private static final Logger logger = LoggerFactory.getLogger(AuthService.class);

@ConfigProperty(name = "rad.home")
Optional<String> radHome;

@Inject Config config;

public String sign(Session session) {
var originalError = System.err;
try {
System.setErr(new PrintStream(new ByteArrayOutputStream()));
var kp = SshKeyUtils.getPrivateKey(new File(radHome.orElse("~/.radicle") + "/keys/radicle"),
config.getRadicle().passphrase());
var dataToSign = session.id + ":" + session.publicKey;
var signedData = kp.getPrivateKey().sign(dataToSign.getBytes(StandardCharsets.UTF_8));

return Multibase.encode(Multibase.Base.Base58BTC, signedData);
} catch (Exception e) {
logger.debug("Failed to sign the session. Error: {}", e.getMessage());
return null;
} finally {
System.setErr(originalError);
}
}
}
Loading

0 comments on commit ea721c7

Please sign in to comment.