Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update metrics fetcher to use a Typescript-based pipeline #90

Merged
merged 25 commits into from
Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
770d4d4
Add first pass at TS Backend
ipc103 Jan 26, 2024
c07560d
Add pagination via plugin instead of manually
ipc103 Jan 29, 2024
3962e63
Add ability to generate JSON file
ipc103 Jan 29, 2024
43f3b16
feat: break out functions into fetchers and add a pipeline
ajhenry Jan 29, 2024
674a1d0
feat: add logging
ajhenry Jan 29, 2024
478210d
feat: add retries to octokit
ajhenry Jan 29, 2024
ddde80a
feat: filter out forks and archived
ajhenry Jan 29, 2024
004a7f6
feat: add type defs for the graphql api
ajhenry Jan 29, 2024
85a4cf3
feat: add discussions count
ajhenry Jan 29, 2024
7591df7
feat: add flags for features enabled
ajhenry Jan 29, 2024
4f0a21a
feat: add open issues age metric
ajhenry Jan 29, 2024
0099733
fix: add paginate to issues fetching
ajhenry Jan 29, 2024
358113b
feat: update ci to fetch data with node backend
ajhenry Jan 29, 2024
92b926c
feat: add request throttling to octokit
ajhenry Jan 29, 2024
3f3fad3
feat: add since date to limit excessive querying
ajhenry Jan 29, 2024
524e6b4
feat: add debugging for ts-backend running
ajhenry Jan 30, 2024
bac1f43
feat: add rate limit checker
ajhenry Jan 30, 2024
893e1f6
feat: add issue response time metrics and rate limit info
ajhenry Jan 30, 2024
64f3968
fix: issue with using REST API on some fields
ajhenry Jan 31, 2024
84b1193
feat: add config file
ajhenry Jan 31, 2024
0b2b0f4
fix: remove response time metric for now
ajhenry Feb 1, 2024
37fb77f
fix: make dropdown work with boolean and update fieldnames to reflect…
ajhenry Feb 1, 2024
e81f585
Update instructions in README
ipc103 Feb 1, 2024
d282c77
Change output location for data.json file
ipc103 Feb 1, 2024
79f35ed
Use license name instead of spdxId
ipc103 Feb 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 6 additions & 19 deletions .github/workflows/nextjs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,26 +34,13 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Go 1.x
uses: actions/setup-go@v4
- name: Fetch Metrics
uses: actions/setup-node@v4
with:
go-version: ^1.19
- name: Get dependencies
run: |
cd backend
go get -v -t -d ./...
if [ -f Gopkg.toml ]; then
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
dep ensure
fi
- name: Build Go
run: make build
# - name: Test
# run: make test
- name: Run Go program and save output
id: run
node-version: "20.x"
run: |
$PWD/backend/bin/metrics
npm i
npm start
- name: Detect package manager
id: detect-package-manager
run: |
Expand All @@ -76,7 +63,7 @@ jobs:
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: "16"
node-version: "20.x"
cache: ${{ steps.detect-package-manager.outputs.manager }}
cache-dependency-path: ${{ steps.detect-package-manager.outputs.cache-dependency-path }}
- name: Setup Pages
Expand Down
38 changes: 23 additions & 15 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -1,16 +1,24 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Launch metrics (local)",
"type": "go",
"request": "launch",
"mode": "debug",
"program": "${workspaceFolder}/backend/cmd/main.go",
"envFile": "${workspaceFolder}/dev.vscode.env",
},
]
}
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Launch metrics (local)",
"type": "go",
"request": "launch",
"mode": "debug",
"program": "${workspaceFolder}/backend/cmd/main.go",
"envFile": "${workspaceFolder}/dev.vscode.env"
},
{
"name": "Next.js: debug metrics",
"type": "node-terminal",
"request": "launch",
"cwd": "${workspaceFolder}/ts-backend",
"command": "npm run start",
"envFile": "${workspaceFolder}/ts-backend/.env"
}
]
}
15 changes: 12 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,9 @@ Issue Project [here](https://github.com/github/SI-skills-based-volunteering/iss
## Development in Codespaces

<!-- TODO: Add min requirements and deployment steps -->
### Backend
### Go Backend

To update the repository data.
To update the repository data using the Go backend

1. Generate a [new GitHub Token](https://github.com/settings/tokens) with the ability to read repo, read org, and read projects scopes.
1. Set the `GRAPHQL_TOKEN` environment variable to be the value of your newly created token.
Expand All @@ -75,7 +75,16 @@ make build
./backend/bin/metrics
```

This will generate a new `data.json` file in the UI directory which can be imported directly as part of the static build.
This will generate a new `data.json` file in the UI directory which can be imported directly as part of the static build.\

### TypeScript Backend

To update the repository data using the TypeScript backend.

1. Generate a [new GitHub Token](https://github.com/settings/tokens) with the ability to read repo, read org, and read projects scopes.
1. Set the `GRAPHQL_TOKEN` environment variable to be the value of your newly created token.
1. `cd ts-backend && npm install && yarn start`. This will generate a `data.json` file in the `who-metrics-ui` directory.
1. `cd ../who-metrics-ui` and run `npm install && npm run dev` to start the server.

### UI

Expand Down
2 changes: 2 additions & 0 deletions config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
organization: "WorldHealthOrganization"
1 change: 1 addition & 0 deletions ts-backend/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
GRAPHQL_TOKEN=your_token
12 changes: 12 additions & 0 deletions ts-backend/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
.cache
.DS_Store
.idea
*.log
*.tgz
coverage
dist
lib-cov
logs
node_modules
temp
*data.json
12 changes: 12 additions & 0 deletions ts-backend/build.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { defineBuildConfig } from 'unbuild'

export default defineBuildConfig({
entries: [
'src/index',
],
declaration: true,
clean: true,
rollup: {
emitCJS: true,
},
})
Loading
Loading