Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(blooms): Initialize bloom gateway client only once #15994

Merged
merged 1 commit into from
Jan 29, 2025

Conversation

chaudum
Copy link
Contributor

@chaudum chaudum commented Jan 29, 2025

What this PR does / why we need it:

The bloom gateway client is used both in the bloom builder and in the index gateways. When running Loki in SSD mode, both services are part of the backend target, and therefore the client is initialised twice, leading to duplicate metrics registration and a subsequent panic.

This commit extracts the initialisation of the bloom gateway client into a separate service that is started once and becomes a dependency of both bloom builder and index gateway.

Which issue(s) this PR fixes:

Fixes: #14083

The bloom gateway client is used both in the bloom builder and in the
index gateways. When running Loki in SSD mode, both services are part of
the `backend` target, and therefore the client is initialised twice,
leading to duplicate metrics registration and a subsequent panic.

This commit extracts the initialisation of the bloom gateway client into
a separate service that is started once and becomes a dependency of both
bloom builder and index gateway.

Ref: #14083

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
@chaudum chaudum requested a review from a team as a code owner January 29, 2025 08:23
@chaudum chaudum changed the title Fix(blooms): Initialize bloom gateway client only once fix(blooms): Initialize bloom gateway client only once Jan 29, 2025
@chaudum chaudum merged commit e271e22 into main Jan 29, 2025
60 checks passed
@chaudum chaudum deleted the chaudum/issue-14083 branch January 29, 2025 08:53
@loki-gh-app
Copy link
Contributor

loki-gh-app bot commented Jan 29, 2025

The backport to release-3.3.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-15994-to-release-3.3.x origin/release-3.3.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x e271e22f7c1fae676e2fce6ab002b358694d9450

When the conflicts are resolved, stage and commit the changes:

git add . && git cherry-pick --continue

If you have the GitHub CLI installed:

# Push the branch to GitHub:
git push --set-upstream origin backport-15994-to-release-3.3.x
# Create the PR body template
PR_BODY=$(gh pr view 15994 --json body --template 'Backport e271e22f7c1fae676e2fce6ab002b358694d9450 from #15994{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title 'fix(blooms): Initialize bloom gateway client only once (backport release-3.3.x)' --body-file - --label 'size/M' --label 'type/bug' --label 'backport' --base release-3.3.x --milestone release-3.3.x --web

Or, if you don't have the GitHub CLI installed (we recommend you install it!):

# Push the branch to GitHub:
git push --set-upstream origin backport-15994-to-release-3.3.x

# Create a pull request where the `base` branch is `release-3.3.x` and the `compare`/`head` branch is `backport-15994-to-release-3.3.x`.

# Remove the local backport branch
git switch main
git branch -D backport-15994-to-release-3.3.x

@loki-gh-app
Copy link
Contributor

loki-gh-app bot commented Jan 29, 2025

The backport to release-3.2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-15994-to-release-3.2.x origin/release-3.2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x e271e22f7c1fae676e2fce6ab002b358694d9450

When the conflicts are resolved, stage and commit the changes:

git add . && git cherry-pick --continue

If you have the GitHub CLI installed:

# Push the branch to GitHub:
git push --set-upstream origin backport-15994-to-release-3.2.x
# Create the PR body template
PR_BODY=$(gh pr view 15994 --json body --template 'Backport e271e22f7c1fae676e2fce6ab002b358694d9450 from #15994{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title 'fix(blooms): Initialize bloom gateway client only once (backport release-3.2.x)' --body-file - --label 'size/M' --label 'type/bug' --label 'backport' --base release-3.2.x --milestone release-3.2.x --web

Or, if you don't have the GitHub CLI installed (we recommend you install it!):

# Push the branch to GitHub:
git push --set-upstream origin backport-15994-to-release-3.2.x

# Create a pull request where the `base` branch is `release-3.2.x` and the `compare`/`head` branch is `backport-15994-to-release-3.2.x`.

# Remove the local backport branch
git switch main
git branch -D backport-15994-to-release-3.2.x

salvacorts pushed a commit that referenced this pull request Feb 12, 2025
The bloom gateway client is used both in the bloom builder and in the index gateways. When running Loki in SSD mode, both services are part of the backend target, and therefore the client is initialised twice, leading to duplicate metrics registration and a subsequent panic.

This commit extracts the initialisation of the bloom gateway client into a separate service that is started once and becomes a dependency of both bloom builder and index gateway.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
loki-gh-app bot pushed a commit that referenced this pull request Feb 14, 2025
The bloom gateway client is used both in the bloom builder and in the index gateways. When running Loki in SSD mode, both services are part of the backend target, and therefore the client is initialised twice, leading to duplicate metrics registration and a subsequent panic.

This commit extracts the initialisation of the bloom gateway client into a separate service that is started once and becomes a dependency of both bloom builder and index gateway.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
(cherry picked from commit e271e22)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bloom-builder: "panic: duplicate metrics collector registration attempted"
3 participants