Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Сoncurrent map writes while parsing table #7297

Closed
Ponywka opened this issue Dec 2, 2024 · 1 comment
Closed

Сoncurrent map writes while parsing table #7297

Ponywka opened this issue Dec 2, 2024 · 1 comment
Assignees
Labels

Comments

@Ponywka
Copy link
Contributor

Ponywka commented Dec 2, 2024

While running the code below, there is a chance of running into an error "fatal error: concurrent map writes"
It happens suddenly and need to use loop running until it fails

while go test -bench=. -benchtime=0.1s; do :; done
go 1.22.6

require (
	github.com/google/uuid v1.6.0
	gorm.io/driver/sqlite v1.5.6
	gorm.io/gorm v1.25.12
)

require (
	github.com/jinzhu/inflection v1.0.0 // indirect
	github.com/jinzhu/now v1.1.5 // indirect
	github.com/mattn/go-sqlite3 v1.14.22 // indirect
	golang.org/x/text v0.20.0 // indirect
)

package main

import (
	"github.com/google/uuid"
	"gorm.io/driver/sqlite"
	"gorm.io/gorm"
	"gorm.io/gorm/schema"
	"math/rand/v2"
	"sync"
	"testing"
)

type Data1 struct {
	UUID uuid.UUID `gorm:"primaryKey;type:uuid"`
}

type Data2 struct {
	UUID uuid.UUID `gorm:"primaryKey;type:uuid"`
}

type Data3 struct {
	UUID uuid.UUID `gorm:"primaryKey;type:uuid"`
}

func BenchmarkParse(b *testing.B) {
	cacheStore := &sync.Map{}
	db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
	if err != nil {
		b.Fatalf("failed to connect database")
	}

	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			switch rand.IntN(2) {
			case 0:
				_, _ = schema.ParseWithSpecialTableName([]*struct {
					Data1
					Data2 Data2 `gorm:"foreignKey:UUID;references:UUID"`
				}{}, cacheStore, db.NamingStrategy, "")
			case 1:
				_, _ = schema.ParseWithSpecialTableName([]*struct {
					Data1
					Data2 Data2 `gorm:"foreignKey:UUID;references:UUID"`
					Data3 Data3 `gorm:"foreignKey:UUID;references:UUID"`
				}{}, cacheStore, db.NamingStrategy, "")
			}
		}
	})
}
fatal error: concurrent map writes

goroutine 87 [running]:
gorm.io/gorm/schema.(*Schema).parseRelation(0xc00041e1e0, 0xc000420600)
        /home/ponywka/go/pkg/mod/gorm.io/gorm@v1.25.12/schema/relationship.go:103 +0x5d8
gorm.io/gorm/schema.ParseWithSpecialTableName({0x71bc80, 0xc000236360}, 0xc00007e000, {0x7cc6b8, 0xc0001e4000}, {0x0, 0x0})
        /home/ponywka/go/pkg/mod/gorm.io/gorm@v1.25.12/schema/schema.go:342 +0x22d8
storage.BenchmarkFind.func1(0xc00007f640)
        /home/ponywka/Projects/test3/main_test.go:38 +0x125
testing.(*B).RunParallel.func1()
        /home/ponywka/sdk/go1.22.6/src/testing/benchmark.go:797 +0xbf
created by testing.(*B).RunParallel in goroutine 11
        /home/ponywka/sdk/go1.22.6/src/testing/benchmark.go:790 +0x116

It happens for me in Find operation:

fatal error: concurrent map writes

goroutine 36 [running]:
gorm.io/gorm/schema.(*Schema).parseRelation(0xc0001f6000, 0xc0001fc800)
	/go/pkg/mod/gorm.io/gorm@v1.25.12/schema/relationship.go:103 +0x5d8
gorm.io/gorm/schema.ParseWithSpecialTableName({0x10cc9c0, 0xc0001720c0}, 0xc000893360, {0x2364c78, 0xc000895300}, {0x0, 0x0})
	/go/pkg/mod/gorm.io/gorm@v1.25.12/schema/schema.go:342 +0x22d8
gorm.io/gorm.(*Statement).ParseWithSpecialTableName(0xc000702fc0, {0x10cc9c0?, 0xc0001720c0?}, {0x0?, 0xc00055b278?})
	/go/pkg/mod/gorm.io/gorm@v1.25.12/statement.go:493 +0x65
gorm.io/gorm.(*Statement).Parse(...)
	/go/pkg/mod/gorm.io/gorm@v1.25.12/statement.go:489
gorm.io/gorm/callbacks.BuildQuerySQL(0xc00016f9e0)
	/go/pkg/mod/gorm.io/gorm@v1.25.12/callbacks/query.go:89 +0x31d
gorm.io/gorm/callbacks.Query(0xc00016f9e0)
	/go/pkg/mod/gorm.io/gorm@v1.25.12/callbacks/query.go:16 +0x36
gorm.io/gorm.(*processor).Execute(0xc00089ad70, 0xc000137e30?)
	/go/pkg/mod/gorm.io/gorm@v1.25.12/callbacks.go:130 +0x3cc
gorm.io/gorm.(*DB).Find(0xc00016f9e0?, {0x10cc9c0, 0xc0001720c0}, {0x0, 0x0, 0x0})
	/go/pkg/mod/gorm.io/gorm@v1.25.12/finisher_api.go:170 +0x134
XXX/XXX.XXX({0x235e398, 0x2d6f260})
	/src/XXX.go:XXX +0x4b4

It happens because this is where the cache is used

Pic 1 Square 1: Initializing "relation" field
Pic 1 Square 2: Initializing "relation.FieldSchema" field using "getOrParse" function
Pic 2 Square 1: Loading parsed value from cache
Pic 1 Square 3: Mutate "relation.FieldSchema" field

image
image

Copy link

github-actions bot commented Dec 2, 2024

The issue has been automatically marked as stale as it missing playground pull request link, which is important to help others understand your issue effectively and make sure the issue hasn't been fixed on latest master, checkout https://github.com/go-gorm/playground for details. it will be closed in 30 days if no further activity occurs. if you are asking question, please use the Question template, most likely your question already answered https://github.com/go-gorm/gorm/issues or described in the document https://gorm.ioSearch Before Asking

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jan 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants