Skip to content
This repository has been archived by the owner on Jan 13, 2024. It is now read-only.

Commit

Permalink
Minor improvements.
Browse files Browse the repository at this point in the history
  • Loading branch information
finnbear committed Apr 7, 2021
1 parent d5546d9 commit 0f6b14d
Show file tree
Hide file tree
Showing 6 changed files with 11,171 additions and 13,211 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ Accuracy was evaluated based on the first 100,000 items from this [dataset of mo

|**Package**|**Time**|**Accuracy**|**Comment**|
|:-----:|:-----:|:-----:|:-----:|
|[finnbear/moderation](https://github.com/finnbear/moderation)|1.50s|90.80%|Current API version is not stable|
|[finnbear/moderation](https://github.com/finnbear/moderation)|1.50s|90.85%|Current API version is not stable|
|[TwinProduction/go-away](https://github.com/TwinProduction/go-away)|2.11s|82.11%|Many false positives from combined words like "push it"|


Expand Down
1 change: 1 addition & 0 deletions generator/generate.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions generator/profanity.csv
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ handjob,0,0,3,0
hate,0,0,0,2
hell,1,0,0,0
heshe,0,0,2,0
hitler,0,2,0,0
hoar,0,3,2,0
hoe,0,2,1,0
homo,0,2,1,0
Expand Down Expand Up @@ -149,6 +150,7 @@ sodomy,0,0,3,0
spunk,0,0,3,0
stfu,0,0,0,2
stupid,0,0,0,1
suck,0,0,0,1
suckit,0,0,2,2
suckmy,0,0,3,0
testical,0,0,3,0
Expand Down
11 changes: 6 additions & 5 deletions moderation.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,9 @@ func IsInappropriate(text string) bool {
return Scan(text).Is(Inappropriate)
}

// Scan returns a bitmask of all types detected within given text
func Scan(text string) (types Type) {
// Scan returns a bitmask of all types detected within given text, which can
// be queried with the Is function
func Scan(text string) (scanResult Type) {
// Figure out if sanitization is needed, and if so, do it
for _, textRune := range text {
if textRune < minNormal || maxNormal < textRune {
Expand Down Expand Up @@ -213,7 +214,7 @@ func Scan(text string) (types Type) {
severity = 0b001 // mild
}

types |= severity << (i * 3)
scanResult |= severity << (i * 3)
}

// Min length is arbitrary, but must be > 0 to avoid dividing by zero
Expand All @@ -223,9 +224,9 @@ func Scan(text string) (types Type) {
// TODO: Define severe spam

if spamPercent > 50 {
types |= 0b010 << (4 * 3) // moderate spam
scanResult |= 0b010 << (4 * 3) // moderate spam
} else if spamPercent > 30 {
types |= 0b001 << (4 * 3) // mild spam
scanResult |= 0b001 << (4 * 3) // mild spam
}
}

Expand Down
Loading

0 comments on commit 0f6b14d

Please sign in to comment.