-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do you need a hand? #3
Comments
Hey. Thanks for the interest! I was planning on integrating this to https://github.com/cxcorp/cyberpunk2077-hacking-solver at some point but right now am working on an improved version that does all of the matrix detection and CV calls directly in C++ that's compiled to wasm. I'll put that on GitHub soon. I also got better results when detecting the matrix with DBSCAN than my previous ad-hoc solution. Some of the things I haven't worked on yet are for example the UI shown to the user, but I had some ideas like showing a confirmation prompt where the user can manually correct the digits tesseract missed. What did you have in mind? If you meant that you'd show the solution tracked on top of the matrix, a major challenge there is how slow and ultimately not completely trustworthy tesseract is, requiring manual corrections from the user. |
I envisioned using AR to show an arrow overlaying the original matrix. You could make the arrow move as codes get selected. But that's a dumb idea
since you have to use the controller with both hands.
Overall, I was thinking that perhaps you could create some issues that I could tackle so as not to do work in duplicity. Maybe let me know here when
you open source the new repo?
…On Sat, 21 Aug 2021, 09:28 Joona Heikkilä, ***@***.***> wrote:
Hey. Thanks for the interest!
I was planning on integrating this to
https://github.com/cxcorp/cyberpunk2077-hacking-solver at some point but
right now am working on an improved version that does all of the matrix
detection and CV calls directly in C++ that's compiled to wasm. I'll put
that on GitHub soon. I also got better results when detecting the matrix
with DBSCAN than my previous ad-hoc solution.
Some of the things I haven't worked on yet are for example the UI shown to
the user, but I had some ideas like showing a confirmation prompt where the
user can manually correct the digits tesseract missed.
What did you have in mind? If you meant that you'd show the solution
tracked on top of the matrix, a major challenge there is how slow and
ultimately not completely trustworthy tesseract is, requiring manual
corrections from the user.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#3 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB2UDXRUC5XCW7KHQUNWVK3T55PUDANCNFSM5CRTSQIQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
|
Yeah, that would definitely be cool and is in the realm of possibility! I also think that it might get a bit old after the first few times since you'd have to work on the controller with just one hand, and keep your phone pointed at the screen all the time 😄 My initial vision was to render the green rectangle on the code matrix and run Tesseract on it, then overlay the guess on top of the actual code matrix. However, Tesseract's performance isn't great, and OpenCV is also quite heavy on older phones (I get like 5 FPS on an older Nokia smartphone (5.1?), whereas I get 20-50 FPS on an OnePlus 8 Pro. So performance does limit creativity here a little bit. Right now the prototype shows my current vision a little bit, which is to have the (possibly low FPS) camera feed visible with the green rectangle showing the detected code matrix' edges, then wait for user to press a button, after which Tesseract is ran. I won't be able to work on this this week, but I'll let you know (probably the week after the next week). |
Sounds good. I think we might want to move towards pictures as opposed to live camera feeds. Also I've been thinking that maybe we don't need OCR but just feature detection. There are only 8 hex possible values so maybe that makes processing cheaper. Have you considered something like that? |
I did initially did train a Tensorflow model with a few hundred images, but
Tesseract just worked better (after training Tesseract's model with some
samples of the correct font). I can't remember how the performance was. The
Tesseract model could be improved by feeding it more images of hex bytes of
varying image quality and distortion.
I did not try OpenCV's feature matching, so that's definitely something
worth checking out! Maybe could run it on the individual cells (of the code
matrix) of the perspective corrected & thresholded image? In the worst case
that'd be something like 8*7*7=392 tests if the feature matcher can't match
for all 8 simultaneously. Would also need to check if feature matching is
compiled into the dist OpenCV.js, or if can even be compiled in.
…On Sun, Aug 22, 2021, 23:44 Gunar Gessner ***@***.***> wrote:
Sounds good. I think we might want to move towards pictures as opposed to
live camera feeds.
Also I've been thinking that maybe we don't need OCR but just feature
detection. There are only 8 hex possible values so maybe that makes
processing cheaper. Have you considered something like that?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFBCGFAJC4DTJ62NDF7ZAGTT6FORVANCNFSM5CRTSQIQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
|
Hi @cxcorp,
In a half-dream last night I've envisioned an "AR" code breaker for Cyberpunk. I was looking into tesseract.js when I've stumbled upon your work!
I was wondering whether we should work together on this (and incur the coordination costs) or if you'd rather work at your own pace (so maybe I'll start from scratch or fork).
Let me know what you think.
Thanks!
The text was updated successfully, but these errors were encountered: