diff --git a/_posts/2024-07-18-Hacking-Meta-Rayban-Glasses.md b/_posts/2024-07-18-Hacking-Meta-Rayban-Glasses.md index 739b43e..8f8c975 100644 --- a/_posts/2024-07-18-Hacking-Meta-Rayban-Glasses.md +++ b/_posts/2024-07-18-Hacking-Meta-Rayban-Glasses.md @@ -21,7 +21,7 @@ Confession: I bought a pair of [Meta Rayban smart glasses](https://www.meta.com # Mini-review and FAQ - Q: How do you like them? Pros/cons. -- A: In summary: they're a camera and bluetooth headphones/microphone that you wear as glasses. Fit and finish is excellent. Battery life is minimum viable product level: you can drain them in a few hours by taking a bunch of photos or videos or bluetooth audio...maybe fine if you are an optional glasses wearer (and can thus take them off to charge), less good if you have to wear glasses all day. It's useful to be able to take hands-free photos (I can see lots of opportunities for making tutorials). AI assistant features are hit-or-miss: It's kind of neat to have "Hey meta, take a look and define this word" (use photo, interpret where finger is pointed, return a definition). But translation failed on a chinese newspaper headline. I am a very text-oriented person, so audio-based interactions are novel for me. +- A: In summary: they're a camera and bluetooth headphones/microphone that you wear as glasses. Fit and finish is excellent. Battery life is minimum viable product level: you can drain them in a few hours by taking a bunch of photos or videos or bluetooth audio...maybe fine if you are an optional glasses wearer (and can thus take them off to charge), less good if you have to wear glasses all day. It's useful to be able to take hands-free photos (I can see lots of opportunities for making tutorials). AI assistant features are hit-or-miss: It's kind of neat to have "Hey meta, take a look and define this word" (use photo, interpret where finger is pointed, return a definition). But translation failed on a chinese newspaper headline (as of 07/2024). I am a very text-oriented person, so audio-based interactions are novel for me. - A: One cool thing I did the other day while reading a book was ask the AI questions and examples related to terms in the book. It was a nice way not to break the flow of reading. @@ -37,4 +37,5 @@ Confession: I bought a pair of [Meta Rayban smart glasses](https://www.meta.com - *The AI asssistant feels like a Llama-3-8B class model, with a system prompt that focuses on brevity.* (as of late July 2024...maybe we'll start seeing Llama-3.1 outputs?) Brevity is certainly valuable for voice assistants, but sometimes you want more. Idea: More comprehensive research assistant: "OK meta, send message to resarch on whatsapp. Do a literature search on xxx." (intercepted by a flask app that runs an agent process, results in a written summary delivered via Whatsapp). By default, when the assistant receives a sufficiently long whatsapp message, it does not read it aloud, and that seems like the right behavior for something like this. - Games: Take a photo and tell me what move I should make in chess/backgammon/go. Apparently [the current generation of vision models is not good at even looking at schematic chess boards and deducing positions](https://www.linkedin.com/posts/aroraaman_given-an-input-image-of-a-chess-board-can-activity-7219166035889856513-tVS6). So maybe you would need to fine tuning a vision model to do this? Could "readily" be simulated (different angles of viewing the board, different colors, lighting, etc. by rendering). - + - **Update: 01/2025:** Some [kids recently did this](https://x.com/eddybuild/status/1878263416080482312) by: (i) Livestreaming video to instagram; (ii) having a remote computer do piece detection (presumably with YOLO or something like it) and feed inputs to stockfish; (iii) send response as an instagram live comment, which gets read as audio during the session to the user. This is essentially the same as the face-recogintion trick described above. +