Feature request: add accessibility for blind players

[germanico5]

hello, pysol dev team.

A big issue that I've found. I'm a blind person that relies on screen reader technology to use computing devices like PC, and well also for playing games.
I downloaded pysol to test it, but found that is incompatible on its majority with assistive technologies. But, given the structure of the game and after talking with you on the forums, maybe it will be easy implement screen reader support on the game.

Some things to consider that I've found

1. The current implementation of the game is built using Tkinter, which lacks built-in accessibility features as a framework (other frameworks like WXpython qt and others have a direct compatibility layer with accessibility APIs of OSs). This presents a challenge to implement the feature as easy, but isn't impossible.
   To address this, we propose implementing manual accessibility and navigation functionality for the game interface.
2. Accessibility frameworks like the experimental accessKit Can be interesting to use on more large projects like tkinter itself, but for this case is better address the issue through the manual way.

So, things that you could do:

1. Integrate on the suite a library to communicate directly with the screen reader software. For python you can use tolk or accessibleOutput with these library, when the accessibility option (or as a default if you want) is turned on, you can send messages directly to the screen readers software that is active (also you can detect if one is active). You can use this to announce messages to the player, describe the selected card or position on the gamefield.
2. Integrate on the game a keyboard navigation. I think that the most important part is the card field. So, first try to add basic movement with the arrows to move through the cards, and announce to the screen reader the selected card.
3. If tin general the games relies on a drag and drop playstyle, add to the keyboard navigation a method to select a card (with enter or spacebar as example) and a final location, to move the card. obviously just if the play is legal for the active game.
4. Send to the screen reader a message when a automatic event happened on the active game. Like if a card is drawn from a pile, you must move a card to a location to continue the game, if you try to do a wrong movement, etc.

With these things I think that we can go through a basic first step.
next is time to clean things like the menus, select a specific game, read scores, stopwatches if they exists, etc.

For now, surprisingly the menu bar is almost readable and navigable with NVDA. One of the screen readers.

Please take in consideration this feature request; these kind of games are a very interesting point to be accessible, and also this suite has more than 1000 games, so a lot of blind people be happy if they can play this game suite without problems.

Also, here are other useful links:

NVDA screen reader, for testing purposes

Jaws for windows, other paid popular screen reader

Thread in forum talking about accessibility

[sanslash332]

Hello guys, sandl here.

As Germán, I'm a member of the staff of the tiflojuegos website, a community dedicated to the blind players.

I helped germán to write the issue, so Im here to try help you if you need some guide or tips to implements these accessibility feature.

Really guys, thanks for take interest on this thing.

[joeraz]

Not a problem, glad to help.

So, we can add one of the screen reader libraries to the project as an optional library - I'll have to take a closer look at their documentation to determine which library is the best fit (easiest to implement, and best supported). We can add an option somewhere to the menu (or the config file) to enable the screen reader, detecting if the library is present and a screen reader is installed. I'd just have to find the best place to put it - the options menu is getting a bit full.

So when playing with the mouse, drag and drop is the default, but there is also an option to switch to point and click style playing (click a card/sequence, then click the stack to move to). I think building off the point and click logic would be a good place to start for keyboard movement, though I haven't done much with this logic, so it needs some review. Currently, the arrow keys, tab, spacebar, and enter are not mapped to anything else, so there won't be any conflicts there. The biggest challenge on this part will likely be figuring out how to find the next stack to move to. I can set up tab to cycle the stacks in order, but this order may be a bit weird in some games.

[sanslash332]

Oh! thanks for the fast answer @joeraz

Let's take this part by part :-)

before starting, here's another TTS library that works with various screen readers. It's in c++ with python wrapper. universal speech is old (but very used) on the field, so is very stable. Maybe a bit harder than others to use, but it can give good results too.

So, let's now answer things!

About how to start the speech system... Hum, maybe a shortcut to toggle it would be a good idea? Like pressing f8 to start the accessibility, making it easy for blind people activate it, and... well for regular players if they accidentally activate the feature, it will be very fast to turn it off. Obviously, show a tooltip /popup when the tts is started to show visual people that feature was turned on.

And in the background, well for remember the player's preferences, maybe save that setting in the game's config file. And if you want to add some customizable options for the TTS when using the sapi engine a small window in options can do that job... but... I think that thing can be for the future. It isn't important for now.

About the playstyle... Yes, absolutely yes. The second option of point and click is the best option to integrate keyboard movement.
After that, It is simple to announce to the player that when he selects a card, well that card is selected and when you move and pass again over that card, remember that that card was selected.
About the stacks, I guess that it's a first good approach allow the player navigate with the arrow keys with all the field like a grid, taking care about all elements of the field. Revealed cards, card stacks, discard pile if it exist, etc.
for example, if in a game you have ... I don't know... something like a 4 columns of 6 cards, with 4 stack slots abobe of each column you must present to the player a grid of 4x7, and allow to the player navigate over the cards, and the stacks simply with the keys.

next of that, with tab you can use that key to make fast movements, like move to the next stack when you have to go to the stacks to place a card, because you selected one, or depending the context of the game maybe the behavior of the tab key can change.

:-)

but with all these things, I guess that you are on a very good way :-)

Again, thanks for taking this into consideration! It's awesome!

[shlomif]

@sanslash332 : hi! I've thought of implementing a text-based REPL ( see https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop ) feature, which can either accept python code, or evaluate it using pyeval("…") or similar. This is given my disabilities which include hands stiffness. will this be accessible enough?

[joeraz]

I'll look at those libraries a bit more closely, and I'll see what needs to be done to get the keyboard movement working (I feel this also has other benefits - perhaps we can remove the mouse requirement from PySol). Though not all games are laid out in a simple grid. For example, Pyramid and the Interlock family games have multiple rows of partially interlocking cards. Or Clock has the piles arranged in a circle. I should be able to set this up so the methods can be overridden on a per-game basis, but given the number and variety of games, they will probably need to be tested individually.

[sanslash332]

Hello guys!

Thanks for your answers, so let's go:

@shlomif
A REPL mode sounds... interesting, as a first approach for a simple accessible mode.
If with that it is implemented a terminal that works flawlessly with the screen readers, can help to some people play the game, but for these times relaying on text-based programs aren't the best solution to target a widely blind audience, because it is... rough in user friendly gui terms.

If that is the way that you take, maybe te game could turns into a text-based game card for blind people and... well, I think that this can be a good first mode, but not the definitive; because are just a feu blind people that are familiar with text-based terminals.

For example doing that, the game for the blind people could be turned in a game similar a MUD.
I don't know if you know these old kind of games; that works with a full text-based system :-) Like a old text adventure.

@joeraz

OK, take the best library that fit into your needs :-)

And well, the grid movement was a first idea, because it is the simplest way to distribute the UI elements for move with the arrows keys.
Maybe for the majority games it can be the good way to move the cursor, but for other games maybe a movement through sections with the arrows can be useful?

I don't know, for clock when you reach the piles section with arrows moves through the piles, independently how graphically they are.

But... I think that this is going to mutch deeper on this thinking. The first things to do is get the speech working, and got read the cards in the field for the most simplest games.
After that, you could go forward with harder strategies for more complex games :-)

Again, thanks for your answer! :-)

[shlomif]

@sanslash332 : hi. Just for the record, I am familiar with such computer games as MUDs, the Z-machine, or https://en.wikipedia.org/wiki/Colossal_Cave_Adventure , but haven't played them a lot.

[joeraz]

Hello again.

So I recently finished building a prototype for the keyboard-based gameplay. It still needs some refinement and code cleanup, but it's functional as far as I've tested. You can find it in my fork here: https://github.com/joeraz/PySolFC/tree/feature/keyboard-movement. I don't want to merge it until it's gotten a bit more polish/refinement.

I've made sure the methods for navigating around the layout can be overridden on a game-by-game basis, so we can change how the navigation works for a single game (or type of game) if it makes sense. Of course, that means we will need to start testing for a lot of different games, to identify where the current logic would be insufficient. Anyone able to assist with this?

Currently, the controls for keyboard movement are as follows:

• Arrow keys - navigate around the game layout - each arrow will try to get the closest stack in that direction, so even if the cards are not laid out in a perfect grid, it should still be usable.
• Plus and minus keys - use the plus (or equals) key to select more cards in the stack, and minus to select fewer. This can be used to move stacks of cards.
• Enter - take action. This will indicate which stack you want to move or where you want to move it (it simulates a click on the selected card, so games that handle clicks differently will be taken into account). I'm also considering adding a shift+enter to simulate a right-click, which would usually be a quick-play move.

Next step would be the speech integration, but I wanted to get this out there, so it can be tested.

[germanico5]

Hi guys,
I'm really happy to see that this is taking its first steps. Once TTS is implemented, rest assured that I'll do all the testing I can. Regarding the idea of turning it into a console-based game, I totally agree with Sandl. It might be a quick way to make the game more playable, but just as some people feel uncomfortable with consoles, people also enjoy it more when accessibility is integrated into the game's own interface, allowing us to play under the same conditions as sighted players.
I'm looking forward to the next updates, and even though I don't talk much here, rest assured that I'm paying attention to everything that's being said. Have an excellent week, and thank you for everything!

[sanslash332]

Hello @joeraz
The feature branch for keyboard, has a testing release built for windows?

I've tried build it for myself, but I don't success.
It isn't easy simply run the game from source for windows.

I wanna try the keyboard movement; I know that I can't ear anything special, but maybe with some help of AI I can check if the movement is working or not.

Also, thinking on that, maybe a sounds when you move over the cards or stacks could be interesting?

Thanks

[joeraz]

@sanslash332 For any build (made within the last 30 days), you can pull a Windows installer from AppVeyor. To do this, go to the commit message for the commit you want - there should be a green checkmark next to it. If you select this, you'll get a list of build checks, which includes AppVeyor. Go to AppVeyor, select the "Artifacts" tab, and you can download an installer for that specific build from there.

[joeraz]

Okay, I have some basic speech logic built - same branch as above. If using the keyboard movement, it will identify each card you highlight as you move around, assuming you have NVDA installed and running. It will also identify empty stacks and the suit of foundations. If this method works, we should be able to call it anywhere in the app to trigger speech.

Still needs more testing though. And the code is a bit of a mess.

Currently, this will only work for French games (I need to build logic to identify the cards for other types of decks). Also, the ability to enable localization is still a work in progress.

[germanico5]

I'm very happy to see this taking shape! I just tested it with Aces Up, and while in theory, it should work fine, I noticed that when I remove a card with nothing underneath, the screen reader stops functioning in the game. @sanslash332 reports experiencing the same issue with the Tab key. This looks very promising—well done!

[sanslash332]

@joeraz

Wow, such amazing update with these two things :-)

Works very smoothli this first atempt :-)

Sometimes happened strange things, for example if you press the tab key the screen readers (and keyboard) stop to interact with the game field, so probably something is trigger abobe the instrucctions.

And well all things aparently works fine.

Next things that I think that should be implemented. For example, can you report when you're doing a bad move?
Currently you know that you do a illegal play simply detecting that the card that you try to move, returned to it original plase. Maybe sen to the screen reader message like "bad move" could be good.

And well, I suppose that more things coul be detteccted playing more.

Again ¡thanks for all! works amazing!

[germanico5]

A small update: I was able to complete a couple of games of Towers of Hanoi. Once you finish polishing the card reading, something will need to be done with all the unreadable dialogues, particularly the help and rules for each game, so I can learn more games and test them to ensure they work properly. I'm very excited about how things are progressing so far—thank you for everything!

[joeraz]

I just dropped a new commit that should address the issue you described with Aces Up. I'm still looking into the Tab key - something on the tkinter side maybe...

Regarding the game rules, you can find a full copy of the documentation online at https://pysolfc.sourceforge.io/doc/index.html, so if your browser has a screen reader, you can use that to check rules for games until we get the screen reader working on the in-app documentation. The one caveat is that this is the documentation from the last release build, so any changes to the documentation we may make will not be reflected there until the next release (though I branched the code off shortly after a release, so this probably isn't much of an issue at the moment).

I've also noticed some lag when I try to trigger the speech output but NVDA is not running, so I'm trying to sort that out before I add too many more speech messages.

[germanico5]

I must say that for being the first attempt at adding screen reader compatibility to the game, I am pleasantly surprised by everything I've been able to play today. While it wasn't too many games, I did learn some classics, and apart from Pyramid Solitaire, I haven't had major issues with others like Golf or Spider. I don't know how complicated it might be since I don’t understand the game structure, but would it be possible to add a message indicating if you're in a key area of the board, such as the talon, and add a button to move the focus directly to the talon or the waste pile, if present? That would make it much easier to play variants like the aforementioned Pyramid.
Speaking of Pyramid, could a message be added at the end of uncovered cards? It would make it much easier to know which cards are playable in this and other variants that require it. Congratulations on such a great implementation, and thank you for quickly addressing the issue with the empty spaces. I have a lot to try out!

[joeraz]

Dropped another update with a few enhancements:

• Shift+Enter will now trigger an auto-drop move (automatically moves the selected card to an appropriate stack if one exists).
• Tab will no longer break the keyboard movement, and will now cycle between the different stack types (talon, waste, rows, reserves, foundations, in order).
• The speech will now correctly identify cards in Hanafuda deck games.

[germanico5]

Greetings! Very good update. The tab key works well, and it makes it clearer that each board area needs to have its name properly identified, as right now, you can get disoriented when cycling quickly through the cards.
I tried to learn a Hanafuda game to give you some feedback, but unfortunately, I still haven’t managed to understand one. Do you know of an easy variant I could try? In any case, the cards are read correctly as expected. I haven’t encountered any major errors—perhaps movement could use some polishing in certain games, but that will become clearer over time.
Another useful feature would be messages indicating card movements and distributions, such as “You moved X card to X location,” “X card discarded,” or “Talon X card.” This would help avoid constantly navigating just to check which card was revealed for drawing.
Very good update! I'll keep working on Hanafuda to look for any issues. Thanks for all your hard work!

[joeraz]

For Hanafuda, Paulownia is basically just Klondike with a different deck, and the sequences are built by same suit, in reverse order. It's pretty easy due to the low number of ranks Hanafuda decks have.

I have a couple questions:

• While testing, I noticed that while the speech code is in place, but NVDA is not running, SAPI5 is being detected as the screen reader. When it tries to execute the speak command through SAPI5 in this scenario, the app hangs for a couple moments before continuing. It is not giving me anything to indicate it failed. This would become very problematic if I try to apply a speech command anywhere outside of the keyboard movement logic. Do you know anything about SAPI5 or an issue similar to this?
• What info do we want to include in the final message when selecting a card? I'm currently just outputting the name of the card, but there are other details of the stack in question, including the type of stack, number of cards, full list of cards, and the full stack description. What information do you feel should be in the outputted message? Is there some info that should only be relayed when pressing a certain key for more info (there are a limited number of keys, and many are already mapped, so I want to be careful how to organize any new key mappings)?

[germanico5]

There isn’t much I can tell you about SAPI, except that depending on where you are, you should make sure you have a voice installed. Normally, you should, but if you're using Linux, I can't confirm that all distros have it by default. I hope that once you manage to fix SAPI, you'll add a way to adjust the speech rate.
Regarding messages, there are a few approaches to consider: too much information can overwhelm the game's flow, but honestly, based on my experience with different blind players, some will need as much information as possible. This could be managed by allowing adjustments to verbosity—ranging from the most basic level to something extremely detailed, perhaps even including coordinates for board positioning. A menu where players can individually enable or disable message types, or even a single key that toggles between multiple verbosity profiles, could be useful. Some information could also be conveyed through sound to reduce the need for spoken messages.
As for the lack of available keys, I’m not sure if you’re really running out of them. You can always use keyboard shortcuts with Ctrl, Shift, or Alt, right? That’s something any blind PC user would already be accustomed to. There are messages that won’t be needed all the time, so having a way to check that information when necessary is important.
I have a question: when playing FreeCell, it tells me to consider red and black cards. What does that mean? Maybe there should be a verbal distinction for the color of each card.
Feel free to ask anything. I don’t send too many suggestions regularly because I don’t want to overwhelm you with things that might be better left for later in development. I’m not a programmer, so I don’t know the exact order in which things are usually worked on.

[sanslash332]

Hello @joeraz

I'm just testing the new added features, but as always thanks for your awesome job :-)

So, a few things that I've discovered.

First, exist a newer version of accessible_output, that have more recent commits that accessible_output2, check if is better for some things:
https://github.com/SigmaNight/accessible_output3

Second, I tested with the accessible_output3 an the sapi module is broken (maybe in accessible_output2 too).
If you use the speak method of the sapi (try to use with accessible_output3.outputs.SAPI5) it results literally in nothing. Any returned value, any speak, nothing.
But, if you use the output method... ¿it works?
The sent message is played over the active SAPI voice, but instantly after that, it throws an exception.
🤣

If you ask my, throw away sapi support. Supporting NVDA, Jaws for windows and maybe ZDR and pcTalk screen readers for asian people, will be OK.

And, add a toggle key (like f8) to turn on or off an accessibility flag.
When somebody press f8, check if one or more of the supported screen readers is active. The library has a is_active method on each output type to determine if the application is on. If almost one of the supported screen readers is active, well turn on the flag and use the auto module to speak throw it. And of course if any of the screen readers is detected as active, dont turn on the fflag to not fall into buggy sapi.

With the flag turned on, call the speak message. without it, nop. With that you can handle these problems when you add speetch messages outside the keyboard flow 😄

That could be a small suggestion for now.
And well, if we really want fix the sapi support, its time to kick away that issue into the lib's supporters. Is a bug from their side.

That for now!

Time to check more games!

Do you have a good and fun recommendation's?

Thanks!

ps: also tested accessible_output2 and yes, the sapi bug is present with the same behavior

[joeraz]

Thanks for confirming - this aligns with what I'm seeing. I disabled SAPI for now (I didn't want to do this without confirming, as I wasn't sure if disabling SAPI would be a problem). I think the issue should be reported to accessible-output - even if it's not a priority on our side, the developers should be made aware of it.

Did another build today - in addition to cleaning up the accessible-output versions, I expanded a few of the messages, to be different depending on different types of stacks - the talon and waste will be identified and output their number of cards, stacks in Pyramid will output which cards are covering them, and I also added a couple additional messages. When drawing cards from the talon, the drawn card will be spoken. Also, I added an "invalid move" message.

I've added the speech methods to my speech.py file - from here on out, it should be easy to add additional speech messages outside of the keyboard flow wherever we need. I currently can't add a speech message when opening a dialog - the dialog title is read automatically and this overrides whatever message I try to speak at this point. Aside from that, anywhere is fair game.

And the custom output in Pyramid is a bit of a proof of concept for being able to add custom speech messages to the stacks of different games. Since the games use inheritance, the change in Pyramid will also carry over to some related games (i.e. Giza). It's going to be important to identify any other games where a custom speech message might be needed.

I'm trying to be a bit conservative regarding the use of keyboard shortcuts as a lot of keys are already mapped (I do not want to mess with the existing keyboard mappings without a very good reason, as this will confuse current users), and if I try to use too many different keys to give different speech messages, it could be restrictive later. Right now, I think we'll need to add a keyboard map to output game data that's not related to specific stacks (i.e. the chosen card in Accordion's Revenge, the counter in Hit or Miss, or the base rank in Canfield). We can also add one to repeat or give a more verbose stack description.

To answer other questions - in a typical French suited deck of playing cards, the hearts and diamonds are the red suits, and the clubs and spades are the black suits. These colors do come into play in many games, as building the tableau piles by alternate color is pretty common.

And my personal favorite solitaire game is Yukon.

Does this all make sense? Do you have any further questions for me?

[germanico5]

Sorry for the delay, my email client didn’t notify me.
In the latest update, the game crashes right after the screen reader announces the name of the loaded game. This likely means it was trying to read the cards.

[joeraz]

@germanico5 Try it again now. There seems to have been a few issues with the output format and accessible-output3 in the compiled packages that were not occurring when I ran from source.

[germanico5]

Everything is working now, thank you very much!
I have a question. The Pyramid rules I knew allowed me to match cards from the talon without sending them to the waste first. Is this how it works in this version, or is there an issue selecting talon cards?
Everything else seems to be working fine. I like how the messages are coming along, though the card drawn from the talon in Pyramid is not being verbalized. Is that intentional?
Besides that, I have a couple of unrelated questions. I saw that a request to add Grid Cannons was made a while ago. Was it impossible to create using the game engine?
Could you add a slight delay before messages are spoken in dialogs? That would prevent them from being interrupted by the window title. Alternatively, making them readable with the arrow keys would help—so if a message gets cut off, pressing the down arrow would repeat it.
Also, you should add a Shift+Tab command to cycle backward through elements, so there’s no need to go all the way around.
Great job, you’re amazing!

[joeraz]

Thank you. So to answer your questions:

• First, as a disclaimer, Pyramid was added to PySol before I became involved in the project, so I can't say anything for certain about my predecessors' intentions. That being said, this is how it was always written. I have seen some dummied out code for an incomplete variant called "Thirteen" which would have the top two cards of the waste being playable, which would make things work as you describe. I've been meaning to look into what would be needed to complete that game (possibly rename it "Pyramid Thirteen" to better differentiate from "Thirteens"), I just haven't got around to it yet.
• The code to read the card when dealing from the talon is part of the base WasteTalonStack. This should work for most games, but may be overridden by some games that use custom logic. So this isn't intentional, though it may be best to have it work differently as the Pyramid talon's top card is face-up. We'll want to test as many games as possible to identify these exceptions.
• Grid Cannon isn't impossible to create with the engine, but it will be difficult. Adding it is still very much on my to-do list, though I'd need to work out some details before I get to it.
• I'll look into the delay idea and the shift+tab function.

[joeraz]

Just dropped another update:

• Cards for Tarock, Hex A Deck, and Ganjifa games will now be spoken correctly.
• The face-up talon, used by Pyramid, will now speak the cards when dealt.
• Memory games should work - the game will announce cards as you flip them. Though this currently won't recognize any alternate decks you may use for them.
• The shift+tab shortcut has been added to cycle through the sections in reverse.
• The Q key is added to have the game speak any extra info about the game, that may not be reflected in the layout of the cards. It's not relevant to all games, but can be important for some. For example, Accordion's Revenge will tell you the goal card, Memory will tell you the current score, Hit or Miss will tell you the current counter, and relevant Canfield games will tell you the base rank for the foundations. We'll need to identify other games where enabling this function will be necessary.
• I identified and fixed a few more bugs.

[germanico5]

What a great update! I have a lot to test, but here are some quick observations:
• Poker-based games should have the Q key working. By the way, I think it’s a great feature, and it works very well!
• From what I can tell so far, the reading of the new card sets works as expected.
• Maybe the foundations should also announce how many cards they contain.
• Although movement in Pyramid is still a bit confusing due to the large number of blank spaces, I almost won a game, so I’d say the progress on this mode has been quite successful.
• I know this is something for later, but if it’s not too much trouble… could you consider making the sound settings accessible? Mainly for adjusting the music, which tends to be a bit loud. The rest of the settings can remain inaccessible for now if making them accessible would slow down the addition of new features.
• I know this won’t change your priorities, but I’m really looking forward to the addition of Grid Cannons—I love that game! Playing it physically is a lot of fun.
I have so many games to check out that I don’t even know where to start, but I’m really happy with what’s been accomplished so far.
It might seem annoying that we thank you so often (if so, let us know, and we’ll tone it down), but what you’re doing is incredibly valuable to us. Accessibility projects don’t usually get this far, and after some recent unsuccessful attempts elsewhere, I’ve come to appreciate even more how amazing your work is.
Thanks again for everything!

[joeraz]

Got another update:

• I set it up so a stack can be configured to be unselectable through the keyboard movement. I'm using this for stacks that the user has no reason to interact with - such as the talon in games where all cards are dealt at the start, or empty/face-down piles in Pyramid family games. Will want to look for other places this can be used.
• I added the number of cards on the foundation messages.
• Also, I looked into the sound settings dialog, and I got the speech working there too (along with on the buttons of a few other windows, as that logic is shared). The components all just had to be replaced with customized versions with the speech built in, so getting it working on other options dialogs would just be a matter of switching out the default components for the custom ones.

Also, I merged over the latest code from master, as some of the dialog changes are dependent on a recent bugfix there.

[germanico5]

Very good update as always! I hope that more menus can become accessible in the future. The removal of unnecessary spaces helps a lot—it makes Pyramid easier to play, and I’ll definitely let you know if I find another game where hiding those elements is useful.
One issue with the menus is that pressing Enter does not activate the "Cancel," "Apply," etc., buttons. However, navigation with Tab and the sliders work well.
I'm having a problem that I’m not sure is due to my lack of knowledge of the game or an accessibility issue. When playing traditional Mahjong, I don’t understand how to find free tiles. I’m not sure if there are blank spaces that I can’t perceive or if I just don’t know how to play it properly. The help says that a good strategy is to remove the deeper tiles, but I can only click on the ones at the outer edges of the grid.
Would it be possible to add a way to view coordinates? In some games, it would be useful for orientation. If added, it should be something the user can enable or disable at will, since its usefulness depends on the game and the player. This idea came to me while testing Mahjong and struggling to understand the tile layout.
That’s all for now—great work!

[joeraz]

So of all the remaining games, Mahjongg is probably the most difficult one to get working, and likely one of the biggest obstacles to getting this branch in a place where I'm comfortable merging it. The speech code will naturally need to be updated to recognize Mahjongg tiles.

But the bigger challenge is the layouts - there are many different Mahjongg layouts. In fact, all of the different Mahjongg "games" are the same rules, but with a different tile layout. A tile is free if either the left or right side of it is exposed, and there are no tiles on top of it. In the traditional layout, there are no gaps in the middle of the layout (just the edges), though this is not true for every layout. On top of this, Mahjongg is the only game with a fully 3D layout. In other games, cards that are played on top of each other overlap fully, so you can't see the lower cards, but the way the Mahjongg tiles are designed, they're offset slightly, so you can see the edge of tiles that are fully overlapped by other tiles, but can't identify the tile. Depending on the layout, it's also possible for tiles to partially overlap, so you can see and identify them, but can't pair them off.

I think the keyboard movement logic might need to be overridden to compensate for the 3D offset, we might need to add some dialog for indicating the blocked tiles, and make the lower layers unselectable (though I wouldn't be sure how to identify if a tile is fully vs. partially covered). Will need to review the code.

[germanico5]

I see. It’s going to take some time before we can comfortably handle Mahjong boards. I still don’t fully understand how it works, so I don’t have many ideas for now. But we’ll see.

Moving on to other things: in some games like Fifteens or Tens, some or all of the cards are not being verbalized. They used to be, but I’m not sure when they stopped working.

And in Bits and Bites, the help mentions 4 columns for bits and bytes, but with the cursor, I can only find 3 of each. Is it just me not understanding, or is it a bug?

Hope your week started off well.

[joeraz]

I'll drop a few more details on Mahjongg when I've done the preliminary work. For now, it's probably best to focus on French, Hex A, Tarock, Hanafuda, and Ganjifa games, until I get the core logic in for the others.

I found and fixed the issue with Fifteens/Tens - some of the updated Pyramid code was inherited by those games and didn't play nice with the layouts, but I took care of it. That's in the build I just dropped.

For Bits n Bytes, it's six columns, but it's two byte stacks and four bit stacks. I added some special speech messages to this game to indicate this, to avoid confusion. That's also in the build I just dropped.

Also, I just extended the Pyramid logic to work the same for the remaining games that have a Pyramid-like layout (i.e. Interlock and Three Peaks), and I added some logic to get the keyboard movement to respect the pause logic. If you pause the game (using the menu or by pressing P), then try to use the keyboard movement, it will just give a message indicating the game is paused, and there's a speech message to notify you when you pause/unpause the game.

[joeraz]

I just took a stab at Mahjongg.

So to try and explain Mahjongg a bit better, each Mahjongg "game" is actually the same game, but with a different layout. To help understand this, I'll try to describe how one would set up the Traditional layout using real tiles. This is tricky for me, so hopefully this makes sense. Let me know if it doesn't, and I'll try and clarify.

• First, set up the bottom layer of tiles. There are eight rows of tiles, containing 12, 8, 10, 12, 12, 10, 8, and 12 tiles in them, from top to bottom, aligned to the center.
• Then, a six by six grid of tiles should be placed on top of this layer, in the center. A four by four layer is placed centered on top of this, and lastly, a two by two grid is placed on the top.
• Finally, a single tile is placed on top. The next two tiles are placed on the bottom layer, in the middle of the two middle rows of 12 on the left and right to make points. A second tile is placed to the rightmost of these two files.

The other layouts naturally just have the tiles arranged in a different way at the start. For example, "Mahjongg Lizard" has the tiles arranged to resemble a picture of a lizard. The traditional layout does not have any gaps in the middle, but other layouts do.

As for the logic, the keyboard movement mostly works, though it can be a bit finicky at times - in the traditional layout, the centermost tile sometimes gets skipped over. It can be more or less reliable depending on the cardset and scaling used, but aside from the center tile, the rest doesn't seem to be an issue. Unfortunately, it was tough to get it working this well, and it took a few hacks to make it happen. Is there another dev who'd be willing to look at the _getKeyboardSelectStack method in Mahjongg.py, to try and improve it a bit?

Other than that, the tiles will speak the correct tile name and any blocking tiles when selected by keyboard. The lower layer tiles cannot be selected, as they're concealed by the tiles laid on top. Currently, partially covered tiles can't be selected, even if you could partially see them. I'm not sure if/how to handle for this.

The Mahjongg changes should also work on Shisen-Sho type games (though there are no blocked tiles there) as they share a lot of logic.

Let me know if you have any thoughts/suggestions on this?

[sanslash332]

I've reading some stuff about the rules and how to play mahjongg solitaire, and... hmm some things that I've Thought so could work or not are.

First, fix the keyboard navigation on a selected layer of the game, and assign a key to jump over layers.
For example, at start, you can move on the botom layer of tyles with arrows keis, to know which tyles are blocked by abobe tyles.
Following your layer description, next if you press... I dont know... the shift+up key, you go one layer upwards, so you are on the 6 x 6 tyle layer to check which tyles are on that.
And you can continue moving through layers with shift+up or shift+down to have a good mental scenario of the game field.

Of course, when you are over a tyle, the hgame should speack you which card is, if it is free for the left or right, if it sealed by the abobe layer and which other tyle, and maybe some other useful information.

Whith that i think that we have a fun and complete system to navigate through the all tyles and layers.

But, a important problem that I'm seeing with that now that maybe should be clarified with a message at the start of the game, that all layers aren't seted exactly one over other with all tyles directly one abobe other, and the field resembles a shape of a figure, to give the player... a small idea how the game area looks to think a strategy 😃

This could work. is a small idea hmm.

Thanks soo far.

[joeraz]

I like the option of making a key option to swap between layers - that would actually resolve most of my current issues from the dev side on the movement throughout the mahjongg layouts. And I can add that description to the mahjongg rules in the documentation.

While I'm working on that, I just dropped another build where I enabled parsing of cards/pieces for the remaining games (Matrix, Pegged, Lights Out, Ishido, Samegame), so those can be tested. The only thing I wasn't able to do is find a way to correctly parse the Tile Puzzle game pieces, as those games are extremely visual.

[germanico5]

I'm glad that Sanslash332 came up with something—I couldn't fully understand the game to make suggestions. The vast majority of games work well, except for SameGame. Not only does the screen reader not read anything, but when I try to load some variants, it loads Klondike instead. Other than that, I loved the new games.

[germanico5]

Now, a couple of things that came to mind: Games that start with many blank spaces and have large boards can become very disorienting. I think a coordinate system and a sound and/or message when hitting one of the edges would be very useful to give the player a sense of the play area. I hope you can make more game windows accessible at some point—particularly the rules window for each game would be very useful, as I wouldn’t have to open the browser to learn a game, which would speed up the process of trying new ones.
I found it funny that puzzles ended up being a number-ordering game. I know that wasn’t the intention, but it’s amusing, so leave it that way, haha. The "Q" key should work in poker-style games. I’m not sure if it would be complicated to implement, but could the Tab key avoid moving to the blank space in games where there’s only one important one? In some games, it can be uncomfortable to find if it appears randomly on the board.
I think that’s all for now—excellent work!

[joeraz]

Dropped another update - I tried to get the help/game rules window and other dialogs to speak. Though it seems to be working, I found it very finicky during testing. The automatic attempts to read the dialog title tend to override my speech triggers - I got it working by adding delays and instructing my speech commands to interrupt, but there have been some times where lag kicks in, or something else triggers the dialog title to be re-read.

Also, fixed the issue with Samegame, and Poker games should give the current score when pressing Q.

I'm not sure I understand what you're asking around the tab logic. Currently, it will switch between the talon, waste, first selectable tableau, first selectable reserve, and first selectable foundation, in that order (skipping whatever doesn't exist). Could you clarify what you'd be looking for?

[germanico5]

I'll explain the issue with empty spaces.

Some games, like FreeCell, rely on these spaces, while others, like the empty spots in SameGame, do not. I'd like the Tab key to move to those empty spaces that are actually important in a game.

Since we're on the topic, could you rename spaces where cards cannot be played to distinguish them from playable spaces?

I’m not sure if you’ve already made all card sets accessible, but in matching games like The Big Tetton (or however it's spelled), some cards are still not read.

Let me know if all cards should already be readable so I don't bring up issues that might not be on your immediate roadmap.

Thanks for everything! As far as I’ve tested, all the special games work really well now.

[joeraz]

So basically you don't want the tab key to select stacks that have no cards and can't be played to? The challenge there is that those stacks are very difficult to identify on a code side. There are multiple combinations of variables that could create a stack like that, and some games have completely custom accept cards logic. I did an initial test, and couldn't get the used up empty stacks in La Belle Lucie identified with the common variables.

I made a few more updates. After these updates, all the card sets should be accessible (except puzzle type cardsets or if you're using a different type of cardset than the game expects). If any other games are still giving the wrong card messages, I'd have to update the individual game logic.

[germanico5]

No worries, I'll drop the Tab idea. It's not a big deal anyway.

Thanks for the amazing work—I'll check it out now!