Parallel letter frequency

[kmarker1101]

This should complete the unimplemented 48 in 24 exercises. The solution uses multiple threads.

[BNAndras]

I'm a bit reluctant here. Multiple threads isn't really parallelism since the threads aren't executed simultaneously. @fapdash, thoughts? I know a few tracks have foregone this exercise because parallelism is a lot more difficult to pull off than multiple-threading.

[kmarker1101]

Understood. I think the issue here is emacs in really single threaded, much like javascript, so I don't think we can do true parallelism. We can use a separate process, but I don't know that that would be true to the exercise either.

[kmarker1101]

from the emacs docs

_Emacs Lisp provides a limited form of concurrency, called threads. All the threads in a given instance of Emacs share the same memory. Concurrency in Emacs Lisp is “mostly cooperative”, meaning that Emacs will only switch execution between threads at well-defined times. However, the Emacs thread support has been designed in a way to later allow more fine-grained concurrency, and correct programs should not rely on cooperative threading.

Currently, thread switching will occur upon explicit request via thread-yield, when waiting for keyboard input or for process output from asynchronous processes (e.g., during accept-process-output), or during blocking operations relating to threads, such as mutex locking or thread-join.

Emacs Lisp provides primitives to create and control threads, and also to create and control mutexes and condition variables, useful for thread synchronization.

_

[fapdash]

@kmarker1101 You can do parallelism via asynchronous processes.

You can have a look at how emacs-async has implemented this:

• function that gets passed the elisp code: https://github.com/jwiegley/emacs-async/blob/cff2bd0be3c78a2eb76717eed60302972fe9b8c5/async.el#L479-L560
• starting of process: https://github.com/jwiegley/emacs-async/blob/cff2bd0be3c78a2eb76717eed60302972fe9b8c5/async.el#L421-L456

@BNAndras You know better what's in the spirit of the exercise, but I think it's purposefully formulated in a vague way to accommodate language differences, as long as it's running in parallel.
I think it's a cool exercise to have because while working with processes is certainly advanced it's also necessary to deal with when writing certain functionality without blocking the UI.

[kmarker1101]

thanks @fapdash . I will look into it now

[fapdash]

Maybe we could guide students by providing an extra file with a #!/usr/bin/emacs -x shebang that they're supposed to call with their processes? (https://www.gnu.org/software/emacs/manual/html_node/emacs/Initial-Options.html#index-_002dx)

But I think we'd have to add a track specific hint either way. In the hint we could also describe how you would pass code from your solution file to a different process like emacs-async does it and it seems to be a more general solution.

[fapdash]

@kmarker1101 See https://github.com/exercism/emacs-lisp/blob/bfde2aab4cd60302fd8a198e1e5307425a23a11e/exercises/practice/nucleotide-count/.docs/hints.md?plain=1 for an example of a track specific hint.

The file needs to have that level-2 heading called General to show up:

The hints must appear as a Markdown list under a ## General heading.

• https://exercism.org/docs/building/tracks/practice-exercises#h-file-docs-hints-md

[kmarker1101]

interesting why FAILED 4/13 ignore-punctuation is failing CI, but does not fail locally

Selector: t
Passed:  13
Failed:  0
Skipped: 0
Total:   13/13

Started at:   2024-06-24 19:09:19-0500
Finished.
Finished at:  2024-06-24 19:09:20-0500

.............

running in batch also passes locally

emacs -batch -l ert -l *-test.el -f ert-run-tests-batch-and-exit
Loading /Users/kevinmarker/projects/exercism/emacs-lisp/exercises/practice/parallel-letter-frequency/.meta/example.el (source)...
Running 13 tests (2024-06-24 19:14:10-0500, selector ‘t’)
   passed   1/13  combination-of-lower-and-uppercase-letters-punctuation-and-white-space (0.110582 sec)
   passed   2/13  ignore-letter-casing (0.110950 sec)
   passed   3/13  ignore-numbers (0.109535 sec)
   passed   4/13  ignore-punctuation (0.109225 sec)
   passed   5/13  ignore-whitespace (0.109308 sec)
   passed   6/13  large-texts (0.110675 sec)
   passed   7/13  many-small-texts (0.109583 sec)
   passed   8/13  no-texts (0.000032 sec)
   passed   9/13  one-text-with-multiple-letters (0.109377 sec)
   passed  10/13  one-text-with-one-letter (0.118365 sec)
   passed  11/13  two-texts-with-multiple-letters (0.110882 sec)
   passed  12/13  two-texts-with-one-letter (0.109489 sec)
   passed  13/13  unicode-letters (0.109667 sec)

Ran 13 tests, 13 results as expected, 0 unexpected (2024-06-24 19:14:12-0500, 1.328610 sec)

[kmarker1101]

@BNAndras This should be ready for review now. I am now processing each string in parallel using processes.

[BNAndras]

As @fapdash mentioned, I do want the hints file and the extra file students can use. I believe to have the extra file show up in the UI, you need to add it to the solution array for the files object in the exercise config.

[kmarker1101]

@fapdash I am a little confused on how a student would use the extra file. Can you give me an example?

Also, would the shebang need to change based on platform(ie windows) and location of emacs binary?

[fapdash]

@BNAndras @kmarker1101 I was just thinking out loud when it came to the extra file. I think the current solution is great. Calling an extra executable script vs passing code to emacs --eval doesn't make a difference and --eval seems more simple. But I also think more information to guide the students is needed.

What I would suggest:

• add a .docs/introduction.append.md: Tell the student that they are meant to use asynchronous processes as it's the only way to do parallelism in Emacs and point them to the documentation for that. You could argue that this is part of figuring out the solution, but I think it's a bad experience if you can pass the tests while missing the point of the exercise.
  @BNAndras I think my view is not really in line with Exercisms policy on that, can you please clarify?
  We could also add a test spy via an advice to check if the solution is calling make-process.
• add a .docs/hints.md: As the docs tell us, we don't want to give away the solution, but point towards it.. The student probably needs a link to batch mode, sentinels, and receiving output. @kmarker1101 You probably know best what is helpful as hints without spelling out the solution.

[BNAndras]

I think we actually want an instructions.append.md since this exercise only has an instructions.md. Otherwise, yeah it's fine to give more track-specific information about what's expected. Python does that a lot, and I've also done it at a few points for Pyret when I'm dealing with a practice exercise that requires something that's not common in the other exercises. Something like "This exercise expects you to work with asynchronous processes blah blah here's a link to documentation" gets the point across. We can add examples later if students are having trouble with just the docs.

f they don't use async code, the student is only cheating themselves here so I'm not convinced we need to test if make-process is used. If we do want to add it, I'd say we make this prominent in the test suite so students aren't left thinking this was a hidden requirement. It can be discouraging when your code which should pass all the tests normally instead doesn't.

[fapdash]

@kmarker1101 I've added the extra file for the code supposed to run in the subprocess. Writing code inside of a string must've been painful and not something I think the students should have to deal with. wdyt?
If we don't provide the extra file explicitly the students won't be able to submit an extra file to the test runner. (see config.json)

[fapdash]

@BNAndras Can you please verify that my changes to the config.json for the extra solution and example file are correct?

[fapdash]

I feel kind of silly now, but I realized that we can put the function for the subprocess into the main solution file, so the additional solution file is not needed. 🤷
See af9b857.

[kmarker1101]

thanks @fapdash I was missing this piece the whole time. That is why that ugly string was in there.

[fapdash]

@kmarker1101 Yey, I found a way to further simplify the solution code. :) See fae9f65.

I had memorized that the printable representation of hash tables can't be read, but that info was wrong:

You can also create a hash table using the printed representation for hash tables. The Lisp reader can read this printed representation, provided each element in the specified hash table has a valid read syntax (see Printed Representation and Read Syntax).

• https://www.gnu.org/software/emacs/manual/html_node/elisp/Creating-Hash.html

This leads to problems if you want to mutate those hashtables after reading as

Note, however, that when using this in Emacs Lisp code, it’s undefined whether this creates a new hash table or not. If you want to create a new hash table, you should always use make-hash-table (see Self-Evaluating Forms).

A self-evaluating form yields a value that becomes part of the program, and you should not try to modify it via setcar, aset or similar operations. The Lisp interpreter might unify the constants yielded by your program’s self-evaluating forms, so that these constants might share structure. See Mutability.

• https://www.gnu.org/software/emacs/manual/html_node/elisp/Self_002dEvaluating-Forms.html

but since we don't want to do this we're good.

[BNAndras]

@BNAndras Can you please verify that my changes to the config.json for the extra solution and example file are correct?

To have additional files in the editor, I think you need to add an editor array with those file paths so they get loaded correctly.

[BNAndras]

Suggested change

	In Emacs Lisp this is achived by using [`asynchronous processes`](https://www.gnu.org/software/emacs/manual/html_node/elisp/Asynchronous-Processes.html#:~:text=An%20asynchronous%20process%20is%20controlled,%2Dtype%20(see%20below)).
	In Emacs Lisp this can be achieved by using [`asynchronous processes`](https://www.gnu.org/software/emacs/manual/html_node/elisp/Asynchronous-Processes.html#:~:text=An%20asynchronous%20process%20is%20controlled,%2Dtype%20(see%20below)).

Small typo fix and grammar fix so we're not saying this is the only way to do parallelism.

[kmarker1101]

done

[BNAndras]

I'm thinking of bumping this from rep/large (+30) to rep/massive (+100). This has been a bit of an involved PR :)

[fapdash]

@kmarker1101 I made the changes that @BNAndras requested to the parts that I contributed, leaving the rest up to you.