Organizing files

[Strauman]

I'm thinking about how to organize my exercises. I like the one-big-file-strategie because it's simpler but it's harder to debug if something goes wrong. I do not fully understand how you organize your files now. Do you have one big file or do you have several files?

Originally posted by @thorstengrothe in #103 (comment)

[Strauman]

My use is for multiple exercise sets. One for each week that students solve.
So each of these weeks have its own \makeset.

I started putting them into categories in the folder structure, like exercises/category1/some-ex1.tex, exercises/category1/some-ex12tex, exercises/category2/some-ex.tex and so forth. This presents a problem when an exercise belongs to e.g. two categories. I can't really think of a smart solution to this, that does not involve an actual database-style (e.g. SQL) of solving this.

[Vouivre]

If I'm right, you work with single files for the exercises, like @thorstengrothe and since the beginning I did it wrong with big-file-strategie 😁 .
I decided to change because I want to use the point system and the mixnmatch feature. But:

• probably I found some problems in the point system or regressions
• I'm not sure anymore if I want to use it..... by writing this answer I'm pretty sure I'll go back.

I splitted some big files last week and slowly I get the impression I'm less efficient and I try to find workarounds to be as efficient as before.

If I split my big files I get basically the same structure as you have with the same problem: what to do if an exercise belongs to two categories. Perhaps the tag system can help.

In short: the problem is when I edit my exercise sets.

1. Sometimes, I reuse an exercise or more with another class.

For example:

week 1, class 1:

problem 1
problem2
problem3
problem4

week1, class2:
problem 1
problem2
problem5
problem6

The theme of the exercises are the same, let's say it's about numbers. If I forget the problem number, it's simple to get all exercises:

\makeset[nohead]{myExerciseset}{databaseNumbers}

where databaseNumbers.tex contains all problems related to numbers. In the PDF with all exercises, I take the exercise Numbers I want: here problem 1 and 2.

Then I have only to use:

\makeset{myExerciseset}{\select{databaseNumbers}{1,2}}

And that's all. If I use:

\xdef\exercisefiles{exercise1}
\foreach\i in {2,...,70}{%
\xdef\exercisefiles{\unexpanded\expandafter{\exercisefiles},exercise\i}
}
\edef\makesetargs{{myExerciseSet}{\exercisefiles}}
\expandafter\makeset\makesetargs

it works but it's less convenient.

And sometimes I don't know if I can reuse a problem for another class. I print all exercises in a PDF, take a look if I can find something I could reuse, then I take the problem number to reuse it.

2. With big files, I understand the tag system. With small files, there is surely a use for tags, but it's less evident for me. If I want to search all problems with "negative numbers", I choose my big file and I filter the problems with those tags. It's fast and easy to get everything I want.

3. Sometimes I write similar problems, for example I change only the numbers. Each week I have some short exercises to review what I saw with the students. If there too much errors, I can take the same exercises, change some numbers and redo them the next week. If I want to redo three exercises, I don't have to find the files, to open them, copy, create new ones, and paste to new ones. In that case with the strategy one exercise, one file I'm much slower.
   The search, copy and paste in only one file is faster.

4. Very often I write similar problems. If I have one file, one problem, which problem can I copy and paste ? I probably have to grep it outside from my editor. Then open the file, copy, paste, create a new file, and I'm done. With a big file, I stay in the editor and I simply search in my big file. I copy and paste at the end of my file. Done.

Take into consideration my biggest file has more 300 problems. Perhaps I will split it. It means about 12000 lines of code. To create a pdf with all exercises, I think I need less than 30 seconds and to include some exercises it's only a few seconds. Your package is very fast, congratulation! With the last package, xsim, sometimes I needed a couple of minutes.

[thorstengrothe]

It's good to see this discussion here. As in the first post said, I like the one-big-file-strategie. In the meantime I tried to split my exercises but for me it seems to inconvenient and less effective. Let me explain.

I got 5 courses with different topics. Each course writes 4 large exams per year. That are 20 exams per year plus sometimes a test in between. Let's say I got 25 exams per year. I have a lot of work so I have to be as effective as possible. I'm working with emacs and auctex and org-mode here. One large exercise file is divided into subtopics. This can be easily done with an fake section command in emacs.

%\section{my topic one}
%\section{my topic two}
...

with outline-minor-mode I can easily move between fakesecs and manipulate theme: moving around, cut, copy and of course fold them away. The advantage is, that latex doesn't see the fakesecs because for latex they are comments :-)

See screenshot here.

Finding some stuff in my large file is absolutely no problem with the powerful emacs search tools (e.g. swiper, grep etc.). I guess with other editors like sublime it should be the same.

I name my makefiles according to the course numbers like so:

\makeset[intro]{Course20191}{filename}
\makeset[intro]{Course20192}{filename}
...

This way, after some time, I have all my exams in one file. Searching for specific subtopics is easy in one large file with the tags-command. One problem stays. When I have a new course of course I want to reuse my exercise material from the course of the last year but not all exercises. Until now I did not find a clever strategie for this. It would help very much if every exercise has a tag that shows two things:

1. last usage
2. in which courses

The disadvantage of this approach is that it is more difficult to debug if somethings goes wrong in the file. For tex capacity exceeded error reported here it took me two day to find the error :-) Small files are easier to debug but less convenient at least for me.

Maybe the best organisation strategie depends not only from the latex sty file but on a good editor too?

[Vouivre]

In between I have grouped again my exercises and I will work from now with one-big-file-strategy. First I notice you use the sames tools as I use: emacs, auctex, swiper and org-mode is on my list. In my workflow I also you use some makefiles. Perhaps with other tools, this strategy is not the best, but I have some doubts.

The disadvantage of this approach is that it is more difficult to debug if somethings goes wrong in the file. For tex capacity exceeded error reported here it took me two day to find the error :-) Small files are easier to debug but less convenient at least for me.

By splitting my big files, I discovered some errors. Now I know how to split and to go back. Perhaps you could try the following: at the beginning search the error a couple of minutes. If you don't find the errors, split the file in multiple files. I don't know which OS you are using. On Linux csplit will help you to easily split your files. It's a one line command. here is what I used:

csplit --prefix=exerciceCm databaseCm.tex    "/\end{solution}/+1" "{*}"

I got exercises exerciceCm01.tex, exerciceCm02.tex, etc. from the file databaseCm.tex. The file databaseCm.tex was splitted after each \end{solution}. Then use the code in my first comment to display multiple exercises. Display the first 20. No errors, the first 40. No errors go on until an error is found. When the error is found, you don't have to concatenate the files because you kept a copy. If you have a file dedicated to debug in that way, it can be more efficient.

One problem stays. When I have a new course of course I want to reuse my exercise material from the course of the last year but not all exercises. Until now I did not find a clever strategie for this. It would help very much if every exercise has a tag that shows two things:
1. last usage
2. in which courses

You have probably think to the manual option. You can use a tag for that or a comment in Latex, but a more efficient one would be better. In another pacakage, there was the possibility to add some metadata. Probably the best would be that a line like

\metadata{last use=..., last exercise set=...}

would be added in the source file. It would be nice.... if everything works perfectly, with no bugs ;-). exercisebank creates the file comment.cut, why not creating a log file and insert those data. After the user could manage this file as he wants to. He could manually copy the informations he wants to in his source file. A little bit less efficient, 100% secure.

One question: are you able to use the point system with big files ?

[thorstengrothe]

@tristelune1

One question: are you able to use the point system with big files ?

in short, because stress level is at maximum :-) at the moment, yes I have no problems with point system. In a few days we can discuss this in deep, maybe I can help...

[Vouivre]

No problem. Perhaps you can send me a document which I can compile to see if it works on my system.

[thorstengrothe]

Sorry for my late response, I have a lot of things to do at the moment and a lot of exams :-)
I can confirm your issue with the points. I have a big file here and for some exercises the points are not shown. At the moment I do not know the reason for this and I have no time to search in deep. I'm pretty sure this is a bug..

Anyway the value of \totalpoints is correct, so they are counted but not shown?