Loremization

[mkoskim]

For generating files for testing purposes, I could implement "loremization" feature. What this does is that it replaces all text blocks with equal amount of "Lorem ipsum", but preserves the structure and statistical info. This sort of "anonymizes" the story, so that I could put actual stories as example & test files, without risking their publicication (most writing competitions and publishers take only previously unpublished stories).

Reason: I would like to put some of my WIP (Work In Progress) stories as an examples to the repo. They can enlight people who to use different features in real world scenarios.

The problem is that writing competiotions and publishers only allow unpublished stories. If I put my actual stories to repo, I can't send them to competitions or publishers anymore (there are some exceptions, thought).

Solution: "Loremize" the story. That is, maintain its structure, but replace all the words with some nonsense. It would be good, if all the other features (like word counts) still give the same result.

This is entirely a "support" feature which is not needed for daily work. It could be good just to produce example files from WIP projects.

[mkoskim]

I added this as a "good first issue". It is pretty simple, but not THAT simple. What is needed:

• First, this works as making a new file. So when the button / hotkey is pressed, we create a new document.
• We scan through the existing document, and create a deep copy of it - except that we replace all words with "lorem ipsum" -sequence.
• Once we are done, we set the new document, just like we do when creating a new one

There are probably lots of small details to be covered, but basically we map() the document through a loremizer, and all the text nodes (inside blocks) we go through and replace the content with suitable lorem sequence. If the sequence feels too hard, all the text blocks can be replaced e.g. sequences of some letter or "test" word (separated by space).

The main thing is that the word count function returns the exact same number of the words (text, missing), and that the structure of the document is maintained.

If you are interested, you can ask me more details about the buffer format, how to go through the buffer, and how to make it a new file.

[ZacharyHandel]

To be clear on this, the sequence of functionality/use case would look like the following:

1. Have an existing document.
2. In the editor, press some button relating to putting it through "Loremization".
3. This creates a new document with the same structure as the initial document but with "Loremized" text that preserves the structure of the initial document.
4. Be able to save and export this new document as a NEW document.

How does this sound? I wouldn't mind any supporting information regarding buffer format and the new file functionality. I can figure it out myself, but any help is appreciated. Thanks!

[ZacharyHandel]

Task Description

User Story

As a writer, I want to be able to "anonymize" my document to avoid risking publication and to use actual stories as test files.

Acceptance Criteria

• Once a document is saved and consists of text, the user should be able to clikc the "Loremize" button on the editor view.
• Upon clicking the "Loremize" button, a new unsaved document should be created.
• The newly created document should hold the same amount of data (same size visually) as the previous document with placeholder text for testing integrity purposes.
• The new file should be saveable as a .mawe file.
• The new file should be saved as a different file from the previous file.
• All development components and services should follow proper contexts.

[mkoskim]

To be clear on this, the sequence of functionality/use case would look like the following:

1. Have an existing document.
2. In the editor, press some button relating to putting it through "Loremization".
3. This creates a new document with the same structure as the initial document but with "Loremized" text that preserves the structure of the initial document.
4. Be able to save and export this new document as a NEW document.

How does this sound? I wouldn't mind any supporting information regarding buffer format and the new file functionality. I can figure it out myself, but any help is appreciated. Thanks!

Yes, that's the general plan. I update the issue description, but the general purpose for this feature is this:

1. I would like to put some of my WIP (Work In Progress) stories as an examples to the repo. They can enlight people who to use different features in real world scenarios.

2. The problem is that writing competiotions and publishers only allow unpublished stories. If I put my actual stories to repo, I can't send them to competitions or publishers anymore (there are some exceptions, thought).

3. Solution: "Loremize" the story. That is, maintain its structure, but replace all the words with some nonsense. It would be good, if all the other features (like word counts) still give the same result.

This is entirely a "support" feature which is not needed for daily work. It could be good just to produce example files from WIP projects.

---

Quick reference: Element types:

https://github.com/mkoskim/mawejs/blob/master/src/document/elements.js

---

The buffer structure is following:

doc.body - Body of the story
doc.body.acts - List of acts in the story
doc.notes - Story notes
doc.notes.acts - List of acts in the notes

acts = [
  {type: "act", ..., children: [ ... ]}, // Act block, containing header and chapters
  {type: "act", ..., children: [ ... ]},
  {type: "act", children: [
    {type: "hact", ..., children: [...]} // Act header (paragraph, see below)
    {type: "chapter", ..., children: [
      {type: "hchapter", ..., children: [...]}, // Chapter header (paragraph)
      {type: "scene", ..., children: [
        {type: "hscene", ..., children: [ ... ]}, // Scene header, three types, paragraph
        {type: "p", ..., children: [...]},
        {type: "p", ...},
    ]}
  ]},
]

Headers are not necessary for the first act, chapter or scene, but they are mandatory to separate those blocks from each other. Scenes have content type, and the content type is determined by the header type:

{type: "hscene", ...} // Content type scene (story text)
{type: "hsynopsis", ...} // Content type synopsis
{type: "hnotes", ...} // Content type comment / notes

Paragraphs are elements editable with SlateJS. They have children, which are text with marks:

{type: "p", ..., children: [
  {text: "Some text"},
  {text: "Some more text", bold: true},
  ...
]}

All the headers are paragraphs, and there are also some other paragraph types. All paragraph types:

• hact: Act header / separator
• hchapter: Chapter header / separator
• hscene: Scene header / separator
• hsynopsis: Scene header / separator, content type synopsis
• hnotes: Scene header / separator, content type notes / comment
• bookmark: a bookmark inserted to scene (shown in index)
• comment: shown green, not exported, not count to word counts
• missing: missing text (shown red, exported, reported as missing text)
• fill: a filler element, containing a number how many missing words it reports
• tags: A comma separated list of words (tags)
• p: plain text
• br: empty line

p and br are differented because they have a bit different rendering (p becomes <p>, br becomes <div>), and they are used as "split breaks" in exporting, so that exported content can process empty lines correctly.

[mkoskim]

• Once a document is saved and consists of text, the user should be able to clikc the "Loremize" button on the editor view.
• Upon clicking the "Loremize" button, a new unsaved document should be created.
• The newly created document should hold the same amount of data (same size visually) as the previous document with placeholder text for testing integrity purposes.

I haven't think this too much, but I think that generally the feature would work bit similar way as creating new file (selecting New from menu) or importing file (selecting Import from menu). This feature creates entirely new file, but it does it by using the previous file (file in the editor) as input. It is good, that when "loremizing" the file it forgets the file name and other things, so you don't accidentially save the loremized file over the original file.

New file menu item:

https://github.com/mkoskim/mawejs/blob/master/src/gui/app/app.js#L400

New file creation:

https://github.com/mkoskim/mawejs/blob/master/src/gui/app/context.js#L44

We might need "cmdCreateFromBuffer" so that we don't need to first convert the loremized tree to buffer, but we could set it as doc.

[mkoskim]

Aaaand one more thing.

Maybe create file "src/document/loremize.js". There could be function which takes doc as parameter, and returns a new loremized doc. That could be then tied to UI.

[mkoskim]

@ZacharyHandel BTW, I can do those command interface things myself, and put loremize.js in the folder, if you want. That command interface (giving access from lower tree to higher parts) is bit cryptic, and I need to replace it with Redux at some point.