Some fixes we implemented

[KomodoGal]

Hey o/
We've been loving Ghostwriter for making better reports faster :) but it had a few features missing and needed a small fix.
Since we've modified it quite a bit our version is a bit outdated so if these have already been fixed or added then sorry.

Fix: correct link behavior

When using normal Ghostwriter links which are inserted using TinyMCE as clickable links came out both as a link and text:

This can be fixed by adding a return to the end of the "hyperlink" section of "_write_xml" in "ghostwriter/modules/reportwriter.py":

Links then come out correctly.

Fix: use lists longer than 2 in report template

This can be done by adding a jinja2 filter, to see how to do that, see #284.
The jinja2 filter we added is:

def get_item(lst,index):
    return lst[index]

and is used like this:
{{ "cat,dog,parrot,kangaroo".split(',') | get_item(3) }} will print "kangaroo".

Feature: add .docm support

We use word document macros to help with the reporting process since Ghostwriter is used to get the first draft which is revised outside of it.
We add the same macros to each document so we wanted our templates to include them.
This was a bit problematic and the solution is not elegant, as one of the libraries Ghostwriter uses, "python-docx", does not support it.
However, there is a pull request which adds this feature: python-openxml/python-docx#716
I integrated the pull request using the Dockerfile by adding these lines:

COPY ./compose/production/django/__init__.py /usr/local/lib/python3.8/site-packages/docx/__init__.py
COPY ./compose/production/django/api.py /usr/local/lib/python3.8/site-packages/docx/api.py
COPY ./compose/production/django/constants.py /usr/local/lib/python3.8/site-packages/docx/opc/constants.py

I also added the updated library files to the directory, they are attached as text files, change them to python files :)
api.py.txt
constants.py.txt
init.py.txt

Ghostwriter also needs to be updated, this is also a bit janky, but works. I changed the "Get" method of the class "GenerateReportDOCX" in ghostwriter/reporting/views.py:

# Template available and passes linting checks, so proceed with generation
            engine = reportwriter.Reportwriter(self.object, template_loc)
            docx = engine.generate_word_docx()
            logger.info(report_template.document)
            response = HttpResponse(
                content_type="application/vnd.openxmlformats-officedocument.wordprocessingml.document"
            )

            # Komodo custom :)
            if report_template.document.name.split('.')[-1] == 'docm':
                response["Content-Disposition"] = f'attachment; filename="{report_name}.docm"'
            else:
                response["Content-Disposition"] = f'attachment; filename="{report_name}.docx"'
            docx.save(response)

We've also rearranged a lot of items, and made fully extendible fields using whitecards, a whole lot of Javascript and a fitting template.

Best,
Ellen and the Komodo team 😄

[KomodoGal]

Oh, and I almost forgot, we also fixed the table sorting by date in the web interface using the sugar parser shown here: https://mottie.github.io/tablesorter/docs/example-parsers-dates.html
We added this to ghostwriter/templates/base_generic.html:

<!-- Komodo custom sugar import ;D -->
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/sugar/2.0.6/sugar.min.js" integrity="sha512-xAmBlm+rRB4bQ/BVDkK88a5OjFNgrOHXTi/8ssY7Msg+3PSj+QJj2BScyEVqm0FynZU3Z0h4Rdaxn4BuM/QimQ==" crossorigin="anonymous" referrerpolicy="no-referrer"></script> 

We also added this to the end, with all the scripts:

 <!-- Sugar parser from tablesorter for Komodo -->
        <script>
            /*! Parser: dates - updated 5/24/2017 (v2.28.11) */
            /* Extract dates using popular natural language date parsers */
            /*jshint jquery:true */
            /*global Sugar*/
            ;(function($) {
                'use strict';

                /*! Sugar (https://sugarjs.com/docs/#/DateParsing) */
                /* demo: http://jsfiddle.net/Mottie/7z0ss5xn/ */
                $.tablesorter.addParser({
                        id: 'sugar',
                        is: function() {
                                return false;
                        },
                        format: function(s) {
                                // Add support for sugar v2.0+
                                var create = Date.create || Sugar.Date.create,
                                        date = create ? create(s) : s ? new Date(s) : s;
                                return date instanceof Date && isFinite(date) ? date.getTime() : s;
                        },
                        type: 'numeric'
                });

            })(jQuery);
        </script>

Then you can add the class "sorter-sugar" to any table with a dates column and the sorting is fixed.

[chrismaddalena]

Hey @KomodoGuy, thanks for posting about your changes!

What do you do in the WYSIWYG editor to get the hyperlinks to output like that? That's an easy fix, but I'm still curious how it happens for you. I haven't heard of it before and can't reproduce it myself.

When I make a hyperlink it looks like I expect in Word:

For the second thing, what do you mean by "use lists longer than 2 in report template" here? The issue you mentioned was for comments. Are you encountering an issue with lists?

I'll look into support for docm files and the sugar parser. For the latter, I assume this means you're having an issue with the current TableSorter parser and your date format or locale. I'm unaware of any issues present for English, but I am aware the parser does not understand the date strings and abbreviations for other languages. If this parser can handle that, it'd be a great inclusion to fix a longstanding issue.

[KomodoGal]

Hey!
I'll go through each of the changes I mentioned:

1. Regarding the links, I checked, this is something that happens to our Ghostwriter, this is probably because of some other modification we made to it, so you can ignore it :)
2. Regarding the lists, I have attached 2 templates, one which does not work, and the other with the work around:
   test template - Fixed.docx
   test template.docx
   The template fails at linting with this error:
   

I don't think this is something most Ghostwriter users will come up against, but when I try to extend Ghostwriter's abilities with less conventional templates and whitecards, this is something that I get quite a lot. I am planning to modify Ghostwriter's server side more rather than client-side and in the template to fit our needs.
FYI, our Ghostwriter's basic cell will be the report rather than the project, as we use it as a draft report writer.

3. Regarding the sugar parser, in basic Ghostwriter when we try to sort reports by creation date, to see the latest ones it is sorted based on the day in the month, rather than the month and the day:
   

By using the sugar parser, we can fix this:

[chrismaddalena]

Thank you for the explanation. The templates are interesting. If you have other examples, I can look into how to make things easier. You can do a lot with Jinja2, but more complicated filtering and looping can get messy, making the template more challenging to maintain. Adding your get_item filter is easy for this.

[chrismaddalena]

Hey @KomodoGuy,

I'm working on an update for this week. I've added the get_item filter for you. I also implemented the Sugar parser for the date columns. Sugar should address issues with the date columns on pages that list single dates (report list, domain library). If it works for you on those columns, I hope it helps with other international dates. It would be a nice improvement to get it working with date ranges.

I set all columns with date ranges to use a standard ddmmyyyy format so TableSorter's parser knows how to handle the date range. That breaks when certain non-English month names appear. It's been an annoying problem that I've never quite resolved. I can add another version of TableSorter's sugar parser with the date range code mentioned here (https://sugarjs.com/releases/1.3.0/) as a second parser called sugar-range.

The docm support will take a little longer. In the meantime, you can use docx files with macros, which make their way into the final document. It will be nice to support docm as an option. Other PRs would be nice to have, too, so it might make sense to create a custom fork of python-docx for Ghostwriter.