feature request: allow customized pandoc args

[ickc]

Currently, in

jupytext/jupytext/pandoc.py

Line 86 in a1b02e3

"--from markdown --to ipynb -s --atx-headers --wrap=preserve --preserve-tabs",

,

jupytext/jupytext/pandoc.py

Line 106 in a1b02e3

"--from ipynb --to markdown -s --atx-headers --wrap=preserve --preserve-tabs",

,
pandoc args are hardcoded.

The feature request is to propose allowing customization.

E.g. see

pandoc --list-extensions=markdown
pandoc --list-extensions=ipynb

for customization for extensions alone (where one can fine-tune their own markdown variants so to speak.)

[mwouts]

Hello @ickc , sure we could do that. How would you like to use this customization?

Maybe we could create a new format option (would pandoc_extra_args be a good name?) for the md:pandoc format, and allow the user to specify a custom value either through jupytext --opt pandoc_extra_args='...', or through pandoc_extra_args in the juptyext.toml file?

Also, what should be the logic for building the conversion command? Do you think anyone would like to use other header styles than --atx-headers? I.e. do you prefer

 f"--from markdown --to ipynb {pandoc_custom_args or '-s --atx-headers --wrap=preserve --preserve-tabs'}"

or

 f"--from markdown --to ipynb -s --atx-headers --wrap=preserve --preserve-tabs {pandoc_extra_args}"

[mfhepp]

Not sure this is the right issue, but this could maybe help support using bibliographic references in notebooks, at least from the Markdown document to the .ipynb document - so that e.g. @foo2022b could be translated into the proper reference by using citeproc.

Also custom filters could be plugged in here.

NB: A quick way of implementing that would be supporting a Pandoc defaults YAML file if it exists - so if e.g. pandoc_jupytext.yaml exists, then the settings from there would be passed to Pandoc. That would be a very flexible and lightweight way of modifying the invocation parameters. Or support a pandoc_defaults property in jupytext.toml. So all the parsing would be delegated to Pandoc.

[mwouts]

Hi @mfhepp , I like your suggestion of adding a new pandoc_defaults (or maybe pandoc_args) option. Unfortunately at the moment I am not available to do so, but maybe you could consider contributing a PR for that? Many thanks!