Path to TeXpresso 1.0

[let-def]

Current objectives

This roadmap is provided for informational purposes only, it does not constitute a commitment or guarantee of any kind. The development is subject to the availability of time and resources, and the objectives outlined in this roadmap are subject to change or reordering as needed.

The roadmap is intended to provide a general outline of the project's goals and objectives.

• Phase 1: Alpha Proof-of-Concept (v0.1)
  • Develop a working alpha proof-of-concept on Linux and macOS, despite potential instability and build difficulties.
• Phase 2: Simplify Deployment (v0.2)
  • Fork XeTeX instead of the entire Tectonic distribution to simplify deployment.
  • Contribute the "bundle server" to the Tectonic project to reduce maintenance.
  • Support TeXlive distribution for broader compatibility.
  • Caveat: I will focus on Linux support first, macOS might be temporarily broken during development
• Phase 3: Remote UI Support (v0.3)
  • Add support for remote UI, enabling users to interact with the renderer from another process (e.g. vscode) and customize display.
  • Relevant issues: viewer feature suggestion: secondary view windows #59
• Phase 4: Focus on built-in UI (v0.4)
  • Revamp the built-in UI to improve user experience and provide a solid foundation for future development.
  • Improve errors and warnings reporting
  • Relevant issues: UI: continuous pages #15, UI: shortcuts cheatsheet #16, UI: Move with HJKL #17, UI: Display a spinner during initialization #28, viewer feature suggestion: secondary view windows #59
• Phase 5: Portability Improvement (v0.5)
  • Assess the feasibility of a Windows port, given limited resources.
    • Get rid of fork(2)
    • Find out how MikTeX works
  • Improve portability across different platforms.
  • Relevant issues: TeXpresso for Windows 11 #57
• Phase 6: Stability and Performance (v0.6)
  • Focus on improving stability, aiming for rock-solid long sessions.
  • Identify and fix main performance bottlenecks and memory hogs.
  • Relevant issues: Root file should exist or TeXpresso will just give up #12, bug: Emacs does not see buffer changes for .tex files which are below symlinks #58
• Phase 7: Build System and Integration (v0.7)
  • Rework the build system to provide pre-built macOS binaries (and Windows binaries if possible).
  • Automate build testing on main Linux distributions.
• Phase 8: First Public Release (v1.0)
  • Finalize the project and prepare for the first public release.
  • Investigate how to include the project in main Linux distributions.

No delivery times or deadlines are provided, and the project's progress may be affected by various factors, including but not limited to changes in priorities, availability of resources, and unforeseen technical challenges.

Future work

Once the project has a solid and stable foundation, consider the following LaTeX feature improvements:

• SyncTeX Precision: Enhance SyncTeX (Use sequence alignment to refine SyncTeX position #21) to improve precision of reported locations, especially when crossing a page.
• PDF Support: Expand PDF support, including cross-references (Clicking on references/links? #35) and drawing primitives (PDF graphics interpreter: handle shading, XObjects #7).
• Editor Integrations:
  • Vim: Integrate with VimTeX for a seamless Vim experience (see also texpresso.vim#4).
  • TeXlab: Collaborate with TeXlab to provide a rich integration if they express interest (see Language Server Protocol support #36).
• Semantic Information: Expose more semantic information, including:
  • Jump to definition
  • Contextual completion
  • Improved error location reporting
    These would be good to have but are unlikely to be feasible.

[gasche]

For the record, an uninformed take on these objectives.

1. Currently texpresso is a "small" project that does something very impressive in a way that, surprisingly, could be called low-effort -- a lot of thinking went into it but the codebase is reasonably small. It could remain a "small project", just better, or it could grow into a "big project" that tries to cover a lot more ground -- with the same overall scope -- at the cost of vastly more development and maintenance work. From the objectives it is not clear to me whether you wish to remain "small", have decided to go "big", or haven't thought explicitly about making a decision there. I think that it would make sense to be explicit about this aspect, and check that the roadmap is in line with your decision.

2. If I was doing the work, I would drop any plan of Windows support besides WSL. I would guess that any kind of multi-process coordination work (with fork, no less!) is probably going to be super-tedious to write in a portable way. And I don't know of many users doing LaTeX seriously on Windows -- the LaTeX users I know are in academia, and they got the word that Linux or OSX are better work environments. But you do you -- maybe you find the work of supporting Windows enjoyable?

3. From a distance, reimplementing all the bells and whistles of a PDF viewer does not sound very reasonable -- at least for a "small" project. Is there a way that you could instead push logic into an existing viewer (for example mupdf, that you already interact with), so that you don't have to reimplement a lot of tedious stuff yourself? (For example: jumping to references, etc.) I don't understand what is the interface between TeXpresso and a PDF viewer, and how unorthodox it would feel to existing PDF viewer codebases. Maybe it is possible to delegate to external viewers only a portion of what we might want to do, for example to use an external viewer as a "secondary view pane" and use an RPC protocol to ask it to "jump your view to reference foo".

[let-def]

TLDR:

1. I want to keep the project small, and the first few steps of the roadmap aim to make it smaller.
2. One of the change I have in mind might make Windows support possible, but this is not guaranteed.
3. It is too early to make a definitive decision on the UI, but "remote UI" is an intermediate step that helps all the scenarios.

My intention is to keep the project's scope small. I am aiming to minimize the maintenance work. Replacing the full Tectonic fork with XeTeX will simplify the codebase and remove the dependency on the Rust toolchain. The compatibility with TeXlive is a plus, it should come at almost no cost as most of the required code is already in place.

Regarding Windows support, it is driven by an alternative snapshotting strategy I have in mind that might also benefit macOS, which has limited fork(2) support. If it works, Windows support will be a by-product, but it is not an explicit goal.

On the topic of PDF viewers, I think that most of them are not designed for incremental updates. The asynchronous interactions between the editor, the driver, and the TeX worker make it challenging to leverage existing PDF software. Also, XeTeX produces XDV, a dialect of DVI that is not supported by document viewers as far as I can tell. Even if we could integrate with an existing viewer, significant plumbing work will be needed. Maybe forking a MuPDF viewer is an option.
Note that the difficulty with implementing the "jump to reference" feature is not in the UI, it is really about extracting the right information from TeX output.

Instead, I want to pursue the "remote UI" approach. It requires separating document preparation and presentation. This clean boundary enables exploring different paths, such as:

• Exposing the incremental document as a C library with a well-defined API; this will be beneficial whatever we decide to do.
• Exposing it as an IPC protocol, for instance to develop high-level plugins (e.g. editor plugins with graphic integration, or a separate Python UI.
• Improving the built-in UI; I have ideas to simplify the current code, which will enable richer features and interactions while keeping the code-base small.

"Remote" might be a bit misleading in the sense that it doesn't have to be separate processes, but I think the right design is to see TeXpresso as an independent actor with which the viewer communicate using well defined messages. A clean API should not try to hide the asynchronous nature of this work, which is something the current UI integration is doing (by polling the TeX process in a single-threaded event loop).