Support Math — somehow (general discussion of approaches and status) #220
Comments
|
Notes from the first failed attempt: I tried translating KaTeX with castl. The translation "succeeded" (using KaTeX v0.7.1 from git and castl 1.2.4), with commands roughly as below: However, trying to run the following Lua script afterwards with Lua 5.2: failed with a Lua error: (Reported on castl project as PaulBernier/castl#18. TODO: try some next translator, maybe colony.) |
|
After some more looking around, I want to try my luck with ASCIIMath. It seems small enough, that even in case js2lua converters won't help, a manual port seems feasible. It does not implement Full Parity With TeX™; but as we all know too well I think, TeX is Turing-complete, and full parity is not necessarily even desirable. Anyway, even if I succeed, in Future™ someone can always just ditch the ASCIIMath and/or incrementally replace it with something More Powerful™ or whatever. Now, with that said, there's one thing I now realized I'm not really clear on. Namely, what would be the best format to output from a math module for SILE to consume? ASCIIMath I think produces MathML; KaTeX claims to emit HTML+CSS; MathJaX is I believe super-duper-dynamic-adaptive-autodetect and can produce MathML, SVG, possibly images, and I don't even know what else. What would SILE like best? Imagine we're free of constraints; would it be PDF? SVG? Lua rendering code? glyphs + coords + scales? PostScript? glues penalties and some other weird stuff like this? (not even sure if it's there for math). On the other hand: is there/are there some simple format(s) (hopefully, vector) that SILE could reasonably accept for a reasonable first prototype/PoC? As a side note for future: in case of failure, I think one more direction to research would be troff's eqn processor. |
It doesn't have Anyway, good luck. It is hard to live without even basic math. |
|
PDF would seem to make the most sense. That way the output can be directly embeded without much chance for scale or other interpretive errors. An SVG route would be an entirely different pipeline and introduce some other pain points, but if there was a Math→SVG processor we might be able to bolt that on entirely outside of Lua. |
|
@suhr Did you ever try groff's eqn? Do you think it's currently more powerful or less, compared to ASCIIMath? If PDF sounds best, then I think eqn sounds like more bang for the buck for a first shot. With various caveats, first among them GPL I suppose (or alternatively GPL-incompatible Lucene license for Plan9's troff+eqn; not sure about PDF support there too). Or do you know by chance of someone already trying to integrate eqn with SILE and stumbling upon some problems? EDIT: Hmm; it seems groff emits PDF by means of a sub-tool named "gropdf", which seems to be written in Perl. I'm not really convinced requiring Perl is a good match for SILE. On the other hand, for the time being it could be treated as a "plugin" to SILE, and as such not part of it, so Perl could be acceptable then. Still, now it's not clear to me how would one approach "merging" PDF into SILE as input. It seems to me that PDF describes whole pages (or am I not right?). But equations are usually just small fragments on a page. Can SILE cut fragments of PDFs? Do you maybe know of some already preexisting mechanisms for something like that, of which I'm not aware yet? @alerque Do you know if there's maybe a PDF parser or something in SILE? EDIT 2: @alerque Hm, but there seems to be already some machinery for SVG in SILE, no? |
|
Prototype translation of ASCIIMathML.js to Lua is done and tested. I used the very cool castl translator. Also, given that it's not my first port, I must shout out, that comprehensive end-to-end tests do really boost porting speed awesomely! TODO: test if svgmath will work reasonably well to translate the resulting MathML to SVG. It seems quite a small and readable MIT codebase, so should be feasible to hand-port to Lua if it's working well. I imagine it should be possible to migrate the SVG emitter to a PDF-or-whatever emitter more fitting for SILE later, should the PoC prove successful. |
|
@akavel SILE already has a way to embed PDFs inside PDFs. Use the Yes there is also rudimentary SVG support, but this isn't really up to snuff yet and involves being able to render whatever is in the SVG and convert it to a post script representation that can be embedded in the PDF. This is nowhere near as reliable as just nesting PDF content. It works alright for some things, but doesn't understand a lot of SVG instructions yet. |
|
@alerque Do you know if it's possible to create a PDF with "tiny paper size", so that it would have size of exactly the bounding box of an equation (i.e. 2cm x 1cm, or something like that)? Or does |
|
Embeding a PDF as an image does nothing in the way of cropping or finding contents, it just uses the bounding box and inserts that at the requested overall width, hence scaling the source's paper size to the embeded size. Obviously if you're trying to bold on some other render you might end up with an awkward mating like this, but ideally of course a native Lua parser would transform the formula to native drawables that could be passed to the output engine (which at this point is basically always the PDF output engine but keep in mind that isn't supported to be a hard coded limitation). |
|
Ok, did a quick test with groff and indeed this direction seems to (kinda) work already? Main initial caveats:
For anyone interested, here comes a quick & rough transcript of my session. Note I'm using Nix package manager:
|
|
For the record, a "more advanced" groff equation worked as well:
|
What do you mean by "original" here? The baseline in the SILE PDF should be straight forward enough. The trick is going to be leaning where the baseline of an equation PDF is in the event of an inline equation such as the ones in your examples. Once you do learn that you can offset the image in SILE using |
|
(@alerque Yes, I meant exactly what you mean by "learning where the baseline of an equation PDF is".) Personally, at this point I still plan to pursue the path of: ASCIIMath → MathML → SVG → SILE. I believe it's more promising in the long run. That said, groff (specifically, eqn + pic) seems totally more useful in the short run, especially given that their language is much more powerful and expressive than ASCIIMath (I imagine commutative diagrams should be totally doable with eqn or eqn+pic). But as far as my current plans go, I'm leaving perfecting of the groff path as an... cue ominous thunder... "excercise for the reader". |
|
Cool, SVGMath seems to work! Below is a quick screenshot of: after passing through: (Note: I had to add The screenshot:
It complained about some missing metrics and whatnot, so I suspect this could look better with some caring hand. (Still positively surprised super much that it actually worked, and so well so soon!) |
|
I don't have much to add to this but I'm really happy it's being worked on. The baseline/leading issues will be quite easy to fix when then time comes. |
|
@simoncozens How about the bounding box? If groff emitted a big, white page with a (comparably) small equation in some quasi-random place on it, would you find it easy to extract just the "dirty" area? If so, it would make the task much easier on the groff side, I think — it should be just deleting all page decorations, and voilà, the thing's done? |
|
@akavel I don't think SILE should be responsible for parsing PDF data. It really isn't setup for that. It knows how to write it and it can take a chunk of PDF data and embed it somewhere, but parsing through the PDF data and doing a virtual rendering to find the bounds of actual content in it is more than a little outside the purview of the typesetting engine. That being said it may not have to be |
|
(@alerque Sorry, but can you please remove my "assignee" status from this issue? I don't want this to start feeling like work to me; I'm doing this just out of fun and because I want to help and like this project. A subjectively perceived connotation of "responsibility" and "obligation" coming from the assignment status unfortunately heavily impacts on my fun and is counter-productive for me working on this. Also, I don't want to make anyone feel like "this issue is occupied so I won't waste my time contributing to it"; I'm totally ok with someone else even finishing it before me and closing the issue (I don't lack things to do). Finally, I never promised I'll complete this, so I don't want anybody to think so by accident. I just fancy to have a stupid plan to try my luck at this, with 100% undefined schedule. As also shown by a ~1.5yr gap between my first and second comment in this thread. TIA! I'd do this myself, but can't find how.) |
|
@akavel Of course. I didn't intend it in the sense of responsibility at all only that because you've got code kicking around in this department it would make sense to coordinate with you before launching into this one. Speaking of which do you have a branch somewhere with your work in progress? If it was posted and linked from here that would give somebody else the chance to come along and extend or clean it up. |
|
@alerque Thanks! Re: branch link — sure, good idea. Summarizing the currently explored approach:
Recapping possible alternatives I thought of, with some pros & cons I'm aware of:
All/some of the above alternatives could potentially peacefully coexist so that they can be played to their relative strengths. |
|
The Lua port of SVGMath is now giving results, and very similar ones to what the original Python version gave. On the available test data/examples (*.mml), some have minor/medium spacing issues Example Lua results (testdata/*.out.svg):Reference Python results (testdata/*.svg):At this point I'd like to ask for help integrating this as an experimental plugin/feature into SILE. Can you possibly give me some guidance on how to start? I'd be very grateful for hints, would considerably speed this up for me. Please note, that I have no practical knowledge/experience of SILE, other than reading the manual. |
|
Uh, oh; do I see correctly that SVG import seems not yet working as of current state of the SILE repo? |
|
@alerque as to
Haven't tried to do anything more advanced with it yet. (Please note I'm kinda early-intermediate with Nix, definitely not an expert.) EDIT: Uhh, this will actually probably still build the 0.11.1 version fetched from github. Need to do some tweaks still to load from local code; sorry, as I said above, I'm still learning Nix :/ I think the next thing to do is to change the |
|
Don't forget to update the hash, otherwise nix might pull a wrong source. |
|
Anyway, you want this if you want to build from master: |
|
@akavel When you get that working I'd be happy to accept a PR to put it in the project source repo. I'm coming up with uses for it already, starting with the SILE website. Building the examples in CI is great but it produces a chicken and egg problem because the CI runner has the last tagged version. Or if there is an official way to setup the sile package in nixpkgs to have a HEAD build option (like Homebrew has) that would be great too. @suhr happen to know anything about that? |
|
I would probably just add |
|
@OlivierNicole hmh; so, I tried, and generally, I managed to fairly quickly get able to pass the asciimath->mathml translator's output to the
A good (most probably ultimate) reference on various MathML outputs that ASCIIMath can produce can be seen at: https://github.com/akavel/silemath/blob/72102bde232f8d3828ad5fee67a3b268dccd872a/asciimath/testall.lua So, the difficult question now: is there a chance you might consider trying to add support for those elements to the |
|
@OlivierNicole Alternatively, if I wanted to try my luck at attempting to add support for those elements, would you possibly fancy giving me some hints as to where in the code could I even start looking into this? I mean especially the math layouting engine, I think... |
|
@akavel Regarding Accents are not supported yet, mainly because it requires to support the construction arbitrarily stretchable glyphs, something conceptually not complicated and all the pieces are there, I just did not prioritize that in the window of time when I was working on this. Limits are supported. However, I did a mistake in my interpretation of the MathML standard, so limits are managed by the I have unfortunately little time these days to work on this side project, but I can try to fix this and support big operator limits through
I believe the math package documentation in the manual (at the end of the section) gives a very brief overview of how to implement support for new MathML tags. Edit.: |
|
@OlivierNicole thanks! I now see the lines near the end in packages/math.lua describing If you could possibly do the As to the stretchable glyphs/accents, if you're not planning to work on that for now, would it be possible you'd try to outline the thought process and general idea how to try approaching this, as much as you could? Any hints here could be helpful! ❤️ I admit I currently would have no idea how to start neither conceptually, nor where to find the "pieces that are there"... if you wouldn't mind helping me with some hints, maybe I could try experimenting with that! Thanks for the outline on |
|
Tags munder, mover and munderover implemented. See #1240.
Yes, as long as stretchable operators are part of the equation (pun intended), it should work. |
When I said “the pieces are there”, I was thinking about the glyphs that were drawn by the OpenType font designers. In the end, the only thing that the math package has to do is positioning these glyphs at the right positions and give them the right sizes to typeset a formula. Stretchable glyphs are a bit trickier. They can be done in two ways. First, some glyphs have several variants of increasing sizes: vertical variants (like vertical braces or parentheses) or horizontal variants (like arrows or horizontal braces). When the biggest variant is still too short, arbitrarily long versions can be constructed by overlapping base components of the glyph. The component glyphs, as well as some numeric parameters governing the scaffolding construction, are provided by the font in the MathGlyphConstruction table (see OpenType spec). The parsing of OpenType MATH tables into Lua tables has already been implemented by @harrysummer, so what's left is actually constructing stretchable glyphs, when needed. I implemented the easy part: for delimiter glyphs, using the vertical variant closest to the target height. I didn't implement anything for horizontal variants, however. OpenType Math Illuminated is probably also a useful reference for the parameters governing the positioning of glyphs. |
|
@OlivierNicole thank you so much for the amazing description and references ❤️, this will most certainly be helpful to anyone interested in trying to implement this! (which might or might not be me - it makes no sense for me to give any promises here, given my poor track record with delivering stuff related to this issue...) |
|
This is what a glyph construction table (in Rust) looks like: let integral = MathGlyphConstruction {
glyphAssembly: Offset16::to(GlyphAssembly {
italicsCorrection: MathValueRecord::new(0),
partRecords: vec![
GlyphPartRecord {
glyphID: 2048,
startConnectorLength: 0,
endConnectorLength: 212,
fullAdvance: 893,
partFlags: 0,
},
GlyphPartRecord {
glyphID: 2075,
startConnectorLength: 893,
endConnectorLength: 893,
fullAdvance: 893,
partFlags: 1,
},
GlyphPartRecord {
glyphID: 2047,
startConnectorLength: 212,
endConnectorLength: 0,
fullAdvance: 893,
partFlags: 0,
},
],
}),
mathGlyphVariantRecord: vec![],
};For the integral sign, there are no math variants, but there is a glyph assembly table, which tells you how to build a stretchable integral sign: start by picking GID 2048 (integralbottom), and then, at anywhere from (top - 212 units) to the top of that glyph, you can place a connector (GID 2075, integral extension). This can be repeated if necessary ( |
|
Thanks @simoncozens for this example, it's actually quite illuminating! |
That is quite odd, I’d expect it to be the other way around for integral signs, what font is that? |
|
Libertinus. |
|
Either your version of the font is broken or the code dumping the MATH table, this is what I see in TTX dump: |
|
Sorry, not Libertinus, LinLibertine_R.otf. |
|
I see, makes sense as that fonts MATH table was not that usable. |
|
This should be closed. |
I'm aware MathJax is listed on the wiki as "up-for-grabs", but I just wanted to start by asking: have you maybe tried pushing some stuff from the MathJax's unpacked/ subdir through some js-to-lua translator (or other)?
I'm on Windows, so personally, before I maybe start trying to contribute, I have to first manage to compile the stuff (yes, I've seen the issue tracking this)...
(By the way, I got here by accidentally watching the 2015 FOSDEM video of yours; very entertaining and interesting :) although I've apparently already +1'ed your project at some point, only this video made me aware of many things about it (both feature demos, architecture, and how it relates and improves over *TeX) and thus more interested.)
edit: A few years later, after some prototyping and experiments, I'm expanding this initial post with more detailed current status of the project. The path I've eventually picked to try and implement this is based on ASCIIMath, not MathJax.
Current TODO tasks
Roughly in priority/painfulness order. I'd be especially grateful if someone could help with the ones marked "HELP NEEDED", I don't know how to proceed with those unfortunately :(. Jumping in to help with any task is enthusiastically encouraged!
Translate ASCIIMathML.js from JS to Lua(done and tested)Translate SVGMath from Python to Lua(done with help of https://github.com/akavel/pylua based on https://github.com/kevinw/pylua)Write SVG parser and renderer for SILE supporting SVGs generated by SVGMath(done)Integrate the tools into a pipeline callable from SILE with an(done)\asciimathdirectiveThe(done)<text>nodes in SVGs generated by svgmath havetext-anchor="middle", so their horizontal position should be tweaked, but I couldn't yet find out how to do that properly (moving them by-shape[1].width/2didn't work for me for now, it broke things more than fixed). This would be important fix to the current buggy layout. This should be verified by rendering some correct SVG (e.g. https://github.com/akavel/svgmath/blob/master/testdata/test13.svg) through the final part of the pipeline, that is,\mathsvg.Use TTF font paths from SILE instead of hard-coded paths in svgmath/config.luasile sample.silshould work now (after recent commit)\mathsvg.\mathsvgfirst, as mentioned in the task above{and}and\in math expressions$ ...math... $or\[ ...math... \]could be used?Current example result
The text was updated successfully, but these errors were encountered: