Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: provide Mobi files #2319

Merged
merged 17 commits into from
Mar 5, 2025
12 changes: 10 additions & 2 deletions .github/workflows/auto-updates.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,12 @@ jobs:
- name: Install requirements
run: python -m pip install -r requirements.txt

- name: Install kindlegen
run: |
mkdir -p ~/.local/bin
wget https://raw.githubusercontent.com/BoboTiG/ebook-reader-dict/refs/heads/kindlegen-backup/kindlegen_linux/kindlegen -O ~/.local/bin/kindlegen
chmod a+x ~/.local/bin/kindlegen

- name: Download the latest Wiktionary dump
run: python -Wd -m wikidict ${{ matrix.locale }} --download

Expand All @@ -66,11 +72,13 @@ jobs:
tag_name: ${{ matrix.locale }}
body_path: release.txt
files: |
data/${{ matrix.locale }}/dicthtml-${{ matrix.locale }}-${{ matrix.locale }}*.zip
data/${{ matrix.locale }}/dicthtml-${{ matrix.locale }}-${{ matrix.locale }}*.zip.sha256
data/${{ matrix.locale }}/dict-${{ matrix.locale }}-${{ matrix.locale }}*.df.bz2
data/${{ matrix.locale }}/dict-${{ matrix.locale }}-${{ matrix.locale }}*.df.bz2.sha256
data/${{ matrix.locale }}/dict-${{ matrix.locale }}-${{ matrix.locale }}*.mobi
data/${{ matrix.locale }}/dict-${{ matrix.locale }}-${{ matrix.locale }}*.mobi.sha256
data/${{ matrix.locale }}/dict-${{ matrix.locale }}-${{ matrix.locale }}*.zip
data/${{ matrix.locale }}/dict-${{ matrix.locale }}-${{ matrix.locale }}*.zip.sha256
data/${{ matrix.locale }}/dicthtml-${{ matrix.locale }}-${{ matrix.locale }}*.zip
data/${{ matrix.locale }}/dicthtml-${{ matrix.locale }}-${{ matrix.locale }}*.zip.sha256
data/${{ matrix.locale }}/dictorg-${{ matrix.locale }}-${{ matrix.locale }}*.zip
data/${{ matrix.locale }}/dictorg-${{ matrix.locale }}-${{ matrix.locale }}*.zip.sha256
6 changes: 6 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ jobs:
- name: Quality checks, and linters
run: ./check.sh

- name: Install kindlegen
run: |
mkdir -p ~/.local/bin
wget https://raw.githubusercontent.com/BoboTiG/ebook-reader-dict/refs/heads/kindlegen-backup/kindlegen_linux/kindlegen -O ~/.local/bin/kindlegen
chmod a+x ~/.local/bin/kindlegen

- name: Unit tests
run: python -Wd -m pytest tests --doctest-modules wikidict

Expand Down
138 changes: 6 additions & 132 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,13 @@
*.bz2
*.df
*.json
*.mobi
*.xml
*.zip
dict-data.*
sections.txt
templates.txt
/data
/data/
tests/data/*/dict-data.*
tests/data/*/INSTALL.txt
tests/data/*/words.*
Expand All @@ -20,137 +21,10 @@ apicache-py3
pywikibot.lwp
throttle.ctrl

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.DS_Store
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
# Python stuff
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
__pycache__/
.mypy_cache/
.pytest_cache/
.ruff_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv*/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# Intellij IDEA
.idea
venv/
Binary file added cover.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
34 changes: 22 additions & 12 deletions tests/test_3_convert.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,18 @@
Export Wiktionnaire : 2020-12-17

Version complète :
- [Kobo](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dicthtml-fr-fr.zip)
- [StarDict](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr.zip)
- [DictFile](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr.df.bz2)
- [DICT.org](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dictorg-fr-fr.zip)
- [Kindle](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr.mobi)
- [Kobo](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dicthtml-fr-fr.zip)
- [StarDict](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr.zip)

Version sans étymologies :
- [Kobo](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dicthtml-fr-fr-noetym.zip)
- [StarDict](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr-noetym.zip)
- [DictFile](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr-noetym.df.bz2)
- [DICT.org](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dictorg-fr-fr-noetym.zip)
- [Kindle](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr-noetym.mobi)
- [Kobo](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dicthtml-fr-fr-noetym.zip)
- [StarDict](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr-noetym.zip)

Mis à jour le"""

Expand Down Expand Up @@ -82,12 +84,6 @@ def test_simple() -> None:
assert (output_dir / "dict-fr-fr-noetym.df.bz2").is_file()
assert (output_dir / f"dict-fr-fr-noetym.df.bz2.{ASSET_CHECKSUM_ALGO}").is_file()

# StarDict
assert (output_dir / "dict-fr-fr.zip").is_file()
assert (output_dir / f"dict-fr-fr.zip.{ASSET_CHECKSUM_ALGO}").is_file()
assert (output_dir / "dict-fr-fr-noetym.zip").is_file()
assert (output_dir / f"dict-fr-fr-noetym.zip.{ASSET_CHECKSUM_ALGO}").is_file()

# DICT.org
assert (output_dir / "dictorg-fr-fr.zip").is_file()
assert (output_dir / f"dictorg-fr-fr.zip.{ASSET_CHECKSUM_ALGO}").is_file()
Expand All @@ -101,6 +97,18 @@ def test_simple() -> None:
assert dicthtml.is_file()
assert (output_dir / f"dicthtml-fr-fr.zip.{ASSET_CHECKSUM_ALGO}").is_file()

# Mobi
assert (output_dir / "dict-fr-fr.mobi").is_file()
assert (output_dir / f"dict-fr-fr.mobi.{ASSET_CHECKSUM_ALGO}").is_file()
assert (output_dir / "dict-fr-fr-noetym.mobi").is_file()
assert (output_dir / f"dict-fr-fr-noetym.mobi.{ASSET_CHECKSUM_ALGO}").is_file()

# StarDict
assert (output_dir / "dict-fr-fr.zip").is_file()
assert (output_dir / f"dict-fr-fr.zip.{ASSET_CHECKSUM_ALGO}").is_file()
assert (output_dir / "dict-fr-fr-noetym.zip").is_file()
assert (output_dir / f"dict-fr-fr-noetym.zip.{ASSET_CHECKSUM_ALGO}").is_file()

# Check the Kobo ZIP content
with ZipFile(dicthtml) as fh:
expected = [
Expand Down Expand Up @@ -187,12 +195,14 @@ def test_generate_primary_dict(formatter: type[convert.BaseFormat], filename: st
@pytest.mark.parametrize(
"formatter, filename, include_etymology",
[
(convert.StarDictFormat, "dict-fr-fr.zip", True),
(convert.StarDictFormat, "dict-fr-fr-noetym.zip", False),
(convert.BZ2DictFileFormat, "dict-fr-fr.df.bz2", True),
(convert.BZ2DictFileFormat, "dict-fr-fr-noetym.df.bz2", False),
(convert.DictOrgFormat, "dictorg-fr-fr.zip", True),
(convert.DictOrgFormat, "dictorg-fr-fr-noetym.zip", False),
(convert.MobiFormat, "dict-fr-fr.mobi", True),
(convert.MobiFormat, "dict-fr-fr-noetym.mobi", False),
(convert.StarDictFormat, "dict-fr-fr.zip", True),
(convert.StarDictFormat, "dict-fr-fr-noetym.zip", False),
],
)
@pytest.mark.dependency(
Expand Down
10 changes: 6 additions & 4 deletions tests/test_4_release.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,18 @@
Export Wiktionnaire : 2020-02-20

Version complète :
- [Kobo](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dicthtml-fr-fr.zip) (dicthtml-fr-fr.zip)
- [StarDict](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr.zip) (dict-fr-fr.zip)
- [DictFile](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr.df.bz2) (dict-fr-fr.df.bz2)
- [DICT.org](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dictorg-fr-fr.zip) (dictorg-fr-fr.zip)
- [Kindle](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr.mobi) (dict-fr-fr.mobi)
- [Kobo](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dicthtml-fr-fr.zip) (dicthtml-fr-fr.zip)
- [StarDict](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr.zip) (dict-fr-fr.zip)

Version sans étymologies :
- [Kobo](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dicthtml-fr-fr-noetym.zip) (dicthtml-fr-fr-noetym.zip)
- [StarDict](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr-noetym.zip) (dict-fr-fr-noetym.zip)
- [DictFile](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr-noetym.df.bz2) (dict-fr-fr-noetym.df.bz2)
- [DICT.org](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dictorg-fr-fr-noetym.zip) (dictorg-fr-fr-noetym.zip)
- [Kindle](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr-noetym.mobi) (dict-fr-fr-noetym.mobi)
- [Kobo](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dicthtml-fr-fr-noetym.zip) (dicthtml-fr-fr-noetym.zip)
- [StarDict](https://github.com/BoboTiG/ebook-reader-dict/releases/download/fr/dict-fr-fr-noetym.zip) (dict-fr-fr-noetym.zip)

<sub>Mis à jour le """

Expand Down
5 changes: 3 additions & 2 deletions wikidict/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,10 @@
--workers=N Set the number of multiprocessing workers,
defaults to the number of CPU in the system.
--convert Convert rendered data to working dictionaries into several files:
- "data/$LOCALE/dicthtml-$LOCALE-$LOCALE.zip": Kobo format.
- "data/$LOCALE/dict-$LOCALE-$LOCALE.df.bz2": DictFile format.
- "data/$LOCALE/dict-$LOCALE-$LOCALE.mobi": Kindle format.
- "data/$LOCALE/dict-$LOCALE-$LOCALE.zip": StarDict format.
- "data/$LOCALE/dicthtml-$LOCALE-$LOCALE.zip": Kobo format.
- "data/$LOCALE/dictorg-$LOCALE-$LOCALE.zip": DICT.org format.
--find-templates DEBUG: Find all templates in use.
--check-words Render words, then compare with the rendering done on the Wiktionary to catch errors.
Expand All @@ -37,7 +38,7 @@
--gen-dict=WORDS DEBUG: Generate dictionary for specific words. Pass multiple words
separated with a comma: WORD1,WORD2,WORD3,...
The generated filename can be tweaked via the --output=FILENAME argument.
--format=FORMAT Format can be "kobo", "stardict", "dictorg" [default: kobo]
--format=FORMAT Format can be dictorg, kobo, mobi, stardict [default: kobo]
--release DEV: Generate the description of a GitHub release.

If no argument given, --download, --parse, --render, and --convert will be done automatically.
Expand Down
1 change: 1 addition & 0 deletions wikidict/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
DOWNLOAD_URL_DICTFILE = f"{GH_REPOS}/releases/download/{{0}}/dict-{{0}}-{{0}}{{1}}.df.bz2"
DOWNLOAD_URL_DICTORGFILE = f"{GH_REPOS}/releases/download/{{0}}/dictorg-{{0}}-{{0}}{{1}}.zip"
DOWNLOAD_URL_KOBO = f"{GH_REPOS}/releases/download/{{0}}/dicthtml-{{0}}-{{0}}{{1}}.zip"
DOWNLOAD_URL_MOBI = f"{GH_REPOS}/releases/download/{{0}}/dict-{{0}}-{{0}}{{1}}.mobi"
DOWNLOAD_URL_STARDICT = f"{GH_REPOS}/releases/download/{{0}}/dict-{{0}}-{{0}}{{1}}.zip"
ASSET_CHECKSUM_ALGO = "sha256"

Expand Down
Loading