Skip to content

Commit

Permalink
wsj: remove WSJ from supported platforms (#194)
Browse files Browse the repository at this point in the history
* wsj: disable downloader command and url matching function
* readme: remove wsj from supported sites 😭
* workflows: remove WSJ tests from status check
  • Loading branch information
thisisparker authored Jul 7, 2024
1 parent 1fd662c commit f17a2ae
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 21 deletions.
18 changes: 0 additions & 18 deletions .github/workflows/status-check-outlets.yml
Original file line number Diff line number Diff line change
Expand Up @@ -169,24 +169,6 @@ jobs:
- name: Test Vox
if: '!cancelled()'
run: xword-dl vox
- name: Test WSJ latest
if: '!cancelled()'
env:
DATADOME_COOKIE: ${{ secrets.DATADOME_COOKIE }}
run: |
xword-dl wsj --settings '{"cookies": {"datadome":"'$DATADOME_COOKIE'"}}'
- name: Test WSJ by URL
if: '!cancelled()'
env:
DATADOME_COOKIE: ${{ secrets.DATADOME_COOKIE }}
run: |
xword-dl --settings '{"cookies": {"datadome":"'$DATADOME_COOKIE'"}}' "https://www.wsj.com/articles/carbon-neutral-saturday-crossword-january-15-11642193133"
- name: Test WSJ Friday contest
if: '!cancelled()'
env:
DATADOME_COOKIE: ${{ secrets.DATADOME_COOKIE }}
run: |
xword-dl --settings '{"cookies": {"datadome":"'$DATADOME_COOKIE'"}}' "https://www.wsj.com/articles/hitting-the-high-notes-friday-crossword-january-5-0f18d7c1"
- name: Test Washington Post latest
if: '!cancelled()'
run: xword-dl wp
Expand Down
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ Supported outlets:
|*Universal*|`uni`|✔️|✔️||
|*USA Today*|`usa`|✔️|✔️||
|*Vox*|`vox`|✔️|||
|*Wall Street Journal*|`wsj`|✔️||✔️|
|*Washington Post*|`wp`|✔️|✔️||

To download a puzzle, install `xword-dl` and run it on the command line.
Expand Down
8 changes: 6 additions & 2 deletions xword_dl/downloader/wsjdownloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,11 @@
from ..util import XWordDLException

class WSJDownloader(BaseDownloader):
command = 'wsj'
# Disabling this downloader for now (2024-07-07) because anti-scraping tech
# is preventing it from working. Hopefully we'll find a workaround or a
# a satisfactory mechanism for getting browser cookies in at runtime.
# Tracking issue: https://github.com/thisisparker/xword-dl/issues/178
# command = 'wsj'
outlet = 'Wall Street Journal'
outlet_prefix = 'WSJ'

Expand All @@ -17,7 +21,7 @@ def __init__(self, **kwargs):

@staticmethod
def matches_url(url_components):
return 'wsj.com' in url_components.netloc
return False # disabling, see above # 'wsj.com' in url_components.netloc

def find_latest(self):
url = "https://www.wsj.com/news/puzzle"
Expand Down

0 comments on commit f17a2ae

Please sign in to comment.