Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problematic site: notallowedto.com #147

Open
jemrobinson opened this issue Feb 18, 2019 · 1 comment
Open

Problematic site: notallowedto.com #147

jemrobinson opened this issue Feb 18, 2019 · 1 comment
Labels
problematic-site Site is broken or structure has changed

Comments

@jemrobinson
Copy link
Member

Our start URL (https://notallowedto.com/news) seems to only have a few articles and there is no paging structure to get through to earlier articles. Have you got any thoughts here @dongpng and @rtromble ?
@jemrobinson jemrobinson added the problematic-site Site is broken or structure has changed label Feb 18, 2019
@martintoreilly martintoreilly mentioned this issue Feb 18, 2019
Open
5 tasks
@edwardchalstrey1
Copy link
Collaborator

This site has hidden pagination, but the crawler is already working with the index page structure correctly. The real issue appears to be that there are no articles more recent than 2018-05, which is the month of the articles on index page 1, so I assume site no longer active.

@edwardchalstrey1 edwardchalstrey1 mentioned this issue May 24, 2019
Closed
@jemrobinson jemrobinson mentioned this issue Jun 4, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
problematic-site Site is broken or structure has changed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jemrobinson @edwardchalstrey1