Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 2.1.0 - Support for scraping multiple URLs in one step #33

Merged
merged 37 commits into from
Dec 10, 2023

Conversation

chriscarrollsmith
Copy link
Contributor

  • Added backward-compatible support for specifying feed_url and file_path as either a string or a JSON array of strings (to scrape multiple URLs in a single step).
  • Added more robust URL and file path validation.
  • Updated documentation to discuss new features and also to emphasize the need to escape quotation marks in JSON.

Closes #29, requested by @ndaidong.

dependabot bot and others added 30 commits October 11, 2023 07:44
Bumps [nock](https://github.com/nock/nock) from 13.3.3 to 13.3.4.
- [Release notes](https://github.com/nock/nock/releases)
- [Changelog](https://github.com/nock/nock/blob/main/CHANGELOG.md)
- [Commits](nock/nock@v13.3.3...v13.3.4)

---
updated-dependencies:
- dependency-name: nock
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env) from 7.22.20 to 7.23.2.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.23.2/packages/babel-preset-env)

---
updated-dependencies:
- dependency-name: "@babel/preset-env"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) from 7.23.0 to 7.23.2.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/@babel/core@7.23.2/packages/babel-core)

---
updated-dependencies:
- dependency-name: "@babel/core"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
…and_yarn/babel/core-7.23.2

Bump @babel/core from 7.23.0 to 7.23.2
…and_yarn/babel/preset-env-7.23.2

Bump @babel/preset-env from 7.22.20 to 7.23.2
…nd_yarn/nock-13.3.4

Bump nock from 13.3.3 to 13.3.4
Bumps [eslint](https://github.com/eslint/eslint) from 8.50.0 to 8.51.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](eslint/eslint@v8.50.0...v8.51.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…nd_yarn/eslint-8.51.0

Bump eslint from 8.50.0 to 8.51.0
Bumps [@vercel/ncc](https://github.com/vercel/ncc) from 0.38.0 to 0.38.1.
- [Release notes](https://github.com/vercel/ncc/releases)
- [Commits](vercel/ncc@0.38.0...0.38.1)

---
updated-dependencies:
- dependency-name: "@vercel/ncc"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 3 to 4.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](actions/setup-node@v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [eslint-plugin-jest](https://github.com/jest-community/eslint-plugin-jest) from 27.4.2 to 27.6.0.
- [Release notes](https://github.com/jest-community/eslint-plugin-jest/releases)
- [Changelog](https://github.com/jest-community/eslint-plugin-jest/blob/main/CHANGELOG.md)
- [Commits](jest-community/eslint-plugin-jest@v27.4.2...v27.6.0)

---
updated-dependencies:
- dependency-name: eslint-plugin-jest
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [nock](https://github.com/nock/nock) from 13.3.4 to 13.3.8.
- [Release notes](https://github.com/nock/nock/releases)
- [Changelog](https://github.com/nock/nock/blob/main/CHANGELOG.md)
- [Commits](nock/nock@v13.3.4...v13.3.8)

---
updated-dependencies:
- dependency-name: nock
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [eslint](https://github.com/eslint/eslint) from 8.51.0 to 8.54.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](eslint/eslint@v8.51.0...v8.54.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…and_yarn/vercel/ncc-0.38.1

Bump @vercel/ncc from 0.38.0 to 0.38.1
…ub_actions/actions/setup-node-4

Bump actions/setup-node from 3 to 4
…and_yarn/eslint-plugin-jest-27.6.0

Bump eslint-plugin-jest from 27.4.2 to 27.6.0
…and_yarn/eslint-8.54.0

Bump eslint from 8.51.0 to 8.54.0
…and_yarn/nock-13.3.8

Bump nock from 13.3.4 to 13.3.8
Bumps [@babel/eslint-parser](https://github.com/babel/babel/tree/HEAD/eslint/babel-eslint-parser) from 7.22.15 to 7.23.3.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.23.3/eslint/babel-eslint-parser)

---
updated-dependencies:
- dependency-name: "@babel/eslint-parser"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [prettier](https://github.com/prettier/prettier) from 3.0.3 to 3.1.0.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](prettier/prettier@3.0.3...3.1.0)

---
updated-dependencies:
- dependency-name: prettier
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…and_yarn/babel/eslint-parser-7.23.3

Bump @babel/eslint-parser from 7.22.15 to 7.23.3
…and_yarn/prettier-3.1.0

Bump prettier from 3.0.3 to 3.1.0
Bumps [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env) from 7.23.2 to 7.23.5.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.23.5/packages/babel-preset-env)

---
updated-dependencies:
- dependency-name: "@babel/preset-env"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) from 7.23.2 to 7.23.5.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.23.5/packages/babel-core)

---
updated-dependencies:
- dependency-name: "@babel/core"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
…and_yarn/babel/core-7.23.5

Bump @babel/core from 7.23.2 to 7.23.5
…and_yarn/babel/preset-env-7.23.5

Bump @babel/preset-env from 7.23.2 to 7.23.5
Bumps [nock](https://github.com/nock/nock) from 13.3.8 to 13.4.0.
- [Release notes](https://github.com/nock/nock/releases)
- [Changelog](https://github.com/nock/nock/blob/main/CHANGELOG.md)
- [Commits](nock/nock@v13.3.8...v13.4.0)

---
updated-dependencies:
- dependency-name: nock
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
chriscarrollsmith and others added 7 commits December 9, 2023 11:35
…and_yarn/nock-13.4.0

Bump nock from 13.3.8 to 13.4.0
Bumps [eslint](https://github.com/eslint/eslint) from 8.54.0 to 8.55.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](eslint/eslint@v8.54.0...v8.55.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…gs (of same length)

- added more robust URL/path validation
- added tests for arrays
- Version bump
…and_yarn/eslint-8.55.0

Bump eslint from 8.54.0 to 8.55.0
@chriscarrollsmith chriscarrollsmith merged commit 0bafbaf into releases/v2 Dec 10, 2023
8 checks passed
@ndaidong
Copy link

@chriscarrollsmith thank you, I believe this release has the potential to significantly address a wider range of use cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants