This commit improves the URL health checking mechanism to reduce false
negatives.
- Treat all 2XX status codes as successful, addressing issues with codes
like `204`.
- Improve URL matching to exclude URLs within Markdown inline code block
and support URLs containing parentheses.
- Add `forceHttpGetForUrlPatterns` to customize HTTP method per URL to
allow verifying URLs behind CDN/WAFs that do not respond to HTTP HEAD.
- Send the Host header for improved handling of webpages behind proxies.
- Improve formatting and context for output messages.
- Fix the defaulting options for redirects and cookie handling.
- Update the user agent pool to modern browsers and platforms.
- Add support for randomizing TLS fingerprint to mimic various clients
better, improving the effectiveness of checks. However, this is not
fully supported by Node.js's HTTP client; see nodejs/undici#1983 for
more details.
- Use `AbortSignal` instead of `AbortController` as more modern and
simpler way to handle timeouts.
- Bump Node.js to version 18. This change is necessary as Node.js v16
will reach end-of-life on 2023-09-11. It also ensure compatibility
with dependencies requiring minimum of Node.js v18, such as `vite`,
`@vitejs`plugin-legacy` and `icon-gen`.
- Bump `setup-node` action to v4.
- Recommend using the `nvm` tool for managing Node.js versions in the
documentation.
- Update documentation to point to code reference for required Node.js
version. This removes duplication of information, and keeps the code
as single source of truth for required Node.js version.
- Refactor code to adopt the `node:` protocol for Node API imports as
per Node.js 18 standards. This change addresses ambiguities and aligns
with Node.js best practices (nodejs/node#38343). Currently, there is
no ESLint rule to enforce this protocol, as noted in
import-js/eslint-plugin-import#2717.
- Replace `cross-fetch` dependency with the native Node.js fetch API
introduced in Node.js 18. Adjust type casting for async iterable read
streams to align with the latest Node.js APIs, based on discussions in
DefinitelyTyped/DefinitelyTyped#65542.
- Move external URL checks to its own module under `tests/`. This
separates them from integration test, addressing long runs and
frequent failures that led to ignoring test results.
- Move `check-desktop-runtime-errors` to `tests/checks` to keep all
test-related checks into one directory.
- Replace `ts-node` with `vite` for running
`check-desktop-runtime-errors` to maintain a consistent execution
environment across checks.
- Implement a timeout for each fetch call.
- Be nice to external sources, wait 5 seconds before sending another
request to an URL under same domain. This solves rate-limiting issues.
- Instead of running test on every push/pull request, run them only
weekly.
- Do not run tests on each commit/PR but only scheduled (weekly) to
minimize noise.
- Fix URLs are not captured correctly inside backticks or parenthesis.
