This commit refactors existing text utility functions into the
application layer for broad reuse and integrates them across
the codebase. Initially, these utilities were confined to test
code, which limited their application.
Changes:
- Move text utilities to the application layer.
- Centralize text utilities into dedicated files for better
maintainability.
- Improve robustness of utility functions with added type checks.
- Replace duplicated logic with centralized utility functions
throughout the codebase.
- Expand unit tests to cover refactored code parts.
This commit improves the URL health checking mechanism to reduce false
negatives.
- Treat all 2XX status codes as successful, addressing issues with codes
like `204`.
- Improve URL matching to exclude URLs within Markdown inline code block
and support URLs containing parentheses.
- Add `forceHttpGetForUrlPatterns` to customize HTTP method per URL to
allow verifying URLs behind CDN/WAFs that do not respond to HTTP HEAD.
- Send the Host header for improved handling of webpages behind proxies.
- Improve formatting and context for output messages.
- Fix the defaulting options for redirects and cookie handling.
- Update the user agent pool to modern browsers and platforms.
- Add support for randomizing TLS fingerprint to mimic various clients
better, improving the effectiveness of checks. However, this is not
fully supported by Node.js's HTTP client; see nodejs/undici#1983 for
more details.
- Use `AbortSignal` instead of `AbortController` as more modern and
simpler way to handle timeouts.
- Bump Node.js to version 18. This change is necessary as Node.js v16
will reach end-of-life on 2023-09-11. It also ensure compatibility
with dependencies requiring minimum of Node.js v18, such as `vite`,
`@vitejs`plugin-legacy` and `icon-gen`.
- Bump `setup-node` action to v4.
- Recommend using the `nvm` tool for managing Node.js versions in the
documentation.
- Update documentation to point to code reference for required Node.js
version. This removes duplication of information, and keeps the code
as single source of truth for required Node.js version.
- Refactor code to adopt the `node:` protocol for Node API imports as
per Node.js 18 standards. This change addresses ambiguities and aligns
with Node.js best practices (nodejs/node#38343). Currently, there is
no ESLint rule to enforce this protocol, as noted in
import-js/eslint-plugin-import#2717.
- Replace `cross-fetch` dependency with the native Node.js fetch API
introduced in Node.js 18. Adjust type casting for async iterable read
streams to align with the latest Node.js APIs, based on discussions in
DefinitelyTyped/DefinitelyTyped#65542.
This commit applies `strictNullChecks` to the entire codebase to improve
maintainability and type safety. Key changes include:
- Remove some explicit null-checks where unnecessary.
- Add necessary null-checks.
- Refactor static factory functions for a more functional approach.
- Improve some test names and contexts for better debugging.
- Add unit tests for any additional logic introduced.
- Refactor `createPositionFromRegexFullMatch` to its own function as the
logic is reused.
- Prefer `find` prefix on functions that may return `undefined` and
`get` prefix for those that always return a value.
- Move external URL checks to its own module under `tests/`. This
separates them from integration test, addressing long runs and
frequent failures that led to ignoring test results.
- Move `check-desktop-runtime-errors` to `tests/checks` to keep all
test-related checks into one directory.
- Replace `ts-node` with `vite` for running
`check-desktop-runtime-errors` to maintain a consistent execution
environment across checks.
- Implement a timeout for each fetch call.
- Be nice to external sources, wait 5 seconds before sending another
request to an URL under same domain. This solves rate-limiting issues.
- Instead of running test on every push/pull request, run them only
weekly.
- Do not run tests on each commit/PR but only scheduled (weekly) to
minimize noise.
- Fix URLs are not captured correctly inside backticks or parenthesis.
