This commit refactors existing text utility functions into the
application layer for broad reuse and integrates them across
the codebase. Initially, these utilities were confined to test
code, which limited their application.
Changes:
- Move text utilities to the application layer.
- Centralize text utilities into dedicated files for better
maintainability.
- Improve robustness of utility functions with added type checks.
- Replace duplicated logic with centralized utility functions
throughout the codebase.
- Expand unit tests to cover refactored code parts.
This commit improves the URL health checking mechanism to reduce false
negatives.
- Treat all 2XX status codes as successful, addressing issues with codes
like `204`.
- Improve URL matching to exclude URLs within Markdown inline code block
and support URLs containing parentheses.
- Add `forceHttpGetForUrlPatterns` to customize HTTP method per URL to
allow verifying URLs behind CDN/WAFs that do not respond to HTTP HEAD.
- Send the Host header for improved handling of webpages behind proxies.
- Improve formatting and context for output messages.
- Fix the defaulting options for redirects and cookie handling.
- Update the user agent pool to modern browsers and platforms.
- Add support for randomizing TLS fingerprint to mimic various clients
better, improving the effectiveness of checks. However, this is not
fully supported by Node.js's HTTP client; see nodejs/undici#1983 for
more details.
- Use `AbortSignal` instead of `AbortController` as more modern and
simpler way to handle timeouts.
This commit applies `strictNullChecks` to the entire codebase to improve
maintainability and type safety. Key changes include:
- Remove some explicit null-checks where unnecessary.
- Add necessary null-checks.
- Refactor static factory functions for a more functional approach.
- Improve some test names and contexts for better debugging.
- Add unit tests for any additional logic introduced.
- Refactor `createPositionFromRegexFullMatch` to its own function as the
logic is reused.
- Prefer `find` prefix on functions that may return `undefined` and
`get` prefix for those that always return a value.
- Move external URL checks to its own module under `tests/`. This
separates them from integration test, addressing long runs and
frequent failures that led to ignoring test results.
- Move `check-desktop-runtime-errors` to `tests/checks` to keep all
test-related checks into one directory.
- Replace `ts-node` with `vite` for running
`check-desktop-runtime-errors` to maintain a consistent execution
environment across checks.
- Implement a timeout for each fetch call.
- Be nice to external sources, wait 5 seconds before sending another
request to an URL under same domain. This solves rate-limiting issues.
- Instead of running test on every push/pull request, run them only
weekly.
- Do not run tests on each commit/PR but only scheduled (weekly) to
minimize noise.
- Fix URLs are not captured correctly inside backticks or parenthesis.
