Fix compiler failing with nested with expression

The previous implementation of `WithParser` used regex, which struggles
with parsing nested structures correctly. This commit improves
`WithParser` to track and parse all nested `with` expressions.

Other improvements:

- Throw meaningful errors when syntax is wrong. Replacing the prior
  behavior of silently ignoring such issues.
- Remove `I` prefix from related interfaces to align with newer code
  conventions.
- Add more unit tests for `with` expression.
- Improve documentation for templating.
- `ExpressionRegexBuilder`:
  - Use words `capture` and `match` correctly.
  - Fix minor issues revealed by new and improved tests:
     - Change regex for matching anything except surrounding
       whitespaces. The new regex ensures that it works even without
       having any preceeding text.
     - Change regex for capturing pipelines. The old regex was only
       matching (non-greedy) first character of the pipeline in tests,
       new regex matches the full pipeline.
- `ExpressionRegexBuilder.spec.ts`:
  - Ensure consistent way to define `describe` and `it` blocks.
  - Replace `expectRegex` tests, regex expectations test internal
    behavior of the class, not the external.
  - Simplified tests by eliminating the need for UUID suffixes/prefixes.

docs/templating.md

@@ -2,79 +2,142 @@
 
 ## Benefits of templating
 
-- Generating scripts by sharing code to increase best-practice usage and maintainability.
+- **Code sharing:** Share code across scripts for consistent practices and easier maintenance.
-- Creating self-contained scripts without cross-dependencies.
+- **Script independence:** Generate self-contained scripts, eliminating the need for external code.
-- Use of pipes for writing cleaner code and letting pipes do dirty work.
+- **Cleaner code:** Use pipes for complex operations, resulting in more readable and streamlined code.
 
 ## Expressions
 
-- Expressions start and end with mustaches (double brackets, `{{` and `}}`).
+**Syntax:**
-  - E.g. `Hello {{ $name }} !`
-- Syntax is close to [Go Templates ❤️](https://pkg.go.dev/text/template) but not the same.
-- Functions enables usage of expressions.
-  - In script definition parts of a function, see [`Function`](./collection-files.md#Function).
-  - When doing a call as argument values, see [`FunctionCall`](./collection-files.md#Function).
-- Expressions inside expressions (nested templates) are supported.
-  - An expression can output another expression that will also be compiled.
-  - E.g. following would compile first [with expression](#with), and then [parameter substitution](#parameter-substitution) in its output.
 
-    ```go
+Expressions are enclosed within `{{` and `}}`.
+Example: `Hello {{ $name }}!`.
+They are a core component of templating, enhancing scripts with dynamic capabilities and functionality.
+
+**Syntax similarity:**
+
+The syntax shares similarities with [Go Templates ❤️](https://pkg.go.dev/text/template), but with some differences:
+
+**Function definitions:**
+
+You can use expressions in function definition.
+Refer to [Function](./collection-files.md#function) for more details.
+
+Example usage:
+
+```yaml
+  name: GreetFunction
+  parameters:
+    - name: name
+  code: Hello {{ $name }}!
+```
+
+If you assign `name` the value `world`, invoking `GreetFunction` would result in `Hello world!`.
+
+**Function arguments:**
+
+You can also use expressions in arguments in nested function calls.
+Refer to [`Function | collection-files.md`](./collection-files.md#functioncall) for more details.
+
+Example with nested function calls:
+
+```yaml
+-
+  name: PrintMessageFunction
+  parameters:
+    - name: message
+  code: echo "{{ $message }}"
+-
+  name: GreetUserFunction
+  parameters:
+    - name: userName
+  call:
+    name: PrintMessageFunction
+    parameters:
+      argument: 'Hello, {{ $userName }}!'
+```
+
+Here, if `userName` is `Alice`, invoking `GreetUserFunction` would execute `echo "Hello, Alice!"`.
+
+**Nested templates:**
+
+You can nest expressions inside expressions (also called "nested templates").
+This means that an expression can output another expression where compiler will compile both.
+
+For example, following would compile first [with expression](#with), and then [parameter substitution](#parameter-substitution) in its output:
+
+```go
   {{ with $condition }}
     echo {{ $text }}
   {{ end }}
-    ```
+```
 
 ### Parameter substitution
 
-A simple function example:
+Parameter substitution dynamically replaces variable references with their corresponding values in the script.
+
+**Example function:**
 
 ```yaml
-  function: EchoArgument
+  name: DisplayTextFunction
   parameters:
-    - name: 'argument'
+    - name: 'text'
-  code: Hello {{ $argument }} !
+  code: echo {{ $text }}
 ```
 
-It would print "Hello world" if it's called in a [script](./collection-files.md#script) as following:
+Invoking `DisplayTextFunction` with `text` set to `"Hello, world!"` would result in `echo "Hello, World!"`.
-
-```yaml
-  script: Echo script
-  call:
-    function: EchoArgument
-    parameters:
-      argument: World
-```
-
-A function can call other functions such as:
-
-```yaml
-  - 
-    function: CallerFunction
-    parameters:
-      - name: 'value'
-    call:
-      function: EchoArgument
-      parameters:
-        argument: {{ $value }}
-  -
-    function: EchoArgument
-    parameters:
-      - name: 'argument'
-    code: Hello {{ $argument }} !
-```
 
 ### with
 
-Skips its "block" if the variable is absent or empty. Its "block" is between `with` start (`{{ with .. }}`) and end (`{{ end }`}) expressions.
+The `with` expression enables conditional rendering and provides a context variable for simpler code.
-E.g. `{{ with $parameterName }} Hi, I'm a block! {{ end }}` would only output `Hi, I'm a block!` if `parameterName` has any value..
 
-It binds its context (value of the provided parameter value) as arbitrary `.` value. It allows you to use the argument value of the given parameter when it is provided and not empty such as:
+**Optional block rendering:**
+
+If the provided variable is falsy (`false`, `null`, or empty), the compiler skips the enclosed block of code.
+A "block" lies between the with start (`{{ with .. }}`) and end (`{{ end }}`) expressions, defining its boundaries.
+
+Example:
+
+```go
+{{ with $optionalVariable }}
+  Hello
+{{ end }}
+```
+
+This would display `Hello` if `$optionalVariable` is truthy.
+
+**Parameter declaration:**
+
+You should set `optional: true` for the argument if you use it like `{{ with $argument }} .. {{ end }}`.
+
+Declare parameters used for `with` condition as optional such as:
+
+```yaml
+name: ConditionalOutputFunction
+parameters:
+  - name: 'data'
+    optional: true
+code: |- 
+  {{ with $data }}
+    Data is: {{ . }}
+  {{ end }}
+```
+
+**Context variable:**
+
+`with` statement binds its context (value of the provided parameter value) as arbitrary `.` value.
+`{{ . }}` syntax gives you access to the context variable.
+This is optional to use, and not required to use `with` expressions.
+
+For example:
 
 ```go
   {{ with $parameterName }}Parameter value is {{ . }} here {{ end }}
 ```
 
-It supports multiline text inside the block. You can have something like:
+**Multiline text:**
+
+It supports multiline text inside the block. You can write something like:
 
 ```go
   {{ with $argument }}
@@ -83,7 +146,9 @@ It supports multiline text inside the block. You can have something like:
   {{ end }}
 ```
 
-You can also use other expressions inside its block, such as [parameter substitution](#parameter-substitution):
+**Inner expressions:**
+
+You can also embed other expressions inside its block, such as [parameter substitution](#parameter-substitution):
 
 ```go
   {{ with $condition }}
@@ -91,32 +156,44 @@ You can also use other expressions inside its block, such as [parameter substitu
   {{ end }}
 ```
 
-💡 Declare parameters used for `with` condition as optional. Set `optional: true` for the argument if you use it like `{{ with $argument }} .. {{ end }}`.
+This also includes nesting `with` statements:
 
-Example:
+```go
-
+  {{ with $condition1 }}
-```yaml
+    Value of $condition1: {{ . }}
-  function: FunctionThatOutputsConditionally
+    {{ with $condition2 }}
-  parameters:
+      Value of $condition2: {{ . }}
-    - name: 'argument'
+    {{ end }}
-      optional: true
-  code: |- 
-    {{ with $argument }}
-      Value is: {{ . }}
   {{ end }}
 ```
 
 ### Pipes
 
-- Pipes are functions available for handling text.
+Pipes are functions designed for text manipulation.
-- Allows stacking actions one after another also known as "chaining".
+They allow for a sequential application of operations resembling [Unix pipelines](https://en.wikipedia.org/wiki/Pipeline_(Unix)), also known as "chaining".
-- Like [Unix pipelines](https://en.wikipedia.org/wiki/Pipeline_(Unix)), the concept is simple: each pipeline's output becomes the input of the following pipe.
+Each pipeline's output becomes the input of the following pipe.
-- You cannot create pipes. [A dedicated compiler](./application.md#parsing-and-compiling) provides pre-defined pipes to consume in collection files.
+
-- You can combine pipes with other expressions such as [parameter substitution](#parameter-substitution) and [with](#with) syntax.
+**Pre-defined**:
-- ❗ Pipe names must be camelCase without any space or special characters.
+
-- **Existing pipes**
+Pipes are pre-defined by the system.
-  - `inlinePowerShell`: Converts a multi-lined PowerShell script to a single line.
+You cannot create pipes in [collection files](./collection-files.md).
-  - `escapeDoubleQuotes`: Escapes `"` characters, allows you to use them inside double quotes (`"`).
+[A dedicated compiler](./application.md#parsing-and-compiling) provides pre-defined pipes to consume in collection files.
-- **Example usages**
+
-  - `{{ with $code }} echo "{{ . | inlinePowerShell }}" {{ end }}`
+**Compatibility:**
-  - `{{ with $code }} echo "{{ . | inlinePowerShell | escapeDoubleQuotes }}" {{ end }}`
+
+You can combine pipes with other expressions such as [parameter substitution](#parameter-substitution) and [with](#with) syntax.
+
+For example:
+
+```go
+{{ with $script }} echo "{{ . | inlinePowerShell | escapeDoubleQuotes }}" {{ end }}
+```
+
+**Naming:**
+
+❗ Pipe names must be camelCase without any space or special characters.
+
+**Available pipes:**
+
+- `inlinePowerShell`: Converts a multi-lined PowerShell script to a single line.
+- `escapeDoubleQuotes`: Escapes `"` characters for batch command execution, allows you to use them inside double quotes (`"`).

src/application/Parser/Script/Compiler/Expressions/Parser/Regex/ExpressionRegexBuilder.ts

@@ -14,45 +14,44 @@ export class ExpressionRegexBuilder {
       .addRawRegex('\\s+');
   }
 
-  public matchPipeline() {
+  public captureOptionalPipeline() {
     return this
-      .expectZeroOrMoreWhitespaces()
+      .addRawRegex('((?:\\|\\s*\\b[a-zA-Z]+\\b\\s*)*)');
-      .addRawRegex('(\\|\\s*.+?)?');
   }
 
-  public matchUntilFirstWhitespace() {
+  public captureUntilWhitespaceOrPipe() {
     return this
       .addRawRegex('([^|\\s]+)');
   }
 
-  public matchMultilineAnythingExceptSurroundingWhitespaces() {
+  public captureMultilineAnythingExceptSurroundingWhitespaces() {
     return this
-      .expectZeroOrMoreWhitespaces()
+      .expectOptionalWhitespaces()
-      .addRawRegex('([\\S\\s]+?)')
+      .addRawRegex('([\\s\\S]*\\S)')
-      .expectZeroOrMoreWhitespaces();
+      .expectOptionalWhitespaces();
   }
 
   public expectExpressionStart() {
     return this
       .expectCharacters('{{')
-      .expectZeroOrMoreWhitespaces();
+      .expectOptionalWhitespaces();
   }
 
   public expectExpressionEnd() {
     return this
-      .expectZeroOrMoreWhitespaces()
+      .expectOptionalWhitespaces()
       .expectCharacters('}}');
   }
 
+  public expectOptionalWhitespaces() {
+    return this
+      .addRawRegex('\\s*');
+  }
+
   public buildRegExp(): RegExp {
     return new RegExp(this.parts.join(''), 'g');
   }
 
-  private expectZeroOrMoreWhitespaces() {
-    return this
-      .addRawRegex('\\s*');
-  }
-
   private addRawRegex(regex: string) {
     this.parts.push(regex);
     return this;

src/application/Parser/Script/Compiler/Expressions/SyntaxParsers/ParameterSubstitutionParser.ts

@@ -6,8 +6,9 @@ export class ParameterSubstitutionParser extends RegexParser {
   protected readonly regex = new ExpressionRegexBuilder()
     .expectExpressionStart()
     .expectCharacters('$')
-    .matchUntilFirstWhitespace() // First match: Parameter name
+    .captureUntilWhitespaceOrPipe() // First capture: Parameter name
-    .matchPipeline() // Second match: Pipeline
+    .expectOptionalWhitespaces()
+    .captureOptionalPipeline() // Second capture: Pipeline
     .expectExpressionEnd()
     .buildRegExp();
 

src/application/Parser/Script/Compiler/Expressions/SyntaxParsers/WithParser.ts

@@ -1,59 +1,222 @@
+// eslint-disable-next-line max-classes-per-file
+import { IExpressionParser } from '@/application/Parser/Script/Compiler/Expressions/Parser/IExpressionParser';
+import { FunctionParameterCollection } from '@/application/Parser/Script/Compiler/Function/Parameter/FunctionParameterCollection';
 import { FunctionParameter } from '@/application/Parser/Script/Compiler/Function/Parameter/FunctionParameter';
-import { RegexParser, IPrimitiveExpression } from '../Parser/Regex/RegexParser';
+import { IExpression } from '../Expression/IExpression';
+import { ExpressionPosition } from '../Expression/ExpressionPosition';
 import { ExpressionRegexBuilder } from '../Parser/Regex/ExpressionRegexBuilder';
 
-export class WithParser extends RegexParser {
+export class WithParser implements IExpressionParser {
-  protected readonly regex = new ExpressionRegexBuilder()
+  public findExpressions(code: string): IExpression[] {
-  // {{ with $parameterName }}
+    if (!code) {
-    .expectExpressionStart()
+      throw new Error('missing code');
-    .expectCharacters('with')
+    }
-    .expectOneOrMoreWhitespaces()
+    return parseWithExpressions(code);
-    .expectCharacters('$')
+  }
-    .matchUntilFirstWhitespace() // First match: parameter name
+}
-    .expectExpressionEnd()
-  // ...
-    .matchMultilineAnythingExceptSurroundingWhitespaces() // Second match: Scope text
-  // {{ end }}
-    .expectExpressionStart()
-    .expectCharacters('end')
-    .expectExpressionEnd()
-    .buildRegExp();
 
-  protected buildExpression(match: RegExpMatchArray): IPrimitiveExpression {
+enum WithStatementType {
-    const parameterName = match[1];
+  Start,
-    const scopeText = match[2];
+  End,
+  ContextVariable,
+}
+
+type WithStatement = {
+  readonly type: WithStatementType.Start;
+  readonly parameterName: string;
+  readonly position: ExpressionPosition;
+} | {
+  readonly type: WithStatementType.End;
+  readonly position: ExpressionPosition;
+} | {
+  readonly type: WithStatementType.ContextVariable;
+  readonly position: ExpressionPosition;
+  readonly pipeline: string | undefined;
+};
+
+function parseAllWithExpressions(
+  input: string,
+): WithStatement[] {
+  const expressions = new Array<WithStatement>();
+  for (const match of input.matchAll(WithStatementStartRegEx)) {
+    expressions.push({
+      type: WithStatementType.Start,
+      parameterName: match[1],
+      position: createPosition(match),
+    });
+  }
+  for (const match of input.matchAll(WithStatementEndRegEx)) {
+    expressions.push({
+      type: WithStatementType.End,
+      position: createPosition(match),
+    });
+  }
+  for (const match of input.matchAll(ContextVariableWithPipelineRegEx)) {
+    expressions.push({
+      type: WithStatementType.ContextVariable,
+      position: createPosition(match),
+      pipeline: match[1],
+    });
+  }
+  return expressions;
+}
+
+function createPosition(match: RegExpMatchArray): ExpressionPosition {
+  const startPos = match.index;
+  const endPos = startPos + match[0].length;
+  return new ExpressionPosition(startPos, endPos);
+}
+
+class WithStatementBuilder {
+  private readonly contextVariables = new Array<{
+    readonly positionInScope: ExpressionPosition;
+    readonly pipeline: string | undefined;
+  }>();
+
+  public addContextVariable(
+    absolutePosition: ExpressionPosition,
+    pipeline: string | undefined,
+  ): void {
+    const positionInScope = new ExpressionPosition(
+      absolutePosition.start - this.startExpressionPosition.end,
+      absolutePosition.end - this.startExpressionPosition.end,
+    );
+    this.contextVariables.push({
+      positionInScope,
+      pipeline,
+    });
+  }
+
+  public buildExpression(endExpressionPosition: ExpressionPosition, input: string): IExpression {
+    const parameters = new FunctionParameterCollection();
+    parameters.addParameter(new FunctionParameter(this.parameterName, true));
+    const position = new ExpressionPosition(
+      this.startExpressionPosition.start,
+      endExpressionPosition.end,
+    );
+    const scope = input.substring(this.startExpressionPosition.end, endExpressionPosition.start);
     return {
-      parameters: [new FunctionParameter(parameterName, true)],
+      parameters,
-      evaluator: (context) => {
+      position,
-        const argumentValue = context.args.hasArgument(parameterName)
+      evaluate: (context) => {
-          ? context.args.getArgument(parameterName).argumentValue
+        const argumentValue = context.args.hasArgument(this.parameterName)
+          ? context.args.getArgument(this.parameterName).argumentValue
           : undefined;
         if (!argumentValue) {
           return '';
         }
-        return replaceEachScopeSubstitution(scopeText, (pipeline) => {
+        const substitutedScope = this.substituteContextVariables(scope, (pipeline) => {
           if (!pipeline) {
             return argumentValue;
           }
           return context.pipelineCompiler.compile(argumentValue, pipeline);
         });
+        return substitutedScope;
       },
     };
   }
+
+  constructor(
+    private readonly startExpressionPosition: ExpressionPosition,
+    private readonly parameterName: string,
+  ) {
+
+  }
+
+  private substituteContextVariables(
+    scope: string,
+    substituter: (pipeline: string) => string,
+  ): string {
+    if (!this.contextVariables.length) {
+      return scope;
+    }
+    let substitutedScope = '';
+    let scopeSubstrIndex = 0;
+    for (const contextVariable of this.contextVariables) {
+      substitutedScope += scope.substring(scopeSubstrIndex, contextVariable.positionInScope.start);
+      substitutedScope += substituter(contextVariable.pipeline);
+      scopeSubstrIndex = contextVariable.positionInScope.end;
+    }
+    substitutedScope += scope.substring(scopeSubstrIndex, scope.length);
+    return substitutedScope;
+  }
 }
 
-const ScopeSubstitutionRegEx = new ExpressionRegexBuilder()
+function buildErrorContext(code: string, statements: readonly WithStatement[]): string {
+  const formattedStatements = statements.map((s) => `- [${s.position.start}, ${s.position.end}] ${WithStatementType[s.type]}`).join('\n');
+  return [
+    'Code:', '---', code, '---',
+    'nStatements:', '---', formattedStatements, '---',
+  ].join('\n');
+}
+
+function parseWithExpressions(input: string): IExpression[] {
+  const allStatements = parseAllWithExpressions(input);
+  const sortedStatements = allStatements
+    .slice()
+    .sort((a, b) => b.position.start - a.position.start);
+  const expressions = new Array<IExpression>();
+  const builders = new Array<WithStatementBuilder>();
+  const throwWithContext = (message: string) => {
+    throw new Error(`${message}\n${buildErrorContext(input, allStatements)}}`);
+  };
+  while (sortedStatements.length > 0) {
+    const statement = sortedStatements.pop();
+    if (!statement) {
+      break;
+    }
+    switch (statement.type) { // eslint-disable-line default-case
+      case WithStatementType.Start:
+        builders.push(new WithStatementBuilder(
+          statement.position,
+          statement.parameterName,
+        ));
+        break;
+      case WithStatementType.ContextVariable:
+        if (builders.length === 0) {
+          throwWithContext('Context variable before `with` statement.');
+        }
+        builders[builders.length - 1].addContextVariable(statement.position, statement.pipeline);
+        break;
+      case WithStatementType.End:
+        if (builders.length === 0) {
+          throwWithContext('Redundant `end` statement, missing `with`?');
+        }
+        expressions.push(builders.pop().buildExpression(statement.position, input));
+        break;
+    }
+  }
+  if (builders.length > 0) {
+    throwWithContext('Missing `end` statement, forgot `{{ end }}?');
+  }
+  return expressions;
+}
+
+const ContextVariableWithPipelineRegEx = new ExpressionRegexBuilder()
 // {{ . | pipeName }}
   .expectExpressionStart()
   .expectCharacters('.')
-  .matchPipeline() // First match: pipeline
+  .expectOptionalWhitespaces()
+  .captureOptionalPipeline() // First capture: pipeline
   .expectExpressionEnd()
   .buildRegExp();
 
-function replaceEachScopeSubstitution(scopeText: string, replacer: (pipeline: string) => string) {
+const WithStatementStartRegEx = new ExpressionRegexBuilder()
-  // Not using /{{\s*.\s*(?:(\|\s*[^{}]*?)\s*)?}}/g for not matching brackets,
+  // {{ with $parameterName }}
-  // but instead letting the pipeline compiler to fail on those.
+  .expectExpressionStart()
-  return scopeText.replaceAll(ScopeSubstitutionRegEx, (_$, match1) => {
+  .expectCharacters('with')
-    return replacer(match1);
+  .expectOneOrMoreWhitespaces()
-  });
+  .expectCharacters('$')
-}
+  .captureUntilWhitespaceOrPipe() // First capture: parameter name
+  .expectExpressionEnd()
+  .expectOptionalWhitespaces()
+  .buildRegExp();
+
+const WithStatementEndRegEx = new ExpressionRegexBuilder()
+  // {{ end }}
+  .expectOptionalWhitespaces()
+  .expectExpressionStart()
+  .expectCharacters('end')
+  .expectOptionalWhitespaces()
+  .expectExpressionEnd()
+  .buildRegExp();

tests/unit/application/Parser/Script/Compiler/Expressions/Parser/Regex/ExpressionRegexBuilder.spec.ts

@@ -1,126 +1,295 @@
-import { randomUUID } from 'crypto';
 import { describe, it, expect } from 'vitest';
 import { ExpressionRegexBuilder } from '@/application/Parser/Script/Compiler/Expressions/Parser/Regex/ExpressionRegexBuilder';
 
+const AllWhitespaceCharacters = ' \t\n\r\v\f\u00A0';
+
 describe('ExpressionRegexBuilder', () => {
   describe('expectCharacters', () => {
-    describe('escape single as expected', () => {
+    describe('expectCharacters', () => {
+      describe('escapes single character as expected', () => {
         const charactersToEscape = ['.', '$'];
         for (const character of charactersToEscape) {
-        it(character, () => {
+          it(`escapes ${character} as expected`, () => expectMatch(
-          expectRegex(
+            character,
-            // act
             (act) => act.expectCharacters(character),
-            // assert
+            `${character}`,
-            `\\${character}`,
+          ));
-          );
-        });
         }
       });
-    it('escapes multiple as expected', () => {
+      it('escapes multiple characters as expected', () => expectMatch(
-      expectRegex(
+        '.I have no $$.',
-        // act
         (act) => act.expectCharacters('.I have no $$.'),
-        // assert
+        '.I have no $$.',
-        '\\.I have no \\$\\$\\.',
+      ));
-      );
+      it('adds characters as expected', () => expectMatch(
-    });
-    it('adds as expected', () => {
-      expectRegex(
-        // act
-        (act) => act.expectCharacters('return as it is'),
-        // assert
         'return as it is',
-      );
+        (act) => act.expectCharacters('return as it is'),
+        'return as it is',
+      ));
     });
   });
-  it('expectOneOrMoreWhitespaces', () => {
+  describe('expectOneOrMoreWhitespaces', () => {
-    expectRegex(
+    it('matches one whitespace', () => expectMatch(
-      // act
+      ' ',
       (act) => act.expectOneOrMoreWhitespaces(),
-      // assert
+      ' ',
-      '\\s+',
+    ));
-    );
+    it('matches multiple whitespaces', () => expectMatch(
+      AllWhitespaceCharacters,
+      (act) => act.expectOneOrMoreWhitespaces(),
+      AllWhitespaceCharacters,
+    ));
+    it('matches whitespaces inside text', () => expectMatch(
+      `start${AllWhitespaceCharacters}end`,
+      (act) => act.expectOneOrMoreWhitespaces(),
+      AllWhitespaceCharacters,
+    ));
+    it('does not match non-whitespace characters', () => expectNonMatch(
+      'a',
+      (act) => act.expectOneOrMoreWhitespaces(),
+    ));
   });
-  it('matchPipeline', () => {
+  describe('captureOptionalPipeline', () => {
-    expectRegex(
+    it('does not capture when no pipe is present', () => expectNonMatch(
-      // act
+      'noPipeHere',
-      (act) => act.matchPipeline(),
+      (act) => act.captureOptionalPipeline(),
-      // assert
+    ));
-      '\\s*(\\|\\s*.+?)?',
+    it('captures when input starts with pipe', () => expectCapture(
-    );
+      '| afterPipe',
+      (act) => act.captureOptionalPipeline(),
+      '| afterPipe',
+    ));
+    it('ignores without text before', () => expectCapture(
+      'stuff before | afterPipe',
+      (act) => act.captureOptionalPipeline(),
+      '| afterPipe',
+    ));
+    it('ignores without text before', () => expectCapture(
+      'stuff before | afterPipe',
+      (act) => act.captureOptionalPipeline(),
+      '| afterPipe',
+    ));
+    it('ignores whitespaces before the pipe', () => expectCapture(
+      '   | afterPipe',
+      (act) => act.captureOptionalPipeline(),
+      '| afterPipe',
+    ));
+    it('ignores text after whitespace', () => expectCapture(
+      '| first Pipe',
+      (act) => act.captureOptionalPipeline(),
+      '| first ',
+    ));
+    describe('non-greedy matching', () => { // so the rest of the pattern can work
+      it('non-letter character in pipe', () => expectCapture(
+        '| firstPipe | sec0ndpipe',
+        (act) => act.captureOptionalPipeline(),
+        '| firstPipe ',
+      ));
     });
-  it('matchUntilFirstWhitespace', () => {
+  });
-    expectRegex(
+  describe('captureUntilWhitespaceOrPipe', () => {
-      // act
+    it('captures until first whitespace', () => expectCapture(
-      (act) => act.matchUntilFirstWhitespace(),
-      // assert
-      '([^|\\s]+)',
-    );
-    it('matches until first whitespace', () => expectMatch(
       // arrange
-      'first second',
+      'first ',
       // act
-      (act) => act.matchUntilFirstWhitespace(),
+      (act) => act.captureUntilWhitespaceOrPipe(),
       // assert
       'first',
     ));
+    it('captures until first pipe', () => expectCapture(
+      // arrange
+      'first|',
+      // act
+      (act) => act.captureUntilWhitespaceOrPipe(),
+      // assert
+      'first',
+    ));
+    it('captures all without whitespace or pipe', () => expectCapture(
+      // arrange
+      'all',
+      // act
+      (act) => act.captureUntilWhitespaceOrPipe(),
+      // assert
+      'all',
+    ));
   });
-  describe('matchMultilineAnythingExceptSurroundingWhitespaces', () => {
+  describe('captureMultilineAnythingExceptSurroundingWhitespaces', () => {
-    it('returns expected regex', () => expectRegex(
+    describe('single line', () => {
-      // act
+      it('captures a line without surrounding whitespaces', () => expectCapture(
-      (act) => act.matchMultilineAnythingExceptSurroundingWhitespaces(),
-      // assert
-      '\\s*([\\S\\s]+?)\\s*',
-    ));
-    it('matches single line', () => expectMatch(
         // arrange
-      'single line',
+        'line',
         // act
-      (act) => act.matchMultilineAnythingExceptSurroundingWhitespaces(),
+        (act) => act.captureMultilineAnythingExceptSurroundingWhitespaces(),
+        // assert
+        'line',
+      ));
+      it('captures a line with internal whitespaces intact', () => expectCapture(
+        `start${AllWhitespaceCharacters}end`,
+        (act) => act.captureMultilineAnythingExceptSurroundingWhitespaces(),
+        `start${AllWhitespaceCharacters}end`,
+      ));
+      it('excludes surrounding whitespaces', () => expectCapture(
+        // arrange
+        `${AllWhitespaceCharacters}single line\t`,
+        // act
+        (act) => act.captureMultilineAnythingExceptSurroundingWhitespaces(),
         // assert
         'single line',
       ));
-    it('matches single line without surrounding whitespaces', () => expectMatch(
+    });
+    describe('multiple lines', () => {
+      it('captures text across multiple lines', () => expectCapture(
         // arrange
-      '  single line\t',
+        'first line\nsecond line\r\nthird-line',
         // act
-      (act) => act.matchMultilineAnythingExceptSurroundingWhitespaces(),
+        (act) => act.captureMultilineAnythingExceptSurroundingWhitespaces(),
         // assert
-      'single line',
+        'first line\nsecond line\r\nthird-line',
       ));
-    it('matches multiple lines', () => expectMatch(
+      it('captures text with empty lines in between', () => expectCapture(
-      // arrange
+        'start\n\nend',
-      'first line\nsecond line',
+        (act) => act.captureMultilineAnythingExceptSurroundingWhitespaces(),
-      // act
+        'start\n\nend',
-      (act) => act.matchMultilineAnythingExceptSurroundingWhitespaces(),
-      // assert
-      'first line\nsecond line',
       ));
-    it('matches multiple lines without surrounding whitespaces', () => expectMatch(
+      it('excludes surrounding whitespaces from multiline text', () => expectCapture(
         // arrange
-      '  first line\nsecond line\t',
+        `  first line\nsecond line${AllWhitespaceCharacters}`,
         // act
-      (act) => act.matchMultilineAnythingExceptSurroundingWhitespaces(),
+        (act) => act.captureMultilineAnythingExceptSurroundingWhitespaces(),
         // assert
         'first line\nsecond line',
       ));
     });
-  it('expectExpressionStart', () => {
+    describe('edge cases', () => {
-    expectRegex(
+      it('does not capture for input with only whitespaces', () => expectNonCapture(
-      // act
+        AllWhitespaceCharacters,
+        (act) => act.captureMultilineAnythingExceptSurroundingWhitespaces(),
+      ));
+    });
+  });
+  describe('expectExpressionStart', () => {
+    it('matches expression start without trailing whitespaces', () => expectMatch(
+      '{{expression',
       (act) => act.expectExpressionStart(),
-      // assert
+      '{{',
-      '{{\\s*',
+    ));
-    );
+    it('matches expression start with trailing whitespaces', () => expectMatch(
+      `{{${AllWhitespaceCharacters}expression`,
+      (act) => act.expectExpressionStart(),
+      `{{${AllWhitespaceCharacters}`,
+    ));
+    it('does not match whitespaces not directly after expression start', () => expectMatch(
+      '   {{expression',
+      (act) => act.expectExpressionStart(),
+      '{{',
+    ));
+    it('does not match if expression start is not present', () => expectNonMatch(
+      'noExpressionStartHere',
+      (act) => act.expectExpressionStart(),
+    ));
   });
-  it('expectExpressionEnd', () => {
+  describe('expectExpressionEnd', () => {
-    expectRegex(
+    it('matches expression end without preceding whitespaces', () => expectMatch(
-      // act
+      'expression}}',
       (act) => act.expectExpressionEnd(),
+      '}}',
+    ));
+    it('matches expression end with preceding whitespaces', () => expectMatch(
+      `expression${AllWhitespaceCharacters}}}`,
+      (act) => act.expectExpressionEnd(),
+      `${AllWhitespaceCharacters}}}`,
+    ));
+    it('does not capture whitespaces not directly before expression end', () => expectMatch(
+      'expression}}   ',
+      (act) => act.expectExpressionEnd(),
+      '}}',
+    ));
+    it('does not match if expression end is not present', () => expectNonMatch(
+      'noExpressionEndHere',
+      (act) => act.expectExpressionEnd(),
+    ));
+  });
+  describe('expectOptionalWhitespaces', () => {
+    describe('matching', () => {
+      it('matches multiple Unix lines', () => expectMatch(
+        // arrange
+        '\n\n',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
         // assert
-      '\\s*}}',
+        '\n\n',
-    );
+      ));
+      it('matches multiple Windows lines', () => expectMatch(
+        // arrange
+        '\r\n',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+        // assert
+        '\r\n',
+      ));
+      it('matches multiple spaces', () => expectMatch(
+        // arrange
+        '  ',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+        // assert
+        '  ',
+      ));
+      it('matches horizontal and vertical tabs', () => expectMatch(
+        // arrange
+        '\t\v',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+        // assert
+        '\t\v',
+      ));
+      it('matches form feed character', () => expectMatch(
+        // arrange
+        '\f',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+        // assert
+        '\f',
+      ));
+      it('matches a non-breaking space character', () => expectMatch(
+        // arrange
+        '\u00A0',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+        // assert
+        '\u00A0',
+      ));
+      it('matches a combination of whitespace characters', () => expectMatch(
+        // arrange
+        AllWhitespaceCharacters,
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+        // assert
+        AllWhitespaceCharacters,
+      ));
+      it('matches whitespace characters on different positions', () => expectMatch(
+        // arrange
+        '\ta\nb\rc\v',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+        // assert
+        '\t\n\r\v',
+      ));
+    });
+    describe('non-matching', () => {
+      it('a non-whitespace character', () => expectNonMatch(
+        // arrange
+        'a',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+      ));
+      it('multiple non-whitespace characters', () => expectNonMatch(
+        // arrange
+        'abc',
+        // act
+        (act) => act.expectOptionalWhitespaces(),
+      ));
+    });
   });
   describe('buildRegExp', () => {
     it('sets global flag', () => {
@@ -134,84 +303,126 @@ describe('ExpressionRegexBuilder', () => {
       expect(actual).to.equal(expected);
     });
     describe('can combine multiple parts', () => {
-      it('with', () => {
+      it('combines character and whitespace expectations', () => expectMatch(
-        expectRegex(
+        'abc def',
-          (sut) => sut
+        (act) => act
-          // act
+          .expectCharacters('abc')
-            // {{ with $variable }}
-            .expectExpressionStart()
-            .expectCharacters('with')
           .expectOneOrMoreWhitespaces()
-            .expectCharacters('$')
+          .expectCharacters('def'),
-            .matchUntilFirstWhitespace()
+        'abc def',
-            .expectExpressionEnd()
+      ));
-            // scope
+      it('captures optional pipeline and text after it', () => expectCapture(
-            .matchMultilineAnythingExceptSurroundingWhitespaces()
+        'abc | def',
-            // {{ end }}
+        (act) => act
+          .expectCharacters('abc ')
+          .captureOptionalPipeline(),
+        '| def',
+      ));
+      it('combines multiline capture with optional whitespaces', () => expectCapture(
+        '\n  abc  \n',
+        (act) => act
+          .expectOptionalWhitespaces()
+          .captureMultilineAnythingExceptSurroundingWhitespaces()
+          .expectOptionalWhitespaces(),
+        'abc',
+      ));
+      it('combines expression start, optional whitespaces, and character expectation', () => expectMatch(
+        '{{ abc',
+        (act) => act
           .expectExpressionStart()
-            .expectCharacters('end')
+          .expectOptionalWhitespaces()
+          .expectCharacters('abc'),
+        '{{ abc',
+      ));
+      it('combines character expectation, optional whitespaces, and expression end', () => expectMatch(
+        'abc }}',
+        (act) => act
+          .expectCharacters('abc')
+          .expectOptionalWhitespaces()
           .expectExpressionEnd(),
-          // assert
+        'abc }}',
-          '{{\\s*with\\s+\\$([^|\\s]+)\\s*}}\\s*([\\S\\s]+?)\\s*{{\\s*end\\s*}}',
+      ));
-        );
-      });
-      it('scoped substitution', () => {
-        expectRegex(
-          (sut) => sut
-          // act
-            .expectExpressionStart().expectCharacters('.')
-            .matchPipeline()
-            .expectExpressionEnd(),
-          // assert
-          '{{\\s*\\.\\s*(\\|\\s*.+?)?\\s*}}',
-        );
-      });
-      it('parameter substitution', () => {
-        expectRegex(
-          (sut) => sut
-          // act
-            .expectExpressionStart().expectCharacters('$')
-            .matchUntilFirstWhitespace()
-            .matchPipeline()
-            .expectExpressionEnd(),
-          // assert
-          '{{\\s*\\$([^|\\s]+)\\s*(\\|\\s*.+?)?\\s*}}',
-        );
-      });
     });
   });
 });
 
-function expectRegex(
+enum MatchGroupIndex {
-  act: (sut: ExpressionRegexBuilder) => ExpressionRegexBuilder,
+  FullMatch = 0,
-  expected: string,
+  FirstCapturingGroup = 1,
-) {
+}
+
+function expectCapture(
+  input: string,
+  act: (regexBuilder: ExpressionRegexBuilder) => ExpressionRegexBuilder,
+  expectedCombinedCaptures: string | undefined,
+): void {
   // arrange
-  const sut = new ExpressionRegexBuilder();
+  const matchGroupIndex = MatchGroupIndex.FirstCapturingGroup;
   // act
-  const actual = act(sut).buildRegExp().source;
   // assert
-  expect(actual).to.equal(expected);
+  expectMatch(input, act, expectedCombinedCaptures, matchGroupIndex);
+}
+
+function expectNonMatch(
+  input: string,
+  act: (sut: ExpressionRegexBuilder) => ExpressionRegexBuilder,
+  matchGroupIndex = MatchGroupIndex.FullMatch,
+): void {
+  expectMatch(input, act, undefined, matchGroupIndex);
+}
+
+function expectNonCapture(
+  input: string,
+  act: (sut: ExpressionRegexBuilder) => ExpressionRegexBuilder,
+): void {
+  expectNonMatch(input, act, MatchGroupIndex.FirstCapturingGroup);
 }
 
 function expectMatch(
   input: string,
-  act: (sut: ExpressionRegexBuilder) => ExpressionRegexBuilder,
+  act: (regexBuilder: ExpressionRegexBuilder) => ExpressionRegexBuilder,
-  expectedMatch: string,
+  expectedCombinedMatches: string | undefined,
-) {
+  matchGroupIndex = MatchGroupIndex.FullMatch,
+): void {
   // arrange
-  const [startMarker, endMarker] = [randomUUID(), randomUUID()];
+  const regexBuilder = new ExpressionRegexBuilder();
-  const markedInput = `${startMarker}${input}${endMarker}`;
+  act(regexBuilder);
-  const builder = new ExpressionRegexBuilder()
+  const regex = regexBuilder.buildRegExp();
-    .expectCharacters(startMarker);
-  act(builder);
-  const markedRegex = builder.expectCharacters(endMarker).buildRegExp();
   // act
-  const match = Array.from(markedInput.matchAll(markedRegex))
+  const allMatchGroups = Array.from(input.matchAll(regex));
-    .filter((matches) => matches.length > 1)
-    .map((matches) => matches[1])
-    .filter(Boolean)
-    .join();
   // assert
-  expect(match).to.equal(expectedMatch);
+  const actualMatches = allMatchGroups
+    .filter((matches) => matches.length > matchGroupIndex)
+    .map((matches) => matches[matchGroupIndex])
+    .filter(Boolean) // matchAll returns `""` for full matches, `null` for capture groups
+    .flat();
+  const actualCombinedMatches = actualMatches.length ? actualMatches.join('') : undefined;
+  expect(actualCombinedMatches).equal(
+    expectedCombinedMatches,
+    [
+      '\n\n---',
+      'Expected combined matches:',
+      getTestDataText(expectedCombinedMatches),
+      'Actual combined matches:',
+      getTestDataText(actualCombinedMatches),
+      'Input:',
+      getTestDataText(input),
+      'Regex:',
+      getTestDataText(regex.toString()),
+      'All match groups:',
+      getTestDataText(JSON.stringify(allMatchGroups)),
+      `Match index in group: ${matchGroupIndex}`,
+      '---\n\n',
+    ].join('\n'),
+  );
+}
+
+function getTestDataText(data: string | undefined): string {
+  const outputPrefix = '\t> ';
+  if (data === undefined) {
+    return `${outputPrefix}undefined (no matches)`;
+  }
+  const getLiteralString = (text: string) => JSON.stringify(text).slice(1, -1);
+  const text = `${outputPrefix}\`${getLiteralString(data)}\``;
+  return text;
 }

tests/unit/application/Parser/Script/Compiler/Expressions/SyntaxParsers/SyntaxParserTestsRunner.ts

@@ -4,25 +4,26 @@ import { IExpressionParser } from '@/application/Parser/Script/Compiler/Expressi
 import { FunctionCallArgumentCollectionStub } from '@tests/unit/shared/Stubs/FunctionCallArgumentCollectionStub';
 import { ExpressionEvaluationContextStub } from '@tests/unit/shared/Stubs/ExpressionEvaluationContextStub';
 import { PipelineCompilerStub } from '@tests/unit/shared/Stubs/PipelineCompilerStub';
+import { scrambledEqual } from '@/application/Common/Array';
 
 export class SyntaxParserTestsRunner {
   constructor(private readonly sut: IExpressionParser) {
   }
 
-  public expectPosition(...testCases: IExpectPositionTestCase[]) {
+  public expectPosition(...testCases: ExpectPositionTestScenario[]) {
     for (const testCase of testCases) {
       it(testCase.name, () => {
         // act
         const expressions = this.sut.findExpressions(testCase.code);
         // assert
         const actual = expressions.map((e) => e.position);
-        expect(actual).to.deep.equal(testCase.expected);
+        expect(scrambledEqual(actual, testCase.expected));
       });
     }
     return this;
   }
 
-  public expectNoMatch(...testCases: INoMatchTestCase[]) {
+  public expectNoMatch(...testCases: NoMatchTestScenario[]) {
     this.expectPosition(...testCases.map((testCase) => ({
       name: testCase.name,
       code: testCase.code,
@@ -30,7 +31,7 @@ export class SyntaxParserTestsRunner {
     })));
   }
 
-  public expectResults(...testCases: IExpectResultTestCase[]) {
+  public expectResults(...testCases: ExpectResultTestScenario[]) {
     for (const testCase of testCases) {
       it(testCase.name, () => {
         // arrange
@@ -47,7 +48,21 @@ export class SyntaxParserTestsRunner {
     return this;
   }
 
-  public expectPipeHits(data: IExpectPipeHitTestData) {
+  public expectThrows(...testCases: ExpectThrowsTestScenario[]) {
+    for (const testCase of testCases) {
+      it(testCase.name, () => {
+        // arrange
+        const { expectedError } = testCase;
+        // act
+        const act = () => this.sut.findExpressions(testCase.code);
+        // assert
+        expect(act).to.throw(expectedError);
+      });
+    }
+    return this;
+  }
+
+  public expectPipeHits(data: ExpectPipeHitTestScenario) {
     for (const validPipePart of PipeTestCases.ValidValues) {
       this.expectHitPipePart(validPipePart, data);
     }
@@ -56,7 +71,7 @@ export class SyntaxParserTestsRunner {
     }
   }
 
-  private expectHitPipePart(pipeline: string, data: IExpectPipeHitTestData) {
+  private expectHitPipePart(pipeline: string, data: ExpectPipeHitTestScenario) {
     it(`"${pipeline}" hits`, () => {
       // arrange
       const expectedPipePart = pipeline.trim();
@@ -73,14 +88,14 @@ export class SyntaxParserTestsRunner {
       // assert
       expect(expressions).has.lengthOf(1);
       expect(pipelineCompiler.compileHistory).has.lengthOf(1);
-      const actualPipeNames = pipelineCompiler.compileHistory[0].pipeline;
+      const actualPipePart = pipelineCompiler.compileHistory[0].pipeline;
       const actualValue = pipelineCompiler.compileHistory[0].value;
-      expect(actualPipeNames).to.equal(expectedPipePart);
+      expect(actualPipePart).to.equal(expectedPipePart);
       expect(actualValue).to.equal(data.parameterValue);
     });
   }
 
-  private expectMissPipePart(pipeline: string, data: IExpectPipeHitTestData) {
+  private expectMissPipePart(pipeline: string, data: ExpectPipeHitTestScenario) {
     it(`"${pipeline}" misses`, () => {
       // arrange
       const args = new FunctionCallArgumentCollectionStub()
@@ -98,42 +113,51 @@ export class SyntaxParserTestsRunner {
     });
   }
 }
-interface IExpectResultTestCase {
+
-  name: string;
+interface ExpectResultTestScenario {
-  code: string;
+  readonly name: string;
-  args: (builder: FunctionCallArgumentCollectionStub) => FunctionCallArgumentCollectionStub;
+  readonly code: string;
-  expected: readonly string[];
+  readonly args: (
+    builder: FunctionCallArgumentCollectionStub,
+  ) => FunctionCallArgumentCollectionStub;
+  readonly expected: readonly string[];
 }
 
-interface IExpectPositionTestCase {
+interface ExpectThrowsTestScenario {
-  name: string;
+  readonly name: string;
-  code: string;
+  readonly code: string;
-  expected: readonly ExpressionPosition[];
+  readonly expectedError: string;
 }
 
-interface INoMatchTestCase {
+interface ExpectPositionTestScenario {
-  name: string;
+  readonly name: string;
-  code: string;
+  readonly code: string;
+  readonly expected: readonly ExpressionPosition[];
 }
 
-interface IExpectPipeHitTestData {
+interface NoMatchTestScenario {
-  codeBuilder: (pipeline: string) => string;
+  readonly name: string;
-  parameterName: string;
+  readonly code: string;
-  parameterValue: string;
+}
+
+interface ExpectPipeHitTestScenario {
+  readonly codeBuilder: (pipeline: string) => string;
+  readonly parameterName: string;
+  readonly parameterValue: string;
 }
 
 const PipeTestCases = {
   ValidValues: [
     // Single pipe with different whitespace combinations
-    ' | pipe1', ' |pipe1', '|pipe1', ' |pipe1', '   |   pipe1',
+    ' | pipe', ' |pipe', '|pipe', ' |pipe', '   |   pipe',
 
     // Double pipes with different whitespace combinations
-    ' | pipe1 | pipe2', '| pipe1|pipe2', '|pipe1|pipe2', ' |pipe1 |pipe2', '| pipe1 | pipe2| pipe3 |pipe4',
+    ' | pipeFirst | pipeSecond', '| pipeFirst|pipeSecond', '|pipeFirst|pipeSecond', ' |pipeFirst |pipeSecond', '| pipeFirst | pipeSecond| pipeThird |pipeFourth',
-
-    // Wrong cases, but should match anyway and let pipelineCompiler throw errors
-    '| pip€', '| pip{e} ',
   ],
   InvalidValues: [
-    ' pipe1  |pipe2', ' pipe1',
+    ' withoutPipeBefore  |pipe', ' withoutPipeBefore',
+
+    // It's OK to match them (move to valid values if needed) to let compiler throw instead.
+    '| pip€', '| pip{e} ', '| pipeWithNumber55', '| pipe with whitespace',
   ],
 };

tests/unit/application/Parser/Script/Compiler/Expressions/SyntaxParsers/WithParser.spec.ts

@@ -7,15 +7,15 @@ import { SyntaxParserTestsRunner } from './SyntaxParserTestsRunner';
 describe('WithParser', () => {
   const sut = new WithParser();
   const runner = new SyntaxParserTestsRunner(sut);
-  describe('finds as expected', () => {
+  describe('correctly identifies `with` syntax', () => {
     runner.expectPosition(
       {
-        name: 'when no scope is not used',
+        name: 'when no context variable is not used',
         code: 'hello {{ with $parameter }}no usage{{ end }} here',
         expected: [new ExpressionPosition(6, 44)],
       },
       {
-        name: 'when scope is used',
+        name: 'when context variable is used',
         code: 'used here ({{ with $parameter }}value: {{.}}{{ end }})',
         expected: [new ExpressionPosition(11, 53)],
       },
@@ -25,38 +25,70 @@ describe('WithParser', () => {
         expected: [new ExpressionPosition(7, 51), new ExpressionPosition(61, 99)],
       },
       {
-        name: 'tolerate lack of whitespaces',
+        name: 'when nested',
+        code: 'outer: {{ with $outer }}outer value with context variable: {{ . }}, inner: {{ with $inner }}inner value{{ end }}.{{ end }}',
+        expected: [
+          /* outer: */ new ExpressionPosition(7, 122),
+          /* inner: */ new ExpressionPosition(77, 112),
+        ],
+      },
+      {
+        name: 'whitespaces: tolerate lack of whitespaces',
         code: 'no whitespaces {{with $parameter}}value: {{ . }}{{end}}',
         expected: [new ExpressionPosition(15, 55)],
       },
       {
-        name: 'match multiline text',
+        name: 'newlines: match multiline text',
         code: 'non related line\n{{ with $middleLine }}\nline before value\n{{ . }}\nline after value\n{{ end }}\nnon related line',
         expected: [new ExpressionPosition(17, 92)],
       },
+      {
+        name: 'newlines: does not match newlines before',
+        code: '\n{{ with $unimportant }}Text{{ end }}',
+        expected: [new ExpressionPosition(1, 37)],
+      },
+      {
+        name: 'newlines: does not match newlines after',
+        code: '{{ with $unimportant }}Text{{ end }}\n',
+        expected: [new ExpressionPosition(0, 36)],
+      },
+    );
+  });
+  describe('throws with incorrect `with` syntax', () => {
+    runner.expectThrows(
+      {
+        name: 'incorrect `with`: whitespace after dollar sign inside `with` statement',
+        code: '{{with $ parameter}}value: {{ . }}{{ end }}',
+        expectedError: 'Context variable before `with` statement.',
+      },
+      {
+        name: 'incorrect `with`: whitespace before dollar sign inside `with` statement',
+        code: '{{ with$parameter}}value: {{ . }}{{ end }}',
+        expectedError: 'Context variable before `with` statement.',
+      },
+      {
+        name: 'incorrect `with`: missing `with` statement',
+        code: '{{ when $parameter}}value: {{ . }}{{ end }}',
+        expectedError: 'Context variable before `with` statement.',
+      },
+      {
+        name: 'incorrect `end`: missing `end` statement',
+        code: '{{ with $parameter}}value: {{ . }}{{ fin }}',
+        expectedError: 'Missing `end` statement, forgot `{{ end }}?',
+      },
+      {
+        name: 'incorrect `end`: used without `with`',
+        code: 'Value {{ end }}',
+        expectedError: 'Redundant `end` statement, missing `with`?',
+      },
+      {
+        name: 'incorrect "context variable": used without `with`',
+        code: 'Value: {{ . }}',
+        expectedError: 'Context variable before `with` statement.',
+      },
     );
   });
   describe('ignores when syntax is wrong', () => {
-    describe('ignores expression if "with" syntax is wrong', () => {
-      runner.expectNoMatch(
-        {
-          name: 'does not tolerate whitespace after with',
-          code: '{{with $ parameter}}value: {{ . }}{{ end }}',
-        },
-        {
-          name: 'does not tolerate whitespace before dollar',
-          code: '{{ with$parameter}}value: {{ . }}{{ end }}',
-        },
-        {
-          name: 'wrong text at scope end',
-          code: '{{ with$parameter}}value: {{ . }}{{ fin }}',
-        },
-        {
-          name: 'wrong text at expression start',
-          code: '{{ when $parameter}}value: {{ . }}{{ end }}',
-        },
-      );
-    });
     describe('does not render argument if substitution syntax is wrong', () => {
       runner.expectResults(
         {
@@ -83,8 +115,9 @@ describe('WithParser', () => {
       );
     });
   });
-  describe('renders scope conditionally', () => {
+  describe('scope rendering', () => {
-    describe('does not render scope if argument is undefined', () => {
+    describe('conditional rendering based on argument value', () => {
+      describe('does not render scope', () => {
         runner.expectResults(
           ...getAbsentStringTestCases().map((testCase) => ({
             name: `does not render when value is "${testCase.valueName}"`,
@@ -101,8 +134,21 @@ describe('WithParser', () => {
           },
         );
       });
-    describe('render scope when variable has value', () => {
+      describe('renders scope', () => {
         runner.expectResults(
+          ...getAbsentStringTestCases().map((testCase) => ({
+            name: `does not render when value is "${testCase.valueName}"`,
+            code: '{{ with $parameter }}dark{{ end }} ',
+            args: (args) => args
+              .withArgument('parameter', testCase.absentValue),
+            expected: [''],
+          })),
+          {
+            name: 'does not render when argument is not provided',
+            code: '{{ with $parameter }}dark{{ end }}',
+            args: (args) => args,
+            expected: [''],
+          },
           {
             name: 'renders scope even if value is not used',
             code: '{{ with $parameter }}Hello world!{{ end }}',
@@ -131,6 +177,11 @@ describe('WithParser', () => {
               .withArgument('letterL', 'l'),
             expected: ['Hello world!'],
           },
+        );
+      });
+    });
+    describe('whitespace handling inside scope', () => {
+      runner.expectResults(
         {
           name: 'renders value in multi-lined text',
           code: '{{ with $middleLine }}line before value\n{{ . }}\nline after value{{ end }}',
@@ -145,11 +196,6 @@ describe('WithParser', () => {
             .withArgument('middleLine', 'value line'),
           expected: ['line before value\nvalue line\nline after value'],
         },
-      );
-    });
-  });
-  describe('ignores trailing and leading whitespaces and newlines inside scope', () => {
-    runner.expectResults(
         {
           name: 'does not render trailing whitespace after value',
           code: '{{ with $parameter }}{{ . }}! {{ end }}',
@@ -172,15 +218,49 @@ describe('WithParser', () => {
           expected: ['Hello world!'],
         },
         {
-        name: 'does not render leading whitespace before value',
+          name: 'does not render leading whitespaces before value',
           code: '{{ with $parameter }}  {{ . }}!{{ end }}',
           args: (args) => args
             .withArgument('parameter', 'Hello world'),
           expected: ['Hello world!'],
         },
+        {
+          name: 'does not render leading newline and whitespaces before value',
+          code: '{{ with $parameter }}\r\n  {{ . }}!{{ end }}',
+          args: (args) => args
+            .withArgument('parameter', 'Hello world'),
+          expected: ['Hello world!'],
+        },
       );
     });
-  describe('compiles pipes in scope as expected', () => {
+    describe('nested with statements', () => {
+      runner.expectResults(
+        {
+          name: 'renders nested with statements correctly',
+          code: '{{ with $outer }}Outer: {{ with $inner }}Inner: {{ . }}{{ end }}, Outer again: {{ . }}{{ end }}',
+          args: (args) => args
+            .withArgument('outer', 'OuterValue')
+            .withArgument('inner', 'InnerValue'),
+          expected: [
+            'Inner: InnerValue',
+            'Outer: {{ with $inner }}Inner: {{ . }}{{ end }}, Outer again: OuterValue',
+          ],
+        },
+        {
+          name: 'renders nested with statements with context variables',
+          code: '{{ with $outer }}{{ with $inner }}{{ . }}{{ . }}{{ end }}{{ . }}{{ end }}',
+          args: (args) => args
+            .withArgument('outer', 'O')
+            .withArgument('inner', 'I'),
+          expected: [
+            'II',
+            '{{ with $inner }}{{ . }}{{ . }}{{ end }}O',
+          ],
+        },
+      );
+    });
+  });
+  describe('pipe behavior', () => {
     runner.expectPipeHits({
       codeBuilder: (pipeline) => `{{ with $argument }} {{ .${pipeline}}} {{ end }}`,
       parameterName: 'argument',