Add initial e2e testing with cypress

Add e2e tests with cypress using `vue add e2e-cypress`.

Vue CLI does not support creating typescript tests at this moment
(vuejs/vue-cli#1350).

.github/workflows/test.yaml

@@ -31,3 +31,7 @@ jobs:
       -
         name: Run integration tests
         run: npm run test:integration
+      -
+        name: Run e2e tests
+        if: always() # Run even if previous step fail
+        run: npm run test:e2e -- --headless

.gitignore

@@ -4,3 +4,6 @@ dist/
 .vscode
 #Electron-builder output
 /dist_electron
+# Cypress
+/tests/e2e/screenshots
+/tests/e2e/videos

README.md

@@ -49,6 +49,9 @@
 - Testing
   - Run unit tests: `npm run test:unit`
   - Run integration tests: `npm run test:integration`
+  - Run e2e (end-to-end) tests
+    - Interactive mode with GUI: `npm run test:e2e`
+    - Headless mode without GUI: `npm run test:e2e -- --headless`
   - Lint: `npm run lint`
 - **Desktop app**
   - Development: `npm run electron:serve`

cypress.json

@@ -0,0 +1,3 @@
+{
+  "pluginsFile": "tests/e2e/plugins/index.js"
+}

docs/tests.md

@@ -3,41 +3,59 @@
 - There are two different types of tests executed:
   1. [Unit tests](#unit-tests)
   2. [Integration tests](#integration-tests)
+  3. [End-to-end (E2E) tests](#e2e-tests)
+- All tests
+  - Uses [Mocha](https://mochajs.org/) and [Chai](https://www.chaijs.com/).
+  - Are written in files that includes `.spec` extension.
 - 💡 You can use path/module alias `@/tests` in import statements.
 
 ## Unit tests
 
-- Tests each component in isolation
-- Defined in [`./tests/unit`](./../tests/unit)
-- They follow same folder structure as [`./src`](./../src)
+- Tests each component in isolation.
+- Defined in [`./tests/unit`](./../tests/unit).
+- They follow same folder structure as [`./src`](./../src).
 
 ### Naming
 
-- Each test suite first describe the system under test
-  - E.g. tests for class `Application` is categorized under `Application`
-- Tests for specific methods are categorized under method name (if applicable)
-  - E.g. test for `run()` is categorized under `run`
+- Each test suite first describe the system under test.
+  - E.g. tests for class `Application` is categorized under `Application`.
+- Tests for specific methods are categorized under method name (if applicable).
+  - E.g. test for `run()` is categorized under `run`.
 
 ### Act, arrange, assert
 
-- Tests use act, arrange and assert (AAA) pattern when applicable
+- Tests use act, arrange and assert (AAA) pattern when applicable.
 - **Arrange**
-  - Should set up the test case
-  - Starts with comment line `// arrange`
+  - Should set up the test case.
+  - Starts with comment line `// arrange`.
 - **Act**
-  - Should cover the main thing to be tested
-  - Starts with comment line `// act`
+  - Should cover the main thing to be tested.
+  - Starts with comment line `// act`.
 - **Assert**
-  - Should elicit some sort of response
-  - Starts with comment line `// assert`
+  - Should elicit some sort of response.
+  - Starts with comment line `// assert`.
 
 ### Stubs
 
-- Stubs are defined in [`./tests/stubs`](./../tests/unit/stubs)
-- They implement dummy behavior to be functional
+- Stubs are defined in [`./tests/stubs`](./../tests/unit/stubs).
+- They implement dummy behavior to be functional.
 
 ## Integration tests
 
-- Tests functionality of a component in combination with others (not isolated)
-- Ensure dependencies to third parties work as expected
-- Defined in [`./tests/integration`](./../tests/integration)
+- Tests functionality of a component in combination with others (not isolated).
+- Ensure dependencies to third parties work as expected.
+- Defined in [`./tests/integration`](./../tests/integration).
+
+## E2E tests
+
+- Test the functionality and performance of a running application.
+- E2E tests are configured by vue plugin [`e2e-cypress`](https://github.com/vuejs/vue-cli/tree/dev/packages/@vue/cli-plugin-e2e-cypress#readme) for Vue CLI.
+- Names and folders are structured logically based on tests.
+- The structure is following:
+  - [`cypress.json`](./../cypress.json): Cypress configuration file.
+  - [`./tests/e2e/`](./../tests/e2e/): Base Cypress folder.
+    - [`/specs/`](./../tests/e2e/specs/): Test files, test are named with `.spec.js` extension.
+    - [`/plugins/index.js`](./../tests/e2e/plugins/index.js): Plugin file executed before project is loaded.
+    - [`/support/index.js`](./../tests/e2e/support/index.js): Support file, runs before every single spec file.
+    - *(Ignored)* `/videos`: Asset folder for videos taken during tests.
+    - *(Ignored)* `/screenshots`: Asset folder for Screenshots taken during tests.

package.json

@@ -9,6 +9,7 @@
     "build": "vue-cli-service build",
     "test:unit": "vue-cli-service test:unit",
     "test:integration": "vue-cli-service test:unit \"tests/integration/**/*.spec.ts\"",
+    "test:e2e": "vue-cli-service test:e2e",
     "lint": "npm run lint:vue && npm run lint:yaml && npm run lint:md && npm run lint:md:relative-urls && npm run lint:md:consistency",
     "electron:build": "vue-cli-service electron:build",
     "electron:serve": "vue-cli-service electron:serve",
@@ -48,6 +49,7 @@
     "@types/file-saver": "^2.0.3",
     "@types/mocha": "^9.0.0",
     "@vue/cli-plugin-babel": "^4.5.14",
+    "@vue/cli-plugin-e2e-cypress": "^4.5.14",
     "@vue/cli-plugin-typescript": "^4.5.14",
     "@vue/cli-plugin-unit-mocha": "^4.5.14",
     "@vue/cli-service": "^4.5.14",

tests/e2e/plugins/index.js

@@ -0,0 +1,25 @@
+/* eslint-disable arrow-body-style */
+// https://docs.cypress.io/guides/guides/plugins-guide.html
+
+// if you need a custom webpack configuration you can uncomment the following import
+// and then use the `file:preprocessor` event
+// as explained in the cypress docs
+// https://docs.cypress.io/api/plugins/preprocessors-api.html#Examples
+
+// /* eslint-disable import/no-extraneous-dependencies, global-require */
+// const webpack = require('@cypress/webpack-preprocessor')
+
+module.exports = (on, config) => {
+  // on('file:preprocessor', webpack({
+  //  webpackOptions: require('@vue/cli-service/webpack.config'),
+  //  watchOptions: {}
+  // }))
+
+  return Object.assign({}, config, {
+    fixturesFolder: 'tests/e2e/fixtures',
+    integrationFolder: 'tests/e2e/specs',
+    screenshotsFolder: 'tests/e2e/screenshots',
+    videosFolder: 'tests/e2e/videos',
+    supportFile: 'tests/e2e/support/index.js'
+  })
+}

tests/e2e/specs/initialization.spec.js

@@ -0,0 +1,39 @@
+describe('application is initialized as expected', () => {
+  it('loads title as expected', () => {
+    // act
+    cy.visit('/');
+    // assert
+    cy.contains('h1', 'privacy.sexy')
+  });
+  it('there are no console.error output', () => {
+    // act
+    // https://docs.cypress.io/faq/questions/using-cypress-faq#How-do-I-spy-on-console-log
+    cy.visit('/', {
+      onBeforeLoad(win) {
+        cy.stub(win.console, 'error').as('consoleError')
+      }
+    });
+    // assert
+    cy.get('@consoleError').should('have.not.been.called');
+  });
+  it('there are no console.warn output', () => {
+    // act
+    cy.visit('/', {
+      onBeforeLoad(win) {
+        cy.stub(win.console, 'warn').as('consoleWarn')
+      }
+    });
+    // assert
+    cy.get('@consoleWarn').should('have.not.been.called');
+  });
+  it('there are no console.log output', () => {
+    // act
+    cy.visit('/', {
+      onBeforeLoad(win) {
+        cy.stub(win.console, 'log').as('consoleLog')
+      }
+    });
+    // assert
+    cy.get('@consoleLog').should('have.not.been.called');
+  });
+});

tests/e2e/support/commands.js

@@ -0,0 +1,25 @@
+// ***********************************************
+// This example commands.js shows you how to
+// create various custom commands and overwrite
+// existing commands.
+//
+// For more comprehensive examples of custom
+// commands please read more here:
+// https://on.cypress.io/custom-commands
+// ***********************************************
+//
+//
+// -- This is a parent command --
+// Cypress.Commands.add("login", (email, password) => { ... })
+//
+//
+// -- This is a child command --
+// Cypress.Commands.add("drag", { prevSubject: 'element'}, (subject, options) => { ... })
+//
+//
+// -- This is a dual command --
+// Cypress.Commands.add("dismiss", { prevSubject: 'optional'}, (subject, options) => { ... })
+//
+//
+// -- This is will overwrite an existing command --
+// Cypress.Commands.overwrite("visit", (originalFn, url, options) => { ... })

tests/e2e/support/index.js

@@ -0,0 +1,20 @@
+// ***********************************************************
+// This example support/index.js is processed and
+// loaded automatically before your test files.
+//
+// This is a great place to put global configuration and
+// behavior that modifies Cypress.
+//
+// You can change the location of this file or turn off
+// automatically serving support files with the
+// 'supportFile' configuration option.
+//
+// You can read more here:
+// https://on.cypress.io/configuration
+// ***********************************************************
+
+// Import commands.js using ES2015 syntax:
+import './commands'
+
+// Alternatively you can use CommonJS syntax:
+// require('./commands')