Cost of Data Collection

There are usually constraints that we want to apply to the data that is stored in our application's database. Our application shouldn't accept notes that have a missing or empty content property. The validity of the note is checked in the route handler:

app.post('/api/notes', (request, response) => {
  const body = request.body
  if (body.content === undefined) {    return response.status(400).json({ error: 'content missing' })  }
  // ...
})

If the note does not have the content property, we respond to the request with the status code 400 bad request.

One smarter way of validating the format of the data before it is stored in the database is to use the validation functionality available in Mongoose.

We can define specific validation rules for each field in the schema:

const noteSchema = new mongoose.Schema({
  content: {    type: String,    minLength: 5,    required: true  },  important: Boolean
})

The content field is now required to be at least five characters long and it is set as required, meaning that it can not be missing. We have not added any constraints to the important field, so its definition in the schema has not changed.

The minLength and required validators are built-in and provided by Mongoose. The Mongoose custom validator functionality allows us to create new validators if none of the built-in ones cover our needs.

If we try to store an object in the database that breaks one of the constraints, the operation will throw an exception. Let's change our handler for creating a new note so that it passes any potential exceptions to the error handler middleware:

app.post('/api/notes', (request, response, next) => {  const body = request.body

  const note = new Note({
    content: body.content,
    important: body.important || false,
    date: new Date(),
  })

  note.save()
    .then(savedNote => {
      response.json(savedNote)
    })
    .catch(error => next(error))})

Let's expand the error handler to deal with these validation errors:

const errorHandler = (error, request, response, next) => {
  console.error(error.message)

  if (error.name === 'CastError') {
    return response.status(400).send({ error: 'malformatted id' })
  } else if (error.name === 'ValidationError') {    return response.status(400).json({ error: error.message })  }

  next(error)
}

When validating an object fails, we return the following default error message from Mongoose:

We notice that the backend has now a problem: validations are not done when editing a note. The documentation explains what is the problem, validations are not run by default when findOneAndUpdate is executed.

The fix is easy. Let us also reformulate the route code a bit:

app.put('/api/notes/:id', (request, response, next) => {
  const { content, important } = request.body
  Note.findByIdAndUpdate(
    request.params.id, 
    { content, important },    { new: true, runValidators: true, context: 'query' }  ) 
    .then(updatedNote => {
      response.json(updatedNote)
    })
    .catch(error => next(error))
})

Deploying the database backend to production

The application should work almost as-is in Fly.io/Render. We do have to generate a new production build of the frontend since changes thus far were only on our backend.

The environment variables defined in dotenv will only be used when the backend is not in production mode, i.e. Fly.io or Render.

For production, we have to set the database URL in the service that is hosting our app.

In Fly.io that is done fly secrets set:

fly secrets set MONGODB_URI='mongodb+srv://fullstack:<password>@cluster0.o1opl.mongodb.net/noteApp?retryWrites=true&w=majority'

When the app is being developed, it is more than likely that something fails. Eg. when I deployed my app for the first time with the database, not a single note was seen:

The network tab of the browser console revealed that fetching the notes did not succeed, the request just remained for a long time in the pending state until it failed with statuscode 502.

The browser console has to be open all the time!

It is also vital to follow continuously the server logs. The problem became obvious when the logs were opened with fly logs:

The database url was undefined, so the command fly secrets set MONGODB_URI was forgotten.

When using Render, the database url is given by definig the proper env in the dashboard:

The Render Dashboard shows the server logs:

You can find the code for our current application in its entirety in the part3-5 branch of this GitHub repository.

Lint

Before we move on to the next part, we will take a look at an important tool called lint. Wikipedia says the following about lint:

Generically, lint or a linter is any tool that detects and flags errors in programming languages, including stylistic errors. The term lint-like behavior is sometimes applied to the process of flagging suspicious language usage. Lint-like tools generally perform static analysis of source code.

In compiled statically typed languages like Java, IDEs like NetBeans can point out errors in the code, even ones that are more than just compile errors. Additional tools for performing static analysis like checkstyle, can be used for expanding the capabilities of the IDE to also point out problems related to style, like indentation.

In the JavaScript universe, the current leading tool for static analysis aka. "linting" is ESlint.

Let's install ESlint as a development dependency to the backend project with the command:

npm install eslint --save-dev

After this we can initialize a default ESlint configuration with the command:

npx eslint --init

We will answer all of the questions:

The configuration will be saved in the .eslintrc.js file:

module.exports = {
    'env': {
        'commonjs': true,
        'es2021': true,
        'node': true    },
    'extends': 'eslint:recommended',
    'parserOptions': {
        'ecmaVersion': 'latest'
    },
    'rules': {
        'indent': [
            'error',
            4
        ],
        'linebreak-style': [
            'error',
            'unix'
        ],
        'quotes': [
            'error',
            'single'
        ],
        'semi': [
            'error',
            'never'
        ]
    }
}

Let's immediately change the rule concerning indentation, so that the indentation level is two spaces.

"indent": [
    "error",
    2
],

Inspecting and validating a file like index.js can be done with the following command:

npx eslint index.js

It is recommended to create a separate npm script for linting:

{
  // ...
  "scripts": {
    "start": "node index.js",
    "dev": "nodemon index.js",
    // ...
    "lint": "eslint ."  },
  // ...
}

Now the npm run lint command will check every file in the project.

Also the files in the build directory get checked when the command is run. We do not want this to happen, and we can accomplish this by creating an .eslintignore file in the project's root with the following contents:

build

This causes the entire build directory to not be checked by ESlint.

Lint has quite a lot to say about our code:

Let's not fix these issues just yet.

A better alternative to executing the linter from the command line is to configure a eslint-plugin to the editor, that runs the linter continuously. By using the plugin you will see errors in your code immediately. You can find more information about the Visual Studio ESLint plugin here.

The VS Code ESlint plugin will underline style violations with a red line:

This makes errors easy to spot and fix right away.

ESlint has a vast array of rules that are easy to take into use by editing the .eslintrc.js file.

Let's add the eqeqeq rule that warns us, if equality is checked with anything but the triple equals operator. The rule is added under the rules field in the configuration file.

{
  // ...
  'rules': {
    // ...
   'eqeqeq': 'error',
  },
}

While we're at it, let's make a few other changes to the rules.

Let's prevent unnecessary trailing spaces at the ends of lines, let's require that there is always a space before and after curly braces, and let's also demand a consistent use of whitespaces in the function parameters of arrow functions.

{
  // ...
  'rules': {
    // ...
    'eqeqeq': 'error',
    'no-trailing-spaces': 'error',
    'object-curly-spacing': [
        'error', 'always'
    ],
    'arrow-spacing': [
        'error', { 'before': true, 'after': true }
    ]
  },
}

Our default configuration takes a bunch of predetermined rules into use from eslint:recommended:

'extends': 'eslint:recommended',

This includes a rule that warns about console.log commands. Disabling a rule can be accomplished by defining its "value" as 0 in the configuration file. Let's do this for the no-console rule in the meantime.

{
  // ...
  'rules': {
    // ...
    'eqeqeq': 'error',
    'no-trailing-spaces': 'error',
    'object-curly-spacing': [
        'error', 'always'
    ],
    'arrow-spacing': [
        'error', { 'before': true, 'after': true }
    ],
    'no-console': 0  },
}

NB when you make changes to the .eslintrc.js file, it is recommended to run the linter from the command line. This will verify that the configuration file is correctly formatted:

If there is something wrong in your configuration file, the lint plugin can behave quite erratically.

Many companies define coding standards that are enforced throughout the organization through the ESlint configuration file. It is not recommended to keep reinventing the wheel over and over again, and it can be a good idea to adopt a ready-made configuration from someone else's project into yours. Recently many projects have adopted the Airbnb Javascript style guide by taking Airbnb's ESlint configuration into use.

You can find the code for our current application in its entirety in the part3-6 branch of this GitHub repository.