Data Validation & Sanitisation with AdonisJS | Full-Stack Google Contacts Clone with AdonisJS (Node.js) and Quasar Framework (Vue.js)

Data Validation & Sanitisation with AdonisJS | Full-Stack Google Contacts Clone with AdonisJS (Node.js) and Quasar Framework (Vue.js)

Ndianabasi Udonkang's photo
Ndianabasi Udonkang
·

12 min read

Table of contents

In the last lesson, we learnt how to create a contact by persisting data sent from the body of our API request. The API request was done via Postman. The data was stored on the contacts table of our database as sent from Postman without any form of validation and sanitisation. That is a dangerous situation because a malicious actor could deliver and store dangerous contents on our database which can lead to various attacks such as SQL injections and Persisted Cross-Site Scripting. I wrote a 4300 word article on these attack vectors and how to mitigate them. You should read the article as a web developer. It is very important to understand the implications of your development decisions as a developer because they can make or mar your web applications.

In this lesson, we will learn how to validate and sanitise data sent to our API server before persisting the data to the database.

Let's start by creating a new branch for our repo:

# Make sure you are within your project
git checkout -b 13-validating-and-sanitise-data-before-storage

Let's make some few changes to our Contact model before we proceed with the validation and sanitisation task.

Firstly, open the api/app/Models/Contact.ts file. Refer to this snapshot. We will change the birthday property of the Contact model from string to a Luxon DateTime object. This is because we want to change the birthday column on our database from being a string (VARCHAR) data type to a DATE data type. We will also attach the @column.date() decorator to the birthday property instead of an ordinary @column() decorator. The @column.date() decorator enables the automatic conversion of the Luxon DateTime objects to string during serialisation to JSON or conversion of Luxon DateTime objects to JavaScript Date object before persistence. We supply the options: { autoCreate: false, autoUpdate: false } to the @column.date() decorator so that the birthday field is not auto-created or auto-updated from the current time/date during creation or update of the contact .

-  @column()
-  public birthday?: string | null | undefined
+  @column.date({ autoCreate: false, autoUpdate: false })
+  public birthday?: DateTime | null | undefined

We must also make changes to the structure of our contacts table. Particularly, we will alter the type of the birthday column from string/VARCHAR to DATE. To do this in AdonisJS, we create need to create a migration file which will hold the changes we want to make in the database.

cd api # If you are not in the `api` folder, change into iti
node ace make:migration ChangeDataTypeForBirthdayToDate --table=contacts

The above command creates a migration file which will alter the contacts table. This argument, ChangeDataTypeForBirthdayToDate, passed to the make:migration command will be used in creating the filename and serves as an easily reminder of what the migration file was meant for. It is recommended that you make the argument descriptive when creating a migration file for altering a table.

Refer to this snapshot. Copy-and-paste the entire content of the snapshot file into the newly-created migration file. Check the api/database/migrations directory. It should have a name like this: xxxxxxxxxxxxxx_change_data_type_for_birthday_to_date.ts

In the migration file:

  1. The line:
    table.date('birthday').alter().index()
    Is used to alter the column from a string data type to a date data type. You must add the modifier alter() else the migration process will attempt to create a new column with the same name birthday and, obviously, it will fail.
  2. The lines: 
    table.dropIndex('birthday')
table.string('birthday').alter()
    Are used to reverse the migration in the event we will to rollback to previous state where the birthday column was a string data type. To do this, first, we must drop the index we applied on the birthday column and then proceed to alter the column to a string data type.

Validating and Sanitising the Contact Data

Now, let's carry on with the main task for this lesson. You might want to study the introduction to AdonisJS validation first.

First, we need to create a validation class/file which will hold the schema for defining the validation rules. AdonisJS has a command for this. Run:

node ace make:validator Contact
# This will create a validation file named `app/Validators/ContactValidator.ts`

Copy the content of this snapshot file and paste in the app/Validators/ContactValidator.ts file you just created. Let's discuss what's going on in the file.

  1. At Line 1, we must import the schema and rules objects from @ioc:Adonis/Core/Validator package.

  2. From Line 7, we create a public instance property named schema. The schema property will hold the schema object created with the schema.create() method. The schema.create(options) method takes an options object which lists all the properties in our JSON data - in no particular order. The structure of the options for the create method must match the structure of your JSON data. For example the JSON for storing a contacts will look like this:

    {
 "id": "309b20ac-bbcd-4268-b01f-43770527540d",
 "firstName": "Hammad",
 "surname": "Pulham",
 "company": null,
 "jobTitle": null,
 "email1": "hpulham0@si.edu",
 "email2": null,
 "phoneNumber1": "+420 (767) 548-7576",
 "phoneNumber2": "+86 (442) 396-1670",
 "country": null,
 "streetAddressLine1": null,
 "streetAddressLine2": null,
 "city": null,
 "postCode": "240022",
 "state": null,
 "birthday": "1970-03-13",
 "website": "http://boston.com",
 "notes": null,
},

    The options object must reflect the JSON data as shown below:

    {
 firstName: schema.string({ escape: true, trim: true }, [rules.maxLength(30)]),
 surname: schema.string({ escape: true, trim: true }, [rules.maxLength(30)]),
 company: schema.string.optional({ escape: true, trim: true }),
 jobTitle: schema.string.optional({ escape: true, trim: true }),
 email1: schema.string({ escape: true, trim: true }, [
   rules.email(),
   rules.unique({ table: 'contacts', column: 'email1', caseInsensitive: false }),
 ]),
 email2: schema.string.optional({ escape: true, trim: true }, [rules.email()]),
 phoneNumber1: schema.string({ escape: true, trim: true }, [rules.maxLength(20)]),
 phoneNumber2: schema.string.optional({ escape: true, trim: true }, [rules.maxLength(20)]),
 country: schema.string.optional({ escape: true, trim: true }, [rules.maxLength(20)]),
 streetAddressLine1: schema.string.optional({ escape: true, trim: true }),
 streetAddressLine2: schema.string.optional({ escape: true, trim: true }),
 city: schema.string.optional({ escape: true, trim: true }),
 postCode: schema.string.optional({ escape: true, trim: true }),
 state: schema.string.optional({ escape: true, trim: true }),
 birthday: schema.date.optional({ format: 'yyyy-MM-dd' }, [rules.before('today')]),
 website: schema.string.optional({ trim: true }, [
   rules.url({
     protocols: ['http', 'https'],
     requireHost: true,
   }),
 ]),
 notes: schema.string.optional({ escape: true, trim: true }),
}

    The value of each property in the options object must be a schema type such as string, boolean, number, date, enum/enumSet, file, array, or object. Read more about schema types here. For example: the firstName property is assigned a schema.string type which is actually a method of the schema object. Each schema type can be chained to an optional() modifier which marks the property as being optional. Schema types with no optional() modifier are required by default. The string method takes two argument: options and rules. The options argument of a schema type is usually used for providing the sanitisation options for the property. In this case, we want to escape any value assigned to the firstName property. Escaping values means that we want to convert any unsafe character into there ASCII representations so that if they were meant for malicious intents, they will become ineffective. (Again, I encourage you to read this detailed article: How to Avoid SQL Injections, XSS Attacks, and File Upload Attacks in your Web Application.) You the value for firstName included a % symbol, it will be converted to %. The rules argument of the schema type is used to provide the validation rules for the schema type. Rules must be wrapped in an array. For example, for firstName, we are setting the maxLength rule to 30 characters. For the email1 and email2 properties, we set the email validation rules so that only valid email addresses will be allowed. The trim properties in the schema type options is used to remove all whitespaces at the beginning and end of the values.

    The unique validation rule assigned to the email1 property is worthy of special explanation. Here, we are enforcing uniqueness of email addresses for column email1 on the contacts table. We are assuming here that this app will be used for personal purposes and email addresses should be unique. The unique rule take some arguments: table, column, caseInsensitive, etc. See the full description of the unique rule. The table property provides the table which contains the column where we want to enforce the uniqueness constraint. The caseInsensitive property indicates that the values stored on the column are case insensitive. Meaning that Abc@Example.com will be the same as abc@example.com. It should be mentioned that while you can set the email1 column as unique while creating the table, it is important to validate for the same uniqueness separately via a validator rule.

    The birthday property takes a date schema type with a single property in the options: format. The date schema type will convert the date string received from the JSON to a Luxon DateTimeobject. Theformatobject is used to specify the format of the date expected in the JSON. For our purpose, the Date component on the frontend will send the date in theyyyy-MM-ddformat. We also applied a rule which limits the dates to all dates beforetoday`. Read more about the date schema type.

    The website property is assigned a string schema type with a url rule. The url rule enforces that either http or https protocol must be provided in the website value. It also enforces that the host portion of the URL is required. This means that you cannot provided this: /about-usas a valid URL. See all the options for theurl` rule here.

  3. We must also define a public messages instance property which will hold all the validation messages which will be sent via API response if the validation fails. Read more about validation messages.

Let's proceed to integrate the ContactValidator class into the store method of the ContactsController class.

Integrating a Validator into a Controller

The validator class on its own will do nothing. For it to work, it has to be initialised and called. For our purpose, we want to call it before the contact is being saved. If you recall from our api/start/routes.ts file, the route for creating a contact is defined as shown below:

Route.post('/contacts', 'ContactsController.store')

So, we need to modify the store method within the ContactsController file. Open api/app/Controllers/Http/ContactsController.ts. Refer to this snapshot file for changes to be made.

  1. At Line 3, we will import the Logger instance from @ioc:Adonis/Core/Logger package. It will be used for error logging in the catch statement.

  2. At Line 4, we will import the ContactValidator class from App/Validators/ContactValidator file.

  3. We will modify the body of the store method by wrapping the statements in try...catch statements.

  4. At Line 11, we call the request.validate method and provide the ContactValidator class we imported as the argument of the validate method. The result of the validator is stored in the payload constant. The payload constant will contain our validated and sanitised properties from our JSON data provided in the request body. The validate method instantiates the class internally and makes use of the schema and messages properties in the ContactValidator class to perform the validator. If the validation fails, an error will be thrown using the relevant validation messages defined. We will manually catch these validation error in the catch statement and return the error as a response to the frontend. Read more here.

  5. From Line 25, we destructure the payload to obtain the properties we want to save. Other parts of the store method remain the same.

  6. In the catch statement - from Line 73, we log any error in the store method to the console. Then return the response with the error. First, we check if the error object contains a status property which represents the status code for the error. If there is a status property, that status code will be used in the response. For validation errors, the status code is 422. Learn more about API status codes. If the error contains no status codes, we will use 500 as a generic internal server error code in the response. We also check if the app is not used in production or error.status is 422. If not true, we do not include the error object in the error property. These are security measures to ensure that you do not spit out sensitive error contents to the frontend. As a developer, you should use the console to log and check error messages and avoid sending them to the frontend (unless you are not in production mode). You can also send errors via email for record purposes.

Testing the Validation with Postman

Let's test our ContactValidation validator by sending API requests through Postman.

  1. Open Postman. Ensure that you are in the Google Contacts Clone workspace. We had created a request named: Create Contact with URL POST /contacts in the previous lesson. We are still using the JSON below for the request.
{
    "firstName": "Hammad",
    "surname": "Pulham",
    "email1": "hpulham0@si.edu",
    "phoneNumber1": "+420 (767) 548-7576",
    "phoneNumber2": "+86 (442) 396-1670",
    "birthday": "1970-03-13",
    "website": "http://boston.com"
}

  1. Click on the Create Contact request to open it. Then click the Send button to send the request. If the contact in the JSON had been created before, you will get an error like this:

    {
 "message": "An error occurred while creating the contact.",
 "error": {
     "flashToSession": false,
     "messages": {
         "errors": [
             {
                 "rule": "unique",
                 "field": "email1",
                 "message": "Email 1 is already registered in your contacts"
             }
         ]
     }
 }
}

    Congratulations, your validation is working.

  2. Now, change the value for website to boston.com and resend the request. You will get an error like this:

    {
 "message": "An error occurred while creating the contact.",
 "error": {
     "flashToSession": false,
     "messages": {
         "errors": [
             {
                 "rule": "unique",
                 "field": "email1",
                 "message": "Email 1 is already registered in your contacts"
             },
             {
                 "rule": "url",
                 "field": "website",
                 "message": "Website is not valid"
             }
         ]
     }
 }
}

    Now, we have two validation messages.

  3. Remove the firstName field and resend the request. You should get an error like this:

    {
 "message": "An error occurred while creating the contact.",
 "error": {
     "flashToSession": false,
     "messages": {
         "errors": [
             {
                 "rule": "required",
                 "field": "firstName",
                 "message": "First Name is required"
             },
             {
                 "rule": "unique",
                 "field": "email1",
                 "message": "Email 1 is already registered in your contacts"
             },
             {
                 "rule": "url",
                 "field": "website",
                 "message": "Website is not valid"
             }
         ]
     }
 }
}

  4. Now, update the JSON body to the JSON below and resend the request:

    {
 "firstName": "Zechariah",
 "surname": "Pollak",
 "company": "Welch, Littel and Rowe",
 "jobTitle": "Account Executive",
 "email1": "zpollak1@blogtalkradio.com",
 "email2": "zpollak1@flickr.com",
 "phoneNumber1": "+66 (700) 444-4282",
 "phoneNumber2": "+237 (446) 364-2728",
 "country": "Thailand",
 "streetAddressLine1": "10327 Pond Pass",
 "streetAddressLine2": "1209 Vermont Drive",
 "city": "Trang",
 "state": null,
 "birthday": "1963-10-15",
 "website": "http://indiegogo.com",
 "notes": null
}

    A new contact should be created because all validations were successfully:

    {
 "message": "Contact was created",
 "data": {
     "id": "ckut90fru00013cvohnixd4g0",
     "first_name": "Zechariah",
     "surname": "Pollak",
     "company": "Welch, Littel and Rowe",
     "job_title": "Account Executive",
     "email1": "zpollak1@blogtalkradio.com",
     "email2": "zpollak1@flickr.com",
     "phone_number1": "+66 (700) 444-4282",
     "phone_number2": "+237 (446) 364-2728",
     "country": "Thailand",
     "street_address_line1": "10327 Pond Pass",
     "street_address_line2": "1209 Vermont Drive",
     "city": "Trang",
     "post_code": null,
     "state": null,
     "birthday": "1963-10-15",
     "website": "http://indiegogo.com",
     "notes": null,
     "created_at": "2021-10-16T04:36:48.000+01:00",
     "updated_at": "2021-10-16T04:36:48.000+01:00"
 }
}

Save the Create Contact request in Postman.

Difference between JSON and JavaScript Object.

The main difference is that you must wrap the properties of JSON object with double quotes (single quotes are not accepted). Example:

{
    "message": "Contact was created",
    "data": {
        "id": "ckut90fru00013cvohnixd4g0",
        "first_name": "Zechariah",
    }
}

While the property of a JavaScript object is only wrapped with double or single quotes when the property is a compound word. Example:

{
    'firstName.required': 'First Name is required',
    'firstName.maxLength': 'First Name should be maximum of {{options.maxLength}} characters.',
  }

This concludes our lesson. In the next lesson, we will learn how to update and delete a contact.

Save all your files, commit and merge with the master branch.

git add .
git commit -m "feat(api): complete validation of contacts data before storage"
git push origin 13-validating-and-sanitise-data-before-storage
git checkout master
git merge master 13-validating-and-sanitise-data-before-storage
git push origin master
AdonisJSQuasar FrameworkVue.jsNode.jsJavaScript