Topic 15: node.js and MongoDB

Introduction

node.js works very well with MongoDB. As you saw last time, MongoDB uses a JavaScript-like syntax to query the database. This means that there is quite a close resemblance between MongoDB query syntax and querying a MongoDB database from Node.

Using MongoDB from node.js

There are a couple of options for using MongoDB from Node:

Use the standard mongodb module;
Use Mongoose. Mongoose is a higher-level API which makes the process of querying a MongoDB database simpler and friendlier. It also allows you to define schemas for your MongoDB collections; when an item is added, any fields not in the schema will be ignored.

Mongoose has a website here. The website contains extensive tutorials and documentation.

Connecting to the database

Here is how you connect to the database using Mongoose:

const mongoose = require('mongoose');

// Connect to a database named 'test' on the local machine (localhost)
mongoose.connect('mongodb://localhost/test');

// Run a callback as soon as the connection is open
mongoose.connection.once('open', ()=> {
    // do your queries in this callback...
});

Note how we open a connection to the 'test' database on the local machine (localhost) and specify a callback to run as soon as the connection has been opened. This is a similar approach to the MySQL API which we looked at last week.

Defining a model and a schema

Mongoose uses the concepts of models and schemas. A model is a JavaScript object acting as an interface to your collection; it's the same concept as the model in MVC. Here is an example of creating a model representing documents in a students collection:

const Student = mongoose.model('student', new mongoose.Schema(
        { firstname: String,
            lastname: String,
            course: String,
               grade: Number} ));

Our model object is Student. We create it using the mongoose.model() method which takes two arguments:

The MongoDB collection that it maps to, in the singular (i.e. if the collection name is students, we use student).
A mongoose.Schema object representing the schema. This is a schema in the normal database sense, i.e. a description of properties (fields) and data types. Here, we have four fields: name, course and grade. These are all Strings; we can also have Numbers. Note that by default the _id will be automatically allocated a unique ObjectId. See the documentation for the full list of data types for a schema.

Once we've created the model, we perform operations on the database using it.

Retrieving data from the database

Here is an example to find all students in the students collection using the Student model.

Student.find(
            (err,result)=> {
                if(err) {
                    console.log(`Error: ${err}`);
                } else {
                    console.log(result);
                }
            });

How is this working? We call the find() method on the model (note how this is similar to calling find() from the MongoDB console). Note how the find() method takes a callback to run once the results have been returned.

Note how this callback takes two parameters: err, an object representing an error (if there was one) and result, an array of all the MongoDB documents returned from find(). This callback pattern, an error object as the first parameter and the results as the second, is a common pattern in Node modules.

Here, we simply display the results in the console.

Retrieving certain records only

By supplying a search query to find() as the first argument (before the callback), in a similar way to queries in the Mongo console, we can retrieve certain records only. For example, to find all students on Software Engineering:

Student.find({course:'Software Engineering'},
            (err,result)=> {
                if(err) {
                    console.log(`Error: ${err}`);
                } else {
                    console.log(result);
                }
            });

and to find all students with a last name of Jones on Software Engineering:

Student.find({course:'Software Engineering', lastname:'Jones'},
            (err,result)=> {
                if(err) {
                    console.log(`Error: ${err}`);
                } else {
                    console.log(result);
                }
            });

We can also use special Mongo operations such as $lt, $gt, $in, $or etc. in our find() queries. For example, to find all students who have passed:

Student.find({grade: {$gte: 40}},
            (err,result)=>{
                res.send(result);
            });
    });

or, to find all students on Software Engineering or Computing:

Student.find({course: { $in: ['Software Engineering', 'Computing']}},
            (err,result)=>{
                res.send(result);
            });
});

Mongoose also gives us some shortcuts for searching. For example, if we want to find a particular document by ID (a common scenario), we can just use the findById() method (the ID is the number associated with an ObjectId):

Student.findById("5c0cf71d4d49560f7c678f08",
            (err,result)=>{
                res.send(result);
            });
    });

Adding a new document

To add a new document to a collection, we create an object of the model type and save() it. save()ing it saves it to the actual MongoDB collection. For example, to create a new student and save it to the collection:

const student = new Student ({ firstname:'Tom', 
                                lastname: 'Smith', 
                                course: 'Software Engineering',     
                                grade: 80});
student.save();

Note that if the collection associated with the model ('students' here) does not exist, saving the object to the database will create it.

Note that this will allocate a unique ObjectId for the document. We can obtain it using student._id.

Updating a document

Updating a document can be carried out by the update() group of methods of your model. There are different variants of update(), such as updateOne() which will only update the first matching document and updateMany() which will update all matching documents.

These take three arguments, the search criteria, the update operation and the callback. For example, this will update the grade of the first matching student with a first name of Tim and last name of Smith to 70.

Student.updateOne({firstname:'Tim', lastname:'Smith'}, {grade: 70},
           (err,result) =>
            {
                if(!err)
                {
                    if(result.nModified==1)
                    {
                        console.log("updated");
                    }
                    else
                    {
                        console.log("not updated - does student exist?");
                    }
                }
                else
                {
                    console.log(err);
                }
            }
);

Note how, for an update, the result object has an nModified field, storing the number of records that were modified. We can use this to test whether we successfully managed to update a document or not; if not, it likely means that there is no student called Tim Smith in the database.

Note that we do not need to use $set with Mongoose. If we only want to update certain fields of the document, we can pass in those fields only. So the above example will update the grade to 70 for Tim Smith, but will not remove all the other fields of the document.

Deleting records

Deleting records is similar to updating them; we can use the deleteOne() and deleteMany() methods of our model. These methods take two arguments; the criteria for deletion, and the callback. For example, this will delete all students with a grade less than 30:

Student.deleteMany({grade: {$lt: 30}},
    (err,result)=>{
        if(err) {
            console.log(`Error deleting: ${err}`);
        } else if (result.n==0) {
            console.log('No matching students found!');
        } else {
            console.log(`Successfully deleted ${result.n} documents.`);
        }
});

Note how, once again, we can test how many documents were successfully deleted using the result object. With delete, we use the n field rather than nModified.

Structure of an Express server using Mongoose

Connect to the database first, e.g:

const express = require('express');
const app = express();
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/test');

mongoose.connection.once('open', ()=> {
    // Create models ...

    // Handle routes ...
});

Exercises

Develop a Node/Express REST HT-Tracks server which uses a MongoDB back end (i.e. the hits collection you worked with last time). It should support the following operations:

/search/:artist - should search the database for all songs by a given artist and return the results as JSON.
/search/:song/:artist - should search the database for a specific song by a specific artist and return the results as JSON.
/newSong - should read the details of a new song as POST data and add a new document to the database. To test this you will need to create an HTML form.
/before/:year - should search for all songs before a given year and return them as JSON.
/song/:id - should do the following depending on HTTP method:
- GET - return JSON of the song with that ID;
- PUT - read in JSON containing selected properites of the song and their values, e.g:
```
{"downloads": 123}
```
  and update the song with that ID to those quantities;
- DELETE - delete the song with that ID.

To test these APIs: you can test those which take GET and POST in the browser directly, or via a form, respectively. For those which use PUT and DELETE, use Postman (which appears to be installable locally). I will demonstrate this.

Finally, connect your AJAX front end from week 7 to your 'search' functionality.