Introduction

This is the third article of the series “Learn MongoDB with me”, if you haven’t read my previous post on this topic, I strongly recommend you to find it here. This is the continuation of exploring the Indexes on MongoDB, we will be discussing about various MongoDB indexes which we can perform on our data. I hope you will find this post useful. Thanks for reading.

Learn MongoDB with me

You can see all the articles on this series below.

• Learn MongoDB with me
• Learn MongoDB with me – Part 2
• Using MongoDB on Node JS Application Using Mongoose

Background

Like I said, it is going to be the third  part of the series. I believe that you have enough knowledge about Mongo DB now,  If not, please consider reading my previous posts again.

Indexes in MongoDB

Let’s import a new collection, products first. 

[
   {
      "id":2,
      "name":"An ice sculpture",
      "price":12.50,
      "tags":[
         "cold",
         "ice"
      ],
      "dimensions":{
         "length":7.0,
         "width":12.0,
         "height":9.5
      },
      "warehouseLocation":{
         "latitude":-78.75,
         "longitude":20.4
      }
   },
   {
      "id":3,
      "name":"A blue mouse",
      "price":25.50,
      "dimensions":{
         "length":3.1,
         "width":1.0,
         "height":1.0
      },
      "warehouseLocation":{
         "latitude":54.4,
         "longitude":-32.7
      }
   },
   {
      "id":4,
      "name":"Keyboard",
      "price":15.50,
      "dimensions":{
         "length":1.1,
         "width":1.0,
         "height":1.0
      },
      "warehouseLocation":{
         "latitude":24.4,
         "longitude":-42.7
      }
   },
   {
      "id":5,
      "name":"Doll",
      "price":10.50,
      "dimensions":{
         "length":5.1,
         "width":1.0,
         "height":7.0
      },
      "warehouseLocation":{
         "latitude":64.4,
         "longitude":-82.7
      }
   },
   {
      "id":6,
      "name":"Wallet",
      "price":5.50,
      "dimensions":{
         "length":1.1,
         "width":1.0,
         "height":1.0
      },
      "warehouseLocation":{
         "latitude":24.4,
         "longitude":-12.7
      }
   }
]

Please be noted that these are just dummy data, and it may sound illogical to you.

C:\Program Files\MongoDB\Server\3.4\bin>mongoimport --db mylearning --collection products --jsonArray --file products.json
2018-03-06T16:48:34.440+0530    connected to: localhost
2018-03-06T16:48:34.607+0530    imported 5 documents

C:\Program Files\MongoDB\Server\3.4\bin>

If you don’t know how the import command works, please read my previous posts where we have seen simple indexes.  Now we have the data, let’s go perform Indexes.

Single Key Indexes

In one of my previous post in this series of article, I had mentioned about simple indexes. Here in this article, we are not going to talk about it, instead  we will explore on other indexing option what MongoDB has. Sounds good? If yes, let’s continue. let’s go and see Multi key indexes

Multi Key Indexes or Compound Indexes

As the name implies, we are actually going to set indexes with more than one key element. On our products collection, we have some product documents right, what we a user needs to filter the same with the price and warehouse location. Yeah, we need to build a query.

MongoDB Enterprise > db.products.find({
... "price: {$lte: 16},
2018-03-06T17:10:15.005+0530 E QUERY    [thread1] SyntaxError: unterminated string literal @(shell):2:0
MongoDB Enterprise > db.products.find({
... "price": {$lte: 16},
... "warehouseLocation.latitude": {$gte: 60}
... })
{ "_id" : ObjectId("5a9e790a1ae1f955c1a70c4a"), "id" : 5, "name" : "Doll", "price" : 10.5, "dimensions" : { "length" : 5.1, "width" : 1, "height" : 7 }, "warehouseLocation" : { "latitude" : 64.4, "longitude" : -82.7 } }
MongoDB Enterprise >

We have got one entry according to  our search, “price”: {$lte: 16} and “warehouseLocation.latitude”: {$gte: 60} that’s cool. Now let’s try to find out the execution status for the same.

Please be noted that we have used $lte and $gte which stands for “less than or equal to” and “greater than or equal to”, remember what I have told you before, “Mongo shell is cool and we can do anything with it”. Let’s find out the examined elements count for our preceding find query now.

db.products.find({ "price": {$lte: 16}, "warehouseLocation.latitude": {$gte: 60} }).explain("executionStats")

And if your query if correct, you will be getting a result as preceding.

"queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "mylearning.products",
                "indexFilterSet" : false,
                "parsedQuery" : {
                        "$and" : [
                                {
                                        "price" : {
                                                "$lte" : 16
                                        }
                                },
                                {
                                        "warehouseLocation.latitude" : {
                                                "$gte" : 60
                                        }
                                }
                        ]
                },
                "winningPlan" : {
                        "stage" : "COLLSCAN",
                        "filter" : {
                                "$and" : [
                                        {
                                                "price" : {
                                                        "$lte" : 16
                                                }
                                        },
                                        {
                                                "warehouseLocation.latitude" : {
                                                        "$gte" : 60
                                                }
                                        }
                                ]
                        },
                        "direction" : "forward"
                },
                "rejectedPlans" : [ ]
        },
        "executionStats" : {
                "executionSuccess" : true,
                "nReturned" : 1,
                "executionTimeMillis" : 107,
                "totalKeysExamined" : 0,
                "totalDocsExamined" : 5,
                "executionStages" : {
                        "stage" : "COLLSCAN",
                        "filter" : {
                                "$and" : [
                                        {
                                                "price" : {
                                                        "$lte" : 16
                                                }
                                        },
                                        {
                                                "warehouseLocation.latitude" : {
                                                        "$gte" : 60
                                                }
                                        }
                                ]
                        },
                        "nReturned" : 1,
                        "executionTimeMillisEstimate" : 0,
                        "works" : 7,
                        "advanced" : 1,
                        "needTime" : 5,
                        "needYield" : 0,
                        "saveState" : 0,
                        "restoreState" : 0,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "direction" : "forward",
                        "docsExamined" : 5
                }
        },
        "serverInfo" : {
                "host" : "PC292716",
                "port" : 27017,
                "version" : "3.4.9",
                "gitVersion" : "876ebee8c7dd0e2d992f36a848ff4dc50ee6603e"
        },
        "ok" : 1
}

You might have already noticed the value we have for totalDocsExamined , if you haven’t please check now. In my case it is 5, which means the query just examined all the records we have. Ah, that sounds bad right? What if we have millions of records on our collection, how long it is gonna take to fetch the results?

MongoDB Enterprise > db.products.createIndex({price:1, "warehouseLocation.latitude":1})
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}

Run your previous query now, and find out what is the value of docs examined.

MongoDB Enterprise > db.products.find({ "price": {$lte: 16}, "warehouseLocation.latitude": {$gte: 60} }).explain("executionStats")
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "mylearning.products",
                "indexFilterSet" : false,
                "parsedQuery" : {
                        "$and" : [
                                {
                                        "price" : {
                                                "$lte" : 16
                                        }
                                },
                                {
                                        "warehouseLocation.latitude" : {
                                                "$gte" : 60
                                        }
                                }
                        ]
                },
                "winningPlan" : {
                        "stage" : "FETCH",
                        "inputStage" : {
                                "stage" : "IXSCAN",
                                "keyPattern" : {
                                        "price" : 1,
                                        "warehouseLocation.latitude" : 1
                                },
                                "indexName" : "price_1_warehouseLocation.latitude_1",
                                "isMultiKey" : false,
                                "multiKeyPaths" : {
                                        "price" : [ ],
                                        "warehouseLocation.latitude" : [ ]
                                },
                                "isUnique" : false,
                                "isSparse" : false,
                                "isPartial" : false,
                                "indexVersion" : 2,
                                "direction" : "forward",
                                "indexBounds" : {
                                        "price" : [
                                                "[-inf.0, 16.0]"
                                        ],
                                        "warehouseLocation.latitude" : [
                                                "[60.0, inf.0]"
                                        ]
                                }
                        }
                },
                "rejectedPlans" : [ ]
        },
        "executionStats" : {
                "executionSuccess" : true,
                "nReturned" : 1,
                "executionTimeMillis" : 1089,
                "totalKeysExamined" : 5,
                "totalDocsExamined" : 1,
                "executionStages" : {
                        "stage" : "FETCH",
                        "nReturned" : 1,
                        "executionTimeMillisEstimate" : 310,
                        "works" : 5,
                        "advanced" : 1,
                        "needTime" : 3,
                        "needYield" : 0,
                        "saveState" : 2,
                        "restoreState" : 2,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "docsExamined" : 1,
                        "alreadyHasObj" : 0,
                        "inputStage" : {
                                "stage" : "IXSCAN",
                                "nReturned" : 1,
                                "executionTimeMillisEstimate" : 270,
                                "works" : 5,
                                "advanced" : 1,
                                "needTime" : 3,
                                "needYield" : 0,
                                "saveState" : 2,
                                "restoreState" : 2,
                                "isEOF" : 1,
                                "invalidates" : 0,
                                "keyPattern" : {
                                        "price" : 1,
                                        "warehouseLocation.latitude" : 1
                                },
                                "indexName" : "price_1_warehouseLocation.latitude_1",
                                "isMultiKey" : false,
                                "multiKeyPaths" : {
                                        "price" : [ ],
                                        "warehouseLocation.latitude" : [ ]
                                },
                                "isUnique" : false,
                                "isSparse" : false,
                                "isPartial" : false,
                                "indexVersion" : 2,
                                "direction" : "forward",
                                "indexBounds" : {
                                        "price" : [
                                                "[-inf.0, 16.0]"
                                        ],
                                        "warehouseLocation.latitude" : [
                                                "[60.0, inf.0]"
                                        ]
                                },
                                "keysExamined" : 5,
                                "seeks" : 4,
                                "dupsTested" : 0,
                                "dupsDropped" : 0,
                                "seenInvalidated" : 0
                        }
                }
        },
        "serverInfo" : {
                "host" : "PC292716",
                "port" : 27017,
                "version" : "3.4.9",
                "gitVersion" : "876ebee8c7dd0e2d992f36a848ff4dc50ee6603e"
        },
        "ok" : 1
}
MongoDB Enterprise > db.products.find({ "price": {$lte: 16}, "warehouseLocation.latitude": {$gte: 60} })
{ "_id" : ObjectId("5a9e790a1ae1f955c1a70c4a"), "id" : 5, "name" : "Doll", "price" : 10.5, "dimensions" : { "length" : 5.1, "width" : 1, "height" : 7 }, "warehouseLocation" : { "latitude" : 64.4, "longitude" : -82.7 } }
MongoDB Enterprise >

Yeah, we got “docsExamined” : 1 , that’s the way to go. Go create some indexes on your top most queries, you can definitely see some magics over there. You can create up to 64 indexes on a collection in MongoDB, but you may need to create only few, only on your top result queries. What you can do is, whenever you are facing any performance issues on any queries, consider that it needs some tuning and definitely a Index. There are so many other complex Indexes, but widely used Indexes are single key index and compound index.

With that, we are done with this post. I will be posting the continuation part of this series very soon. Till then, bye.

Conclusion

Thanks a lot for reading. Did I miss anything that you may think which is needed? Could you find this post as useful? I hope you liked this article. Please share me your valuable suggestions and feedback.

Your turn. What do you think?

A blog isn’t a blog without comments, but do try to stay on topic. If you have a question unrelated to this post, you’re better off posting it on C# Corner, Code Project, Stack Overflow, Asp.Net Forum instead of commenting here. Tweet or email me a link to your question there and I’ll definitely try to help if I can.

Kindest Regards
Sibeesh Venu

Introduction

This is the second article of the series “Learn MongoDB with me”, if you haven’t read my previous post on this topic, I strongly recommend you to find it here. This is the continuation of exploring the Mongo shells, we will be performing some commands on the Mongo shells. For easy reference I will try to add screenshots for each steps I am following. I hope it will help you come along with me. Thanks for reading.

Background

Like I said, it is going to be the second part of the series. I believe that you have enough knowledge about Mongo DB and how to set up it? How a Mongo shells can be used? If you are not able to answer these question yourself, please consider reading my previous posts again.

Mongo shells, the perfect CLI

We can do anything in the Mongo shell, to make the statement clear. I am going to perform some CRUD operations (Create, Read, Update, Delete) within Mongo shells. To do so, we can use Mongo import commands. The Mongo import commands can do the work for you, even if the data is in .tsv, .csv, .json etc. Let’s see those in action.

As a first step, let us see some documentation.

C:\Program Files\MongoDB\Server\3.4\bin>mongoimport --help

The above command will give you all the options available to get started.

Usage:
  mongoimport <options> <file>

Import CSV, TSV or JSON data into MongoDB. If no file is provided, mongoimport reads from stdin.

See http://docs.mongodb.org/manual/reference/program/mongoimport/ for more information.

general options:
      /help                                       print usage
      /version                                    print the tool version and
                                                  exit

verbosity options:
  /v, /verbose:<level>                            more detailed log output
                                                  (include multiple times for
                                                  more verbosity, e.g. -vvvvv,
                                                  or specify a numeric value,
                                                  e.g. --verbose=N)
      /quiet                                      hide all log output

connection options:
  /h, /host:<hostname>                            mongodb host to connect to
                                                  (setname/host1,host2 for
                                                  replica sets)
      /port:<port>                                server port (can also use
                                                  --host hostname:port)

kerberos options:
      /gssapiServiceName:<service-name>           service name to use when
                                                  authenticating using
                                                  GSSAPI/Kerberos ('mongodb' by
                                                  default)
      /gssapiHostName:<host-name>                 hostname to use when
                                                  authenticating using
                                                  GSSAPI/Kerberos (remote
                                                  server's address by default)

ssl options:
      /ssl                                        connect to a mongod or mongos
                                                  that has ssl enabled
      /sslCAFile:<filename>                       the .pem file containing the
                                                  root certificate chain from
                                                  the certificate authority
      /sslPEMKeyFile:<filename>                   the .pem file containing the
                                                  certificate and key
      /sslPEMKeyPassword:<password>               the password to decrypt the
                                                  sslPEMKeyFile, if necessary
      /sslCRLFile:<filename>                      the .pem file containing the
                                                  certificate revocation list
      /sslAllowInvalidCertificates                bypass the validation for
                                                  server certificates
      /sslAllowInvalidHostnames                   bypass the validation for
                                                  server name
      /sslFIPSMode                                use FIPS mode of the
                                                  installed openssl library

authentication options:
  /u, /username:<username>                        username for authentication
  /p, /password:<password>                        password for authentication
      /authenticationDatabase:<database-name>     database that holds the
                                                  user's credentials
      /authenticationMechanism:<mechanism>        authentication mechanism to
                                                  use

namespace options:
  /d, /db:<database-name>                         database to use
  /c, /collection:<collection-name>               collection to use

uri options:
      /uri:mongodb-uri                            mongodb uri connection string

input options:
  /f, /fields:<field>[,<field>]*                  comma separated list of
                                                  fields, e.g. -f name,age
      /fieldFile:<filename>                       file with field names - 1 per
                                                  line
      /file:<filename>                            file to import from; if not
                                                  specified, stdin is used
      /headerline                                 use first line in input
                                                  source as the field list (CSV
                                                  and TSV only)
      /jsonArray                                  treat input source as a JSON
                                                  array
      /parseGrace:<grace>                         controls behavior when type
                                                  coercion fails - one of:
                                                  autoCast, skipField, skipRow,
                                                  stop (defaults to 'stop')
                                                  (default: stop)
      /type:<type>                                input format to import: json,
                                                  csv, or tsv (defaults to
                                                  'json') (default: json)
      /columnsHaveTypes                           indicated that the field list
                                                  (from --fields, --fieldsFile,
                                                  or --headerline) specifies
                                                  types; They must be in the
                                                  form of
                                                  '<colName>.<type>(<arg>)'.
                                                  The type can be one of: auto,
                                                  binary, bool, date, date_go,
                                                  date_ms, date_oracle, double,
                                                  int32, int64, string. For
                                                  each of the date types, the
                                                  argument is a datetime layout
                                                  string. For the binary type,
                                                  the argument can be one of:
                                                  base32, base64, hex. All
                                                  other types take an empty
                                                  argument. Only valid for CSV
                                                  and TSV imports. e.g.
                                                  zipcode.string(),
                                                  thumbnail.binary(base64)

ingest options:
      /drop                                       drop collection before
                                                  inserting documents
      /ignoreBlanks                               ignore fields with empty
                                                  values in CSV and TSV
      /maintainInsertionOrder                     insert documents in the order
                                                  of their appearance in the
                                                  input source
  /j, /numInsertionWorkers:<number>               number of insert operations
                                                  to run concurrently (defaults
                                                  to 1) (default: 1)
      /stopOnError                                stop importing at first
                                                  insert/upsert error
      /mode:[insert|upsert|merge]                 insert: insert only. upsert:
                                                  insert or replace existing
                                                  documents. merge: insert or
                                                  modify existing documents.
                                                  defaults to insert
      /upsertFields:<field>[,<field>]*            comma-separated fields for
                                                  the query part when --mode is
                                                  set to upsert or merge
      /writeConcern:<write-concern-specifier>     write concern options e.g.
                                                  --writeConcern majority,
                                                  --writeConcern '{w: 3,
                                                  wtimeout: 500, fsync: true,
                                                  j: true}'
      /bypassDocumentValidation                   bypass document validation

Insert data to MondoDB using Mongo shell

Now let’s say I have a following JSON data, and we are going to insert the same to our db collection.

[ 
{
 "color": "black",
 "category": "hue",
 "type": "primary",
 "code": {
 "rgba": [255,255,255,1],
 "hex": "#000"
 }
 },
 {
 "color": "white",
 "category": "value",
 "code": {
 "rgba": [0,0,0,1],
 "hex": "#FFF"
 }
 },
 {
 "color": "red",
 "category": "hue",
 "type": "primary",
 "code": {
 "rgba": [255,0,0,1],
 "hex": "#FF0"
 }
 },
 {
 "color": "blue",
 "category": "hue",
 "type": "primary",
 "code": {
 "rgba": [0,0,255,1],
 "hex": "#00F"
 }
 },
 {
 "color": "yellow",
 "category": "hue",
 "type": "primary",
 "code": {
 "rgba": [255,255,0,1],
 "hex": "#FF0"
 }
 },
 {
 "color": "green",
 "category": "hue",
 "type": "secondary",
 "code": {
 "rgba": [0,255,0,1],
 "hex": "#0F0"
 }
 }
]

To do so, we need to use the following command.

C:\Program Files\MongoDB\Server\3.4\bin>mongoimport --db mylearning --collection colors --jsonArray --file colors.json

Here, as you can see, we are providing the db name, collection, name, what is the data type of the file and finally the file name.

If you ever get the error as “Failed: open colors.json: The system cannot find the file specified.”, please make sure that the document is in the server folder, in my case it is “C:\Program Files\MongoDB\Server\3.4\bin”. If everything is fine, you will be able to see an output as preceding.

C:\Program Files\MongoDB\Server\3.4\bin>mongoimport --db mylearning --collection colors --jsonArray --file colors.json
2018-03-01T16:35:35.470+0530    connected to: localhost
2018-03-01T16:35:36.012+0530    imported 6 documents

C:\Program Files\MongoDB\Server\3.4\bin>

Reading the data from a collection in MongoDB

Now that, we have colors collection, let’s go and check whether the database has the data we are expecting.

C:\Program Files\MongoDB\Server\3.4\bin>mongo
MongoDB shell version v3.4.9
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.9
Server has startup warnings:
2018-03-01T16:24:43.793+0530 I CONTROL  [initandlisten]
2018-03-01T16:24:43.793+0530 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2018-03-01T16:24:43.793+0530 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2018-03-01T16:24:43.793+0530 I CONTROL  [initandlisten]
MongoDB Enterprise > use mylearning
switched to db mylearning
MongoDB Enterprise > show collections
chats
colors
messages
MongoDB Enterprise > db.colors.count
function (query, options) {
    query = this.find(query);

    // Apply options and return the result of the find
    return QueryHelpers._applyCountOptions(query, options).count(true);
}
MongoDB Enterprise > db.colors.count()
6
MongoDB Enterprise >

When you use count, make sure you are treating it as a function as count(). We have the count as 6, and that’s what we are expecting. Am I right? Don’t you think, that we should go fetch some data from that collection, yeah with some filter?

MongoDB Enterprise > db.colors.find({"type": "primary"})
{ "_id" : ObjectId("5a97de7f2fcdf731d255a19d"), "color" : "black", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 255, 255, 1 ], "hex" : "#000" } }
{ "_id" : ObjectId("5a97de7f2fcdf731d255a19e"), "color" : "yellow", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 255, 0, 1 ], "hex" : "#FF0" } }
{ "_id" : ObjectId("5a97de7f2fcdf731d255a1a1"), "color" : "red", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 0, 0, 1 ], "hex" : "#FF0" } }
{ "_id" : ObjectId("5a97de7f2fcdf731d255a1a2"), "color" : "blue", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 0, 0, 255, 1 ], "hex" : "#00F" } }
MongoDB Enterprise >

We have successfully imported that data, and we have fetched the colors with type as primary. I think, there should be a custom color, which has the type as primary. Can we do that now?

MongoDB Enterprise > db.colors.insert({
... "color": "custome",
...  "category": "hue",
...  "type": "primary",
...  "code": {
...  "rgba": [255,1,255,1],
...  "hex": "#FF1"
...  }
... }
... )
WriteResult({ "nInserted" : 1 })
MongoDB Enterprise >

Now if you run our previous query, you can see an output as preceding.

MongoDB Enterprise > db.colors.find({"type": "primary"})
{ "_id" : ObjectId("5a97de7f2fcdf731d255a19d"), "color" : "black", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 255, 255, 1 ], "hex" : "#000" } }
{ "_id" : ObjectId("5a97de7f2fcdf731d255a19e"), "color" : "yellow", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 255, 0, 1 ], "hex" : "#FF0" } }
{ "_id" : ObjectId("5a97de7f2fcdf731d255a1a1"), "color" : "red", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 0, 0, 1 ], "hex" : "#FF0" } }
{ "_id" : ObjectId("5a97de7f2fcdf731d255a1a2"), "color" : "blue", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 0, 0, 255, 1 ], "hex" : "#00F" } }
{ "_id" : ObjectId("5a97e3202c19f0e958477e06"), "color" : "custome", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 1, 255, 1 ], "hex" : "#FF1" } }
MongoDB Enterprise >

Updating a document in MongoDB

Wow, we have the data now. And we performed, Create and Read operations on our DB. It is time for updating the document. Let’s go ahead and add a new property “manuallyCreated” to the color we have created. It is going to help us in finding this kind of entries easily.

MongoDB Enterprise > db.colors.update({
... "color":"custome"
... }
... ,{
... $set:{"manuallyCreated":"True"}}
... )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
MongoDB Enterprise >

Let’s find all the manually created colors now, and yeah we know there is going to be one record.

MongoDB Enterprise > db.colors.find({"manuallyCreated":"True"})
{ "_id" : ObjectId("5a97e3202c19f0e958477e06"), "color" : "custome", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 1, 255, 1 ], "hex" : "#FF1" }, "manuallyCreated" : "True" }
MongoDB Enterprise >

Let’s update the color name to “custom” instead of “custome”, sorry for the typo.

MongoDB Enterprise > db.colors.update({"_id" : ObjectId("5a97e3202c19f0e958477e06")},
... {$set:{"color":"custom"}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
MongoDB Enterprise >

Ah, I made a mistake. I ran the following query by mistake.

MongoDB Enterprise > db.colors.update({ "color":"custom" } ,{ $set:{"code.rgba" : [ 255, 1, 255]}} )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

Do you know, what that query just did? It just updated the rgba of our custom color to three point array “[ 255, 1, 255]”. That’s not what I wanted. Now what we can do? We need to add one value to that set. Let’s do that now.

MongoDB Enterprise > db.colors.update({ "color":"custom" } ,{ $addToSet:{"code.rgba" : "1" }} )
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

MongoDB Enterprise > db.colors.find({"manuallyCreated":"True"})
{ "_id" : ObjectId("5a97e3202c19f0e958477e06"), "color" : "custom", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 1, 255, "1" ], "hex" : "#FF1" }, "manuallyCreated" : "True" }
MongoDB Enterprise >

Please be noted that, there is one more update, which will just update the entire document. In this case, if you are not providing the value for each attributes, it will be overwriting the same. Let’s see an example.

MongoDB Enterprise > db.colors.update({"color":"red"},{"color":"test"})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
MongoDB Enterprise > db.colors.find({})

Here we just give a command to update the color red to color test, as we have not provided other attributes as part of our query, after running the query, there will be only one attribute that is color.

MongoDB Enterprise > db.colors.find({})
{ "_id" : ObjectId("5a97f73f2fcdf731d255a1f1"), "color" : "black", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 255, 255, 1 ], "hex" : "#000" } }
{ "_id" : ObjectId("5a97f73f2fcdf731d255a1f2"), "color" : "test" }
{ "_id" : ObjectId("5a97f73f2fcdf731d255a1f3"), "color" : "white", "category" : "value", "code" : { "rgba" : [ 0, 0, 0, 1 ], "hex" : "#FFF" } }
{ "_id" : ObjectId("5a97f73f2fcdf731d255a1f4"), "color" : "blue", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 0, 0, 255, 1 ], "hex" : "#00F" } }
{ "_id" : ObjectId("5a97f73f2fcdf731d255a1f5"), "color" : "yellow", "category" : "hue", "type" : "primary", "code" : { "rgba" : [ 255, 255, 0, 1 ], "hex" : "#FF0" } }
{ "_id" : ObjectId("5a97f73f2fcdf731d255a1f6"), "color" : "green", "category" : "hue", "type" : "secondary", "code" : { "rgba" : [ 0, 255, 0, 1 ], "hex" : "#0F0" } }
MongoDB Enterprise > db.colors.find({"color":"test"})
{ "_id" : ObjectId("5a97f73f2fcdf731d255a1f2"), "color" : "test" }
MongoDB Enterprise >

Deleting a document in MongoDB shell

So, we added a custom color, and later we found that it is no more needed in our document as we found an exact match with a different color. Now we need to remove the one we have added.

MongoDB Enterprise > db.colors.remove({"color":"custom"})
WriteResult({ "nRemoved" : 1 })
MongoDB Enterprise >

Let’s go and check again whether it is actually removed or not.

MongoDB Enterprise > db.colors.find({"color":"custom"})
MongoDB Enterprise >

The find query returns no records.

Deleting an entire collection

We have performed CRUD operations on a collection, what if we need to remove the entire collection?

MongoDB Enterprise > db.colors.drop()
true
MongoDB Enterprise >

With that, we are done with this post. I will be posting the continuation part of this series very soon. Till then, bye.

Conclusion

Thanks a lot for reading. Did I miss anything that you may think which is needed? Could you find this post as useful? I hope you liked this article. Please share me your valuable suggestions and feedback.

Your turn. What do you think?

A blog isn’t a blog without comments, but do try to stay on topic. If you have a question unrelated to this post, you’re better off posting it on C# Corner, Code Project, Stack Overflow, Asp.Net Forum instead of commenting here. Tweet or email me a link to your question there and I’ll definitely try to help if I can.

Kindest Regards
Sibeesh Venu

Introduction

This is going to be a series of article on MongoDB. Here we are going to do some exercises with MongoDB, we will be talking about Mongo Shell, how can we configure MongoDB?, What are Indexes in MongoDB etc. We all know what an Indexes is, you might have already done that with any relational databases like SQL and MySQL . Have you every done indexing for your MongoDB? If your answer is “no”, no worries, here we are going to see indexes in MongoDB , if it is a “yes” please read this post and correct me if I am wrong anywhere. Let’s begin now.

Prerequisites

I hope you might have got a basic information about MongoDB, if not, I strongly recommend you to read these posts. Now that, you have a basic idea, I am assuming that you have already set up the environment for MongoDB development. Let’s recall what you might have done so far.

1. Install MongoDB
2. Set the environment variable for MongoDB
3. Start the MongoDB services

To set the environment variable for MongoDB, you may have to add a new path to the system variable path with the value as “C:\Program Files\MongoDB\Server\3.4\bin”, please note that the version number will be varied according to your MongoDB version. Once you are done the above steps, you should be able to start both Mongo server and Mongo shell from the command line interface.

Setting up MongoDB using CLI

Now let’s just open our command line interface, and create the data directory for Mongo. We will have to create a directory for the same. Please go along with the below commands.

md \data
md \data\db
mongod

Now let’s open a new CLI and run the command “mongo”, please do not worry about the warnings you are getting, as we are not working in production data, we may not need to secure and optimize it.

Exploring MongoDB

Once you are connected to MongoDB, by default you are connected to test DB. You can check that by running the command MongoDB Enterprise > db

Playing with Mongo Shell

Let’s just use a new database now.

MongoDB Enterprise > use MongoIndex
switched to db MongoIndex
MongoDB Enterprise >

Please be noted that the database MongoIndex doesn’t exist as of now, as we haven’t created it. Still, Mongo just switched our context to the new database. You can see this if you run the command show dbs

The database will be created once we insert any document associated with it. Now we are going to create a new collection called “User”, so once we made the entry to this collection, the database will also be created automatically. Let’s do that.

MongoDB Enterprise > db.users.insert({"name":"Sibees Venu"})
WriteResult({ "nInserted" : 1 })
MongoDB Enterprise >

Now if you run the “show dbs” command again, the database MongoIndex will show up. If you ever need to see the collections you have in the DB, you just need to run the command “show collections”.

MongoDB Enterprise > show collections
users
MongoDB Enterprise >

The MongoDB is very friendly when it comes to data, it doesn’t require any schema to get it started. The learning is so easy, am I right?

The other benefit of MongoDB is its JavaScript interpreted shell, where we can actually type JavaScript code and run. To test it out, let’s create a variable and use it.

MongoDB Enterprise > var name = "Sibeesh Venu"
MongoDB Enterprise > name
Sibeesh Venu
MongoDB Enterprise >

This way, we can interact with the database with a JavaScript program. Now let’s go ahead and creating a collection called “Numbers” and insert 26,000 rows in it. So how are we going to do that? Yes, you are right, we are going to write a for loop, the mongo shell gives that kind of flexibility. Let’s see that in action.

MongoDB Enterprise > for(i=0;i<=26000;i++){
... db.Numbers.insert({
... "number":i
... })
... }
WriteResult({ "nInserted" : 1 })
MongoDB Enterprise >

So we have done that. Note that, we are able to break the commands into multiple lines, this allows to break the complex codes to much readable format in the shell itself. Sounds good?

Even though we have inserted 26,000 rows,  it always shows,"nInserted" : 1 this is because it is counting a number of operations, not the individual documents. Let’s see this by checking the count now.

MongoDB Enterprise > db.numbers.count()
0
MongoDB Enterprise > db.Numbers.count()
26001
MongoDB Enterprise >

Please note that it is case sensitive.

Indexes in MongoDB

Now if you need to see any particular record,  you can always write the query in the shell as follows.

MongoDB Enterprise > db.Numbers.find(
... {"number":24000}
... )
{ "_id" : ObjectId("5a8d3be2020a0071d115cf62"), "number" : 24000 }
MongoDB Enterprise >

So in the query, we are using the function “find” with the filter “number: 24000”, so that the Mongo can return the record which has the number value as 24000. Now that we have got the output we needed, would you like to see what just happened in the background? To do so, we can use the function “explain()”.

MongoDB Enterprise > db.Numbers.find( {"number":24000} ).explain()
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "MongoIndex.Numbers",
                "indexFilterSet" : false,
                "parsedQuery" : {
                        "number" : {
                                "$eq" : 24000
                        }
                },
                "winningPlan" : {
                        "stage" : "COLLSCAN",
                        "filter" : {
                                "number" : {
                                        "$eq" : 24000
                                }
                        },
                        "direction" : "forward"
                },
                "rejectedPlans" : [ ]
        },
        "serverInfo" : {
                "host" : "PC292716",
                "port" : 27017,
                "version" : "3.4.9",
                "gitVersion" : "876ebee8c7dd0e2d992f36a848ff4dc50ee6603e"
        },
        "ok" : 1
}
MongoDB Enterprise >

And, if you need to get more information about the execution, you can pass the parameter “executionStats” to the “explain” function.

The parameter is always case sensitive, you will get an errors as below, if you give it wrong. So please make sure you are passing executionStats not executionstats.

“MongoDB Enterprise > db.Numbers.find( {“number”:24000} ).explain(“executionstats”)
2018-02-21T15:12:34.197+0530 E QUERY [thread1] Error: explain verbosity must be one of {‘queryPlanner’,’executionStats’,’allPlansExecution’} :
parseVerbosity@src/mongo/shell/explainable.js:22:1
constructor@src/mongo/shell/explain_query.js:83:27
DBQuery.prototype.explain@src/mongo/shell/query.js:520:24
@(shell):1:1″

MongoDB Enterprise > db.Numbers.find( {"number":24000} ).explain("executionStats")
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "MongoIndex.Numbers",
                "indexFilterSet" : false,
                "parsedQuery" : {
                        "number" : {
                                "$eq" : 24000
                        }
                },
                "winningPlan" : {
                        "stage" : "COLLSCAN",
                        "filter" : {
                                "number" : {
                                        "$eq" : 24000
                                }
                        },
                        "direction" : "forward"
                },
                "rejectedPlans" : [ ]
        },
        "executionStats" : {
                "executionSuccess" : true,
                "nReturned" : 1,
                "executionTimeMillis" : 13,
                "totalKeysExamined" : 0,
                "totalDocsExamined" : 26001,
                "executionStages" : {
                        "stage" : "COLLSCAN",
                        "filter" : {
                                "number" : {
                                        "$eq" : 24000
                                }
                        },
                        "nReturned" : 1,
                        "executionTimeMillisEstimate" : 11,
                        "works" : 26003,
                        "advanced" : 1,
                        "needTime" : 26001,
                        "needYield" : 0,
                        "saveState" : 203,
                        "restoreState" : 203,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "direction" : "forward",
                        "docsExamined" : 26001
                }
        },
        "serverInfo" : {
                "host" : "PC292716",
                "port" : 27017,
                "version" : "3.4.9",
                "gitVersion" : "876ebee8c7dd0e2d992f36a848ff4dc50ee6603e"
        },
        "ok" : 1
}
MongoDB Enterprise >

Now you can see more information on the execution of how much time it took for the execution and how many docs it is examined etc. If you have noticed, it has examined all the 26001 records and took 13 milliseconds. That’s just a case, that we had only less number of records in the table, what if, we have millions of records in it? And examining all the records would be a bad idea, am I right? So what do we do at that time? What would be a permanent solution for this? This is where the importance of Indexes comes into action.

Let’s create an Index for the number that we are going to search.

MongoDB Enterprise > db.Numbers.createIndex({number:1})
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}
MongoDB Enterprise >

Here the number is a special variable, not a string. As you can see, we had created the index. You can see that the property value of createdCollectionAutomatically is false, as the collection had already created and it didn’t have to create it again.

Let’s run our find query again.

MongoDB Enterprise > db.Numbers.find( {"number":24000} ).explain("executionStats")
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "MongoIndex.Numbers",
                "indexFilterSet" : false,
                "parsedQuery" : {
                        "number" : {
                                "$eq" : 24000
                        }
                },
                "winningPlan" : {
                        "stage" : "FETCH",
                        "inputStage" : {
                                "stage" : "IXSCAN",
                                "keyPattern" : {
                                        "number" : 1
                                },
                                "indexName" : "number_1",
                                "isMultiKey" : false,
                                "multiKeyPaths" : {
                                        "number" : [ ]
                                },
                                "isUnique" : false,
                                "isSparse" : false,
                                "isPartial" : false,
                                "indexVersion" : 2,
                                "direction" : "forward",
                                "indexBounds" : {
                                        "number" : [
                                                "[24000.0, 24000.0]"
                                        ]
                                }
                        }
                },
                "rejectedPlans" : [
                        {
                                "stage" : "FETCH",
                                "inputStage" : {
                                        "stage" : "IXSCAN",
                                        "keyPattern" : {
                                                "number" : 24000
                                        },
                                        "indexName" : "number_24000",
                                        "isMultiKey" : false,
                                        "multiKeyPaths" : {
                                                "number" : [ ]
                                        },
                                        "isUnique" : false,
                                        "isSparse" : false,
                                        "isPartial" : false,
                                        "indexVersion" : 2,
                                        "direction" : "forward",
                                        "indexBounds" : {
                                                "number" : [
                                                        "[24000.0, 24000.0]"
                                                ]
                                        }
                                }
                        }
                ]
        },
        "executionStats" : {
                "executionSuccess" : true,
                "nReturned" : 1,
                "executionTimeMillis" : 36,
                "totalKeysExamined" : 1,
                "totalDocsExamined" : 1,
                "executionStages" : {
                        "stage" : "FETCH",
                        "nReturned" : 1,
                        "executionTimeMillisEstimate" : 0,
                        "works" : 3,
                        "advanced" : 1,
                        "needTime" : 0,
                        "needYield" : 0,
                        "saveState" : 1,
                        "restoreState" : 1,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "docsExamined" : 1,
                        "alreadyHasObj" : 0,
                        "inputStage" : {
                                "stage" : "IXSCAN",
                                "nReturned" : 1,
                                "executionTimeMillisEstimate" : 0,
                                "works" : 2,
                                "advanced" : 1,
                                "needTime" : 0,
                                "needYield" : 0,
                                "saveState" : 1,
                                "restoreState" : 1,
                                "isEOF" : 1,
                                "invalidates" : 0,
                                "keyPattern" : {
                                        "number" : 1
                                },
                                "indexName" : "number_1",
                                "isMultiKey" : false,
                                "multiKeyPaths" : {
                                        "number" : [ ]
                                },
                                "isUnique" : false,
                                "isSparse" : false,
                                "isPartial" : false,
                                "indexVersion" : 2,
                                "direction" : "forward",
                                "indexBounds" : {
                                        "number" : [
                                                "[24000.0, 24000.0]"
                                        ]
                                },
                                "keysExamined" : 1,
                                "seeks" : 1,
                                "dupsTested" : 0,
                                "dupsDropped" : 0,
                                "seenInvalidated" : 0
                        }
                }
        },
        "serverInfo" : {
                "host" : "PC292716",
                "port" : 27017,
                "version" : "3.4.9",
                "gitVersion" : "876ebee8c7dd0e2d992f36a848ff4dc50ee6603e"
        },
        "ok" : 1
}
MongoDB Enterprise >

As we had given the index on what exactly we are going to search, it just examined only that document when we run the query, that’s why the value of the property totalDocsExamined is 1. Indexing will not have many impacts on the database which has few records in it, but it has a massive effect on very large data sets which has millions of records in it. Using this simple Indexes can reduce the execution time to almost nothing.

With that, we are done with this post. I will be posting the continuation part of this series very soon. Till then, bye.

Conclusion

Thanks a lot for reading. Did I miss anything that you may think which is needed? Could you find this post as useful? I hope you liked this article. Please share me your valuable suggestions and feedback.

Your turn. What do you think?

A blog isn’t a blog without comments, but do try to stay on topic. If you have a question unrelated to this post, you’re better off posting it on C# Corner, Code Project, Stack Overflow, Asp.Net Forum instead of commenting here. Tweet or email me a link to your question there and I’ll definitely try to help if I can.

Kindest Regards
Sibeesh Venu

Introduction

In this article, we are going to a chat application in Node JS  with the back end MongoDB.  We will also be using Mongoose for creating the MongoDB models and Socket.io for making multi directional chats on multiple client window. If you are really new to the Node JS, I strongly recommend you to read some articles on the same here. You can also see how you can create a sample Node application with MongoDB and Mongoose here. At the end of this article, I guarantee that you will be having some basic knowledge on the mentioned technologies. I hope you will like this article.

Source Code

You can always clone or download the source code here

Background

Creating a chat application is always an interesting this to do. And it is a good way to learn a lot, because you are creating some interactions on your application. And with the release of few technologies we can create such application without any hassle. It is much more easier than ever. Here we are also going to create a chat application. A basic knowledge of Node JS, MongoDB, JavaScript, JQuery is more than enough to create this project. So, please be with me. Let’s just skip the talking and start developing.

Setting up Node application

This step requires some basic knowledge, please see some of that here.  So as mentioned in that article, we have successfully created our Package.json file and installed the required packages. Let’s review our package.json file.

{
 "name": "chatapp",
 "version": "1.0.0",
 "description": "A chat application in Node JS, which uses MongoDB, Mongoose and Socket.io",
 "main": "index.js",
 "scripts": {
 "test": "echo \"Error: no test specified\" && exit 1"
 },
 "keywords": [
 "Node",
 "JS",
 "MongoDB",
 "Mongoose",
 "Socket.io"
 ],
 "author": "Sibeesh Venu",
 "license": "ISC",
 "dependencies": {
 }
}

Let’s install the packages now by running the below commands.

npm install --save express
npm install --save mongoose
npm install --save body-parser
npm install --save socket.io

Now that we have all the dependencies added to the package.json file.

"dependencies": {
    "body-parser": "^1.18.2",
    "express": "^4.16.2",
    "mongoose": "^4.13.6",
    "socket.io": "^2.0.4"
  }

Creating an App using Express

Let’s create a file server.js and build an app using Express.

var express = require("express")
var app = express()
app.listen(3020,()=>{
 console.log("Well done, now I am listening...")
})

I hope you are getting  the desired output, if not, please don’t worry, just double check the codes you had written.

Running our application on browser

Let’s run our application on port 3020 and see what we are getting http://localhost:3020/..

Yes you will get an error as Cannot GET /  , no worries. To fix that you need to add the following code block to your server.js file

app.use(express.static(__dirname))

Creating Index page

Here I am going to create a HTML 5 page with JQuery and Bootstrap referenced in it.

<!DOCTYPE html>
<title>Creating a Chat Application in Node JS with Express, MongoDB, Mongoose and Socket.io</title>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta.2/css/bootstrap.min.css" crossorigin="anonymous">
<script src="https://code.jquery.com/jquery-3.2.1.min.js" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.3/umd/popper.min.js" crossorigin="anonymous"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta.2/js/bootstrap.min.js" crossorigin="anonymous"></script>

<div class="container">
    <br>
    <div class="jumbotron">
        <h1 class="dispaly-4">Start Chatting</h1>
        <br>
        <input id="txtName" class="form-control" placeholder="Name" type="text">
        <br>
        <textarea id="txtMessage" class="form-control" placeholder="Message"></textarea>
        <br>
        <button id="send" class="btn btn-success">Send</button>
    </div>
    <div id="messages"></div>
</div>

As you can see, the page has two text boxes and one button. We will be creating some scripts very soon which uses these controls.

Start developing the chat app

Till now, it was all basic, and we have done it well. Now it is time to go and write some advanced codes. So, are you ready?

Create model from page data

Here we are going to create the model from the page data, that is, from the controls we have in our page. We will be using JQuery to do so.

<script>
    $(() => {
        $("send").click(() => {
            var chatMessage = {
                name: $("#txtName").val(), chat: $("#txtMessage").val()
            }
            postChat(chat)
        })
    })

    function postChat(chat){
        console.log(chat)
    }

</script>

Now that we have got the model from the user, let’s save it to the DB.

Model values on browser console

Setting up database

We are going to set up our database in mLab as mentioned in this article Using MongoDB on Node JS Application Using Mongoose. So let’s just require Mongoose and do the needed changes.

var express = require("express")
var mongoose = require("mongoose")

var app = express()

var conString = "mongodb://admin:admin@ds038319.mlab.com:38319/mylearning"
app.use(express.static(__dirname))

var Chats = mongoose.model("Chats", {
    name: String,
    chat: String
})

mongoose.connect(conString, { useMongoClient: true }, (err) => {
    console.log("Database connection", err)
})

app.listen(3020, () => {
    console.log("Well done, now I am listening...")
})

Creating a Post request

Now let’s create a post requests in our index.html file which will the post API in our server.js file.

index.html

function postChat(chat) {
        $.post("http://localhost:3020/chats", chat)
}

server.js

app.post("/chats", async (req, res) => {
    try {
        var chat = new Chats(req.body)
        await chat.save()
        res.sendStatus(200)
    } catch (error) {
        res.sendStatus(500)
        console.error(error)
    }
})

Let’s just run our application and test it out.

You may be getting an error as “(node:10824) DeprecationWarning: Mongoose: mpromise (mongoose’s default promise library) is deprecated, plug in your own promise library instead: http://mongoosejs.com/docs/promises.html””, here to fix this, we need to use the default promise instead of the Mongoose promise. Let’s change that. Add this code mongoose.Promise = Promise to our server.js file. Give it a try after setting it.

Still it is not working right, you are getting undefined at the place, var chat = new Chats(req.body) in our Post API . At this stage, we will have to use our body-parser packages, do you remember the package we have installed? Let’s just require that package var bodyParser = require("body-parser") add the preceding codes to our server.js file.

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }))

Now this will convert the request to a JSON object by default. Give it a try again, I am sure you will get the actual value instead of undefined.

Creating a Get request for all the chat data

We have written codes to write our data into our database, we will have to show this data on our page right? Here we are going to write the get requests in out index.html page which will call the get API.

index.html

function getChats() {
     $.get("/chats", (chats) => {
         chats.forEach(addChat)
     })
}

function addChat(chatObj){
    $("#messages").append(`<h5>${chatObj.name} </h5><p>${chatObj.chat}</p>`);
}

And call the function getChats() in document ready event.

server.js

app.get("/chats", (req, res) => {
    Chats.find({}, (error, chats) => {
        res.send(chats)
    })
})

Here we are passing {} to our find function, this means, we are going to get all the chats without any filter. Just run your application now, and check whether you are getting the chat messages you had sent.

Retrieving data from a MongoDB

Implementing Socket.io

Now the chat we are sending is getting saved to our database and we are able to load the same on page load. Is it the behavior of a perfect chat application? Absolutely no, a perfect chat application will be able to,

1. Show the chat message to the UI right after the data gets inserted to database
2. Show the chats in multiple clients, here in our application, if you are opening the URL in multiple browser instance, and if you need to show the chats to both instances, you will have to refresh the pages right? This is not a recommended way

That’s why we are going to implement Socket.io in our application.

Require the package

Unlike the other packages, adding socket.io to the application is a different process, we will have to require the http server first, then, set our app. You can see more information on socket.io here.

var http = require("http").Server(app)
var io= require("socket.io")(http)

Enabling the connection

To enable the connection, we need to use the function io.on.

io.on("connection", (socket) => {
    console.log("Socket is connected...")
})

Listen using new http server

Now that we have a new http server, and we should change our listen code to our new http.

var server = http.listen(3020, () => {
    console.log("Well done, now I am listening on ", server.address().port)
})

Changes in script

Let’s do listen for the event “chat” now in our html page.

var socket = io()
socket.on("chat", addChat)

Please do not forget to include the socket.io.js reference in our index page, other wise you may get an error as “Uncaught ReferenceError: io is not defined”

Emits the event

Once the above step is completed we need to make sure we are emitting the new event on our Post API.

app.post("/chats", async (req, res) => {
 try {
 var chat = new Chats(req.body)
 await chat.save()
 res.sendStatus(200)

 //Emit the event
 io.emit("chat", req.body)

 } catch (error) {
 res.sendStatus(500)
 console.error(error)
 }
})

Let’s just run our application in two browser instances and check for the output.

Socket.io Output

Please make sure that you are getting the chats in the second instance when you send it from the first instance of the browser and vice versa.

Conclusion

Thanks a lot for reading. Did I miss anything that you may think which is needed? Could you find this post as useful? I hope you liked this article. Please share me your valuable suggestions and feedback.

Your turn. What do you think?

A blog isn’t a blog without comments, but do try to stay on topic. If you have a question unrelated to this post, you’re better off posting it on C# Corner, Code Project, Stack Overflow, Asp.Net Forum instead of commenting here. Tweet or email me a link to your question there and I’ll definitely try to help if I can.

Kindest Regards
Sibeesh Venu

Introduction

In this post, we are going to see how we can use a MongoDB on our Node JS application with the help of the package Mongoose. We will also be covering some facts about MongoDB so that as a reader, you will understand why we had chosen MongoDB as our backend. We will be going through some steps to install the required packages using a node package manager terminal, so please be with me. At the end of this article, I guarantee that you will be proficient on how to connect a MongoDB in our Node JS application. I hope you will like this article.

Why MongoDB?

Here, I am going to list down the reasons why I had chosen MongoDB.

1. MongoDB is a NoSQL database when I say NoSQL, it means that it doesn’t have any structure to be followed.
2. There are no relationships, we can perform the relation using separate packages like Mongoose
3. Everything is JSON, even the data and collections are stored as JSON
4. Since the data is stored as JSON, no more conversions are needed. We can directly use it in our application.

Source Code

You can always clone or download the source code here

Background

Node JS has become a trend now, thus some most used databases. One of the recommended database for a Node application is MongoDB. As we discussed a MongoDB doesn’t have any structure inbuilt, here we are going to use a package called Mongoose, with that we can create some structure for our Database. Let’s just skip the talking and start developing.

Setting up Node application

To get started with the Node JS, create a folder and name as per your wish, this is going to be our project directory. Here I am going to use one of my favorite editor, Visual Studio Code. Now please point to your project container, and run the command below.

npm init

NPM Init

This will create a file named “Package.json” in your directory. We will be adding all of our dependencies in this file very soon. The command may ask you some default questions which you need to answer. If you need to create the package.json file with default values in it, you may consider running the below command.

npm init --yes

Let’s review our package.json file.

{
  "name": "node-mongodb-mongoose",
  "version": "1.0.0",
  "description": "A Node application with MongoDB and Mongoose",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "Node",
    "Mongoose",
    "MongoDB"
  ],
  "author": "Sibeesh Venu",
  "license": "ISC"
}

Getting the required packages

Now that we have the application ready, let’s get the required packages ready.

npm install --save mongoose

This will create a new folder npm_modules on your project directory where you can see the package Mongoose in it.

npm install --save express

The above command will add the package express to our application.

Creating the Node App

Now we can create our server.js file where we are going to write most of our application code.

var express = require("express")
var app = express()
app.listen("3010",()=>{
    console.log("I just started listening!.")
})

Now run the command node server.jsand make sure that you are getting a console as ” I just started listening!.”.

If you are getting an error as preceding, please make sure that you are running the application on an unused port.

PS F:\Visual Studio\Node.JS\Node-MongoDB-Mongoose> node server.js
events.js:136
throw er; // Unhandled ‘error’ event
^

Error: listen EADDRINUSE :::3000
at Object._errnoException (util.js:1031:13)
at _exceptionWithHostPort (util.js:1052:20)
at Server.setupListenHandle [as _listen2] (net.js:1367:14)
at listenInCluster (net.js:1408:12)
at Server.listen (net.js:1496:7)
at Function.listen (F:\Visual Studio\Node.JS\Node-MongoDB-Mongoose\node_modules\express\lib\application.js:618:24)
at Object.<anonymous> (F:\Visual Studio\Node.JS\Node-MongoDB-Mongoose\server.js:3:5)
at Module._compile (module.js:641:30)
at Object.Module._extensions..js (module.js:652:10)
at Module.load (module.js:560:32)”

Setting up database

Now that we have our application ready, let’s go and create our database. I am going to use mLab for creating the database. If you have not installed MongoDB on your machine, I strongly recommend creating a database in mLab. I had created a database there and have got my connection string as preceding.

mongodb://<dbuser>:<dbpassword>@ds038319.mlab.com:38319/mylearning

We will be updating the connection string with actual DB user and password later. Please make sure that you are creating a user for your DB and remember the password.

Creating a Mongoose model

Let’s create a Mongoose model and set up our connection string now, which is going to be our collection. No worries, it is just a JavaScript model.

var conString = "mongodb://admin:admin@ds038319.mlab.com:38319/mylearning"

/**
 * Models 
 */
var User = mongoose.model("User", {
    firstName: String,
    lastName: String
})

Please be noted that, when you move the codes to production, make sure that your password and user fields on the connection string are separated from this file and encrypted

Now let’s connect our database.

mongoose.connect(conString, { useMongoClient: true }, () => {
    console.log("DB is connected")
})

Setup the data

We have our model ready, what else is needed now? Yes, you are right, we need data.

var dummyUser = {
    firstName: "Sibeesh",
    lastName: "Venu"
}

Inserting the data into MongoDB

We have got everything ready, we can insert the model now. Let’s do the saving logic now.

mongoose.connect(conString, { useMongoClient: true }, () => {
    console.log("DB is connected")
    saveData()
})

function saveData() {
    var user = new User(dummyUser);
    user.save();
}

Verify the data

Once you run your application, go to your database and check for the entries there. I am sure there will be a new collection “User” and our inserted data. As I am using mLab database, here is how my data saved.

MongoDB Data Insertion

If you have noticed, you can see there are an another property names “_id” in our database collection. This is a unique id generated by MongoDB for us, so no need to think about it.

Todo

As the purpose of this article is to show how we can insert a data to MongoDB, I called the save method right after the database is connected. In our real life, this is not a recommended way. You must create an Ajax post request and handle the same. I am leaving that task to you. Please try to do that and let me know. I would love to hear back from you.

Conclusion

Thanks a lot for reading. Did I miss anything that you may think which is needed? Could you find this post as useful? I hope you liked this article. Please share me your valuable suggestions and feedback.

Your turn. What do you think?

A blog isn’t a blog without comments, but do try to stay on topic. If you have a question unrelated to this post, you’re better off posting it on C# Corner, Code Project, Stack Overflow, Asp.Net Forum instead of commenting here. Tweet or email me a link to your question there and I’ll definitely try to help if I can.

Kindest Regards
Sibeesh Venu