Developing Java and Node JS Application with MongoDB Tutorial

Welcome to the sixth chapter of the MongoDB tutorial (part of the MongoDB Developer and Administrator Course). This chapter will explain the process of developing JAVA and Node JS applications using MongoDB.  

Let us explore the objectives of Java and Node JS application with MongoDB in the next section.

Objectives

After completing this lesson, you will be able to:

  • Explain what capped collections are

  • Explain how Grid File System, or GridFS (Read as Grid F-S), is used for storing large data

  • Explain how to create a connection and perform the CRUD operation using Java Node JS (Read as J-S) applications

  • Identify the steps to install the Java driver

  • Explain the process to create a collection from a Java program

  • Explain the procedure to use the Java program to perform functions, such as insert, update, and retrieve

In the next section, we will begin with what capped collections are.

Capped Collection

Capped collections in MongoDB are collections with predefined sizes and support operations that insert and retrieve documents. Capped collections function like circular buffers; once a collection fills its allocated space, it overwrites older documents to allocate space for new documents.

Capped collections exhibit the following behaviors:

  • They preserve an insertion order. Therefore, queries can return documents in the insertion order without the help of an index. This helps them support higher insertion throughput.

  • They ensure that the insertion order functions similar to the order on the disk or the natural order. They do so by prohibiting updates that increase document size. They allow only those updates that fit the original document size. This ensures that a document’s location on the disk remains unaltered.

  • They delete the oldest documents in a collection automatically. They do not require scripts or explicit remove operations to do so.

In the next section, we will discuss how to create a capped collection.

Interested in learning more about MongoDB? Enroll in our MongoDB course today!

Capped Collection Creation

To create a capped collection, you need to use the createCollection() method explicitly.

When creating a capped collection, you need to specify the maximum size of the collection in bytes. MongoDB will pre-allocate the size of the collection. The size must include space for the internal overhead.

Use the syntax given below to specify the size.

db.createCollection( "serverlog," { capped: true, size: 100000 } )

If the specified size field is less than or equal to 4096 (Read as four thousand ninety-six) bytes, then the collection will have a cap of 4096 bytes. Otherwise, MongoDB will increase the size to make it an integer multiple of 256 bytes.

Additionally, you may also specify the maximum number of documents for the collection using the max field as shown below.

db.createCollection("server", { capped : true, size : 5242880, max : 5000 } )  


We will continue our discussion on the creation of a capped collection in the next section.

How to create a capped collection in MongoDB

 When creating a capped collection, you need to do the following:

  • Query a Capped Collection

    If you perform the find() query on a capped collection without specifying any order, MongoDB ensures that the ordering of the results is the same as the insertion order.
    To retrieve documents in the reverse insertion order, issue the find() query along with the sort() method and set the $natural parameter to -1 (Read as minus one), as shown in the example given below.

    db.cappedCollection.find().sort( { $natural: -1 } )

  • Check if a Collection is Capped

    Use the isCapped() method to determine if a collection is capped, as given in the command below.

    db.collection.isCapped()

  • Convert a Collection to Capped

    Convert a non-capped collection to a capped collection with the convertToCapped (Read as convert to capped) command.

    db.runCommand({"convertToCapped": "items", size: 100000});

In the next section, we will discuss capped collection restriction.

Capped Collection Restriction

When updating a capped collection, you can make only in-place updates to documents. If the update operation increases the original document size, the operation does not succeed.
To update documents in a capped collection, remember the following:

  • Create an index to avoid a table scan.

  • When you update a document to a smaller size than the original document size, a secondary resyncs from the primary.

  • The secondary replicates and allocates space based on the current size of the updated document.

  • If the primary receives an update which restores the original document size, the primary accepts the update, whereas the secondary fails. The secondary receives the error message “objects in a capped ns (Read as N-S) cannot grow.”

Note that you cannot remove documents from a capped collection. However, you can remove all documents from a collection by dropping the entire collection using the drop() method. You also cannot shard a capped collection.
In the next section, we will discuss TTL collection features.

TTL Collection Features

For a capped collection, data is automatically removed once the collection grows beyond the threshold size and the defined number of documents. To allow flexibility for expiring data, consider MongoDB’s time to live, or TTL, indexes.

The features of TTL indexes are as follows:

  • TTL indexes help delete data from normal collections using a special type field and a TTL value for the index.

  • For TTL collections, mongod (Read as mongo D) automatically removes data after a specified duration of time in seconds or at a specific clock time.

Note that TTL collections are not compatible with capped collections.
A special TTL index property supports the implementation of TTL collections as shown below.

db.log_events.createIndex( { "createdAt": 1 }, { expireAfterSeconds: 3600 } )

You can use a tailable cursor with capped collections. Similar to the Unix tail -f command, the tailable cursor “tails” the end of a capped collection. As new documents are inserted into the capped collection, you can use the tailable cursor to continue retrieving documents.

In the next section, we will discuss GridFS in MongoDB.

Want to test your MongoDB skills? Take our MongoDB Free Practice Test Today

GridFS

GridFS is a specification used for storing and retrieving files exceeding 16 MB, the BSON-limit for document size.

GridFS does the following:

  • GridFS splits a file into different parts or chunks and stores each unit as a separate document. The default chunk size specified by GridFS is 255 KB.

  • GridFS stores files in two collections. In one collection, the file chunks are stored, whereas the metadata is stored in the other collection.

  • Allows range queries on files stored in it. For example, you can “skip” into the middle of a video or an audio file to access information.

  • GridFS allows you to access any files without having to load the entire file into memory.

When returning a query result, from GridFS, the driver or client reassembles the chunks as required.

In the next section, we will discuss GridFS collection.

GridFS Collection

You can store and retrieve files from GridFS using a MongoDB driver or the “mongofiles” command-line tool in the mongo shell.

The “mongofiles” command-line tool in the mongo shell GridFS stores files in two collections:

  • FS.chunks (Read as F-S dot chunks) store the binary chunks.

  • FS.files (Read as F-S dot files) store the file’s metadata.

You can choose a different bucket name and also create multiple buckets in a single database. Each document in the chunks collection represents a distinct chunk of a file as represented in the GridFS store. Each chunk is identified by its unique ObjectId stored in its _id field.

Following are the characteristics of GridFS collections:

  • GridFS Index uses a unique, compound index on the chunks collection for the files_id (Read as files underscore ID) and n fields. The files_id field contains the _id of the chunk’s “parent” document. The n field contains the sequence number of the chunk.

  • GridFS numbers all chunks, starting with 0 (Read as zero).

  • For the descriptions of the documents and fields in the chunks collection, GridFS index allows efficient retrieval of chunks using the files_id and n values, as shown in the following example given below.

cursor = db.fs.chunks.find({files_id: myFileID}).sort({n:1});

If your driver does not create this index, issue the following operation given below using the mongo shell.

db.fs.chunks.createIndex( { files_id: 1, n: 1 }, { unique: true } );

In the next section, we will discuss MongoDB drivers and client libraries.

MongoDB Drivers and Client Libraries

Applications communicate with MongoDB through client libraries or drivers that handle all the interactions with the database in a language that is appropriate and sensible. The following languages are used for communication:

  • JavaScript

  • Python

  • Ruby

  • PHP

  • Perl

  • Java

  • Scala

  • C++ Haskell

  • Erlang

In the next section, we will discuss how to develop a Java Application with MongoDB.

Develop Java Application with MongoDB

MongoDB uses the Java driver from its earliest versions. The Java driver is the popular choice among enterprise developers because of its stability.

To install the Java driver, perform the following steps.

Step 1: The Java driver comes as a JAR file. You can download this JAR file from the link given below.

https://github.com/mongodb/mongo-java-driver/downloads

Step 2: Add the JAR file to your classpath.
The Java classes you need to use in a normal application are in the com.mongodb (Read as com dot MongoDB) and com.mongodb.gridfs (Read as com dot MongoDB dot grid FS) packages. The JAR file contains a number of other packages. 
You need these packages only to either modify the driver’s internal functionality or extend its functionality.
In the next section, we will discuss how to connect to MongoDB from a JAVA Program.

Connecting to MongoDB from Java Program

To create a normal Java application, you need to first create a new Java project in eclipse and add the MongoDB Java driver into its build and classpath. The Java driver of MongoDB provides the “com.mongodb.Mongo”class to create a connection to the MongoDB server. 

To establish a connection, you need to create a new Mongo Object and pass the IP address and port number of the MongoDB server as a parameter, as shown in the code given below.

import com.mongodb.MongoClient;

import com.mongodb.DB;

import com.mongodb.DBCollection;

class itemSearch {

public static void main(String[] args)

{

MongoClient mongoClient = new MongoClient("localhost", 27017);

DB db = mongoClient (“test");

DBCollection items = db.getCollection(“items");

} }

Once a mongo object is created, you can create a MongoClient instance to access the database and perform create, retrieve, update, delete or CRUD operations.

In the next section, we will discuss how to create a collection from a Java program.

Create Collection From Java Program

You can create a collection from a Java program using the createCollection()(Read as create collection) method of the com.mongodb.DB (Read as com dot MongoDB dot DB) class. A DB object instance is created from the return object of the mongoClient.getDB()(Read as mongo client dot get DB) method.

The code shown below is used for first connecting to the MongoDB server by creating a mongoClient object.

public static void main(String args[]) {

try {

// To connect to mongodb server

MongoClient mongoClient = new MongoClient("localhost", 27017);

// Now connect to your databases

DB db = mongoClient.getDB("test");

DBCollection coll = db.createCollection("mycol", null);

} catch (Exception e) {

System.err.println(e.getClass().getName() + ": " + e.getMessage());

}

}

}  

After a mongoClient object is created, specify the name of the database in the getDB() method call of the mongoClient object and then connect to the database. Use the DB object to call the createCollection method to create a collection in MongoDB.

In the next section, we will discuss how to insert documents using a Java program.

Insert Documents From Java Program

Documents in Java must be instances of org.bson.DB Object, (Read as org dot B-SON dot DB Object) an interface that is an ordered “java.util.Map”(Read as java dot util dot map). A document can be created in multiple ways in Java.

However, the simplest way to create one is using the “com.mongodb.BasicDB Object” (Read as com dot MongoDB dot basic DB Object) class. To create a document that is represented by the shell as item “Book”, “SoldQty": 500 (Read as Sold quantity five hundred), use the command given below.

BasicDBObject doc = new BasicDBObject();
doc.put(“item”:“Book");
doc.put("SoldQty", 500);

In the next section, we will discuss how to insert documents using the Java code example.

Insert Documents Using Java Code Example

To insert a document in a collection, do the following:

  • Create an instance of the DBCollection object by calling the getCollection() method of the DB object.

  • Create BasicDBObject to represent the JSON document in the Java code.

  • Call the append method to set the values for all the fields that need to be populated.

  • Call the insert method of DBCollection after BasicDBObject is initialized. This inserts the document into the collection by passing BasicDBObject as a parameter.

In the next section, we will discuss how to u[dating documents using Java.

Update Documents Using Java Code

The db.collection.update (Read as D-B dot collection dot update) method is used to update an existing document in a collection. For updating documents, you can use the update methods of DBCollection. 

In the code given below, the DBCursor object is used to iterate through the list of documents in the while loop and update each document by inserting the field “like” with its value as 200.

public class UpdateDocuments {

public static void main(String args[]) {

try {

DBCollection coll = db.getCollection("items");

DBCursor cursor = coll.find();

while (cursor.hasNext()) {

DBObject updateDocument = cursor.next();

updateDocument.put("likes", "200");

coll.update(null, updateDocument); } 

In the next section, we will discuss how to delete documents using Java.

Delete Documents Using Java Code

To remove documents from a collection, use the DBCollection’s remove method.

public class DeleteDocument { public static void main(String args[]) { try {

// To connect to mongodb server

DBCollection coll = db.getCollection("mycol");

System.out.println("Collection mycol selected successfully");

DBObject myDoc = coll.findOne(); coll.remove(myDoc);

DBCursor cursor = coll.find();

In the code given above, you retrieve the document to be deleted by calling the findOne method. You also pass the retrieved document in the remove method as a parameter to remove it from the collection. In the next screen, we will view a demo on how to delete a document using Java.

Store Images Using GridFS API

In the code shown below, “SaveImageApp” (Read as save image app) written in Java will store images in MongoDB using the GridFS function.

public class SaveImageApp {

public static void main(String[] args) { try {

Mongo mongo = new Mongo("192.168.133.128", 27017); DB db = mongo.getDB("imagedatabase1");

//DBCollection collection = db.getCollection("test");

String newFileName = "gridFS-java-image";

File imageFile = new File("c:\\butterfly.jpg");

// create a "photo" namespace GridFS gfsPhoto = new GridFS(db, "photo");

GridFSInputFile gfsFile = gfsPhoto.createFile(imageFile);

gfsFile.setFilename(newFileName);

// save the image file into mongoDB gfsFile.save();

For this, first establish a connection with the MongoDB server by creating a new Mongo()(Read as mongo)object. This object constructor takes the IP address and port number as parameters. To insert an image into MongoDB, you need to create an instance of the GridFS object by passing the DB instance and the collection name as parameters.  

Next, you need to pass the image path to the createFile method of the GridFS object to store the image. Here, you are storing a butterfly image from the “C:\\butterfly.jpg” (Read as C drive) into the “photo” collection. This image will be inserted as a document with the “filename” field as “gridFS-java-image.”

In the next section, we will discuss how to retrieve images using GridFS API.

Retrieve Images Using GridFS API

To retrieve all the DBCursor objects that have reference to the files stored in GridFS, you can use a GridFS object instance and call the “getFileList ”(Read as get file list) method. You can then iterate through each file in the while loop to print all file names.

In addition, you can use the findOne method of GridFS to fetch the files based on the file names. This will return the GridFSDBFile (Read as GridFS DB file) object.

To retrieve DBCursor objects having reference to the files stored in GridFS, use the code given below:

DBCursor cursor = gfsPhoto.getFileList(); while (cursor.hasNext())

{

System.out.println(cursor.next());

}

// get image file by it's filename

GridFSDBFile imageForOutput = gfsPhoto.findOne(newFileName);

} }  

In the next section, we will discuss how to remove images using GridFS API.

Remove Image Using GridFS API

To remove the image file from GridFS, use the code below. You can use an instance of the GridFS object and call the remove method.

// remove the image file from mongoDB

gfsPhoto.remove(gfsPhoto.findOne(newFileName));

}

catch (UnknownHostException e) { e.printStackTrace();

}

catch (MongoException e) { e.printStackTrace();

}

catch (IOException e) { e.printStackTrace();

}

}

}

In the next section, we will continue our discussion on removing images using GridFS API.

Remove Image Using GridFS API (contd..)

To work with a database, you first need to create a connection. You can use MongoDB’s native Node.js (Read as Node J-S) driver to create a connection with the MongoDB server.

To install the MongoDB native drivers, first install the MongoDB module by using the npm (Read as N-P-M) command. Next, run the command given below in your project directory.

npm install mongodb

npm is a package manager. It provides a central repository for custom open source modules for Node JS and JavaScript. npm helps manage modules, their versions, and distribution easily.

The npm install command is used to install the required module in a project.

In the next section, we will discuss how to create a connection using Node JS.

Connection Creation Using Node JS

To create a connection using Node JS, perform the following steps.

Step 1: To create a connection to the database, first load the MongoDB module. The code given below uses “require” to load the MongoDB module.

//lets require/import the mongodb native drivers. var mongodb = require('mongodb');

The MongoDB module represents the native MongoDB drivers for Node JS.

Step 2: Define the URL you need to connect to.

// Connection URL. This is where your mongodb server is running. var url = 'mongodb://localhost:27017/test';

You need to know where the MongoDB server is running. The URL provides the location where the MongoDB server instance is running. The URL also contains the database name to which you want to connect.

Step 3: Connect to the database: Use the MongoClient interface’s connect method to connect to the database.

MongoClient.connect(url, function (err, db) {}

In the callback, you either get an error or get the db object. Use the db object to communicate with the database.

In the next section, we will discuss how to perform insert operations using Node JS.

Insert Operations Using Node JS

After you have a db connection ready, you can communicate with the database. You can perform some basic operations on the MongoDB database.
When using the MongoDB native driver, you need to ensure that:

  • The query function names and their parameters are similar to those of the native MongoDB commands.

  • The query functions take the callback as the last argument. The callback function returns the first argument as an error and the second argument as a result. The result is actually the result or output provided by MongoDB on running these commands.

This is similar to the results you get when you run these queries in the MongoDB shell. 

In the next section, we will continue our discussion on performing the insert operation using Node JS.

Insert Operations Using Node JS (contd.)

Suppose you need to create a collection of items and save a few of them in it. For this, you will use the insert method to save the item’s information.

To create an items collection and save a few items in it, use the code given below.

// Get the documents collection var collection = db.collection(‘items');

//Create some users var item1 = {item: ‘Book', SoldQty: 500}

var item2= {item: ‘Pencil', SoldQty : 400}

var item3= {item: ‘Pen', SoldQty : 300}

// Insert some items collection.insert([item1,item2,item3],

function (err, result) {}

In the insert operation, you perform the following steps:

  • Use the db.collection method: Use the db.collection method to get the reference to your items collection. If the collection does not exist, then it will be created on the first insert. If it already exists, then you will be working on the existing one. You need not explicitly create a collection.

  • Prepare data to be inserted: Create some sample item record objects named item1, item2, and item3. There is no fixed schema of the document; they are simple JS objects with the entity's data.

  • Insert into the database: Use collection.insert (Read as collection dot insert) to insert your data into the database collection. Pass an array of items to it and it saved all of them in one step.

In the next section, we will discuss how to update operations using Node JS.

Update Operations Using Node JS

You can update records using the collection.update (Read as collection dot update) method. The name and parameters of the update method are similar to that of native MongoDB queries. The code given below updates the documents from Node JS.

// Get the documents collection var collection = db.collection(‘items');

// Insert some users collection.update({item: ‘Pencil'}, {

$set: {available: false}}, function (err, numUpdated) {

if (err) { console.log(err); } else if (numUpdated) {

console.log('Updated Successfully %d document(s).', numUpdated); } else {

console.log('No document found with defined "find" criteria!');

}

In the next section, we will discuss how to retrieve documents using Node JS.

Retrieve Documents Using Node JS

To retrieve documents from MongoDB, trigger the find command on the collection object just as in the MongoDB shell. The example given below is used for retrieving a document.

// Get the documents collection var collection = db.collection(‘items');

collection.find({item ‘Pencil'}).toArray(function (err, result) {

if (err) { console.log(err); } else if (result.length) {

console.log('Found:', result); } else {

console.log('No document(s) found with defined "find" criteria!'); }

In this code, toArray is used along with the find method. Running the find method alone returns a db cursor. You will need to iterate over the cursor to fetch the results. For performance reasons, MongoDB does not fetch the complete record data by default. Using toArray tells the driver to fetch the full data of each item.

In the next section, we will discuss how to use the DB cursor to retrieve documents.

Using DB Cursor to Retrieve Documents

You can use the db cursor instead of each document or use toArray to return the full array of all documents.

var collection = db.collection(‘items');

//We have a cursor now with our find criteria

var cursor = collection.find({item: ‘Book'});

cursor.sort({item: -1});

cursor.limit(10);

cursor.skip(0);

//Lets iterate on the result cursor.each(function (err, doc) { if (err) {

console.log(err); } else {

console.log('Fetched:', doc); }

}

);

Using each will load the documents with each iteration. In addition, some other methods are available, such as “remove” to delete documents, “count” to count the number of documents, and so on.

In the next section, we will discuss the Mongoose Object Design Management or ODM module.

Mongoose ODM Module in Node JS

MongooseJS (Read as Mongoose JS) is a popular library in Node JS. It provides a straightforward, schema-based solution to model your application data and includes built-in typecasting, validation, query building, business logic hooks, and so on. 

Using the Mongoose interface features, such as automatic connection and connection pool management, you can easily work with MongoDB.
To work with Mongoose, you need to first install the mongoose module. Run the npm install mongoose command to install the mongoose module.

In the next section, we will discuss how to define a schema using Mongoose.

Defining Schema Using Mongoose

In the code shown below, do the following things

  1. Connecting to the database: You use the mongoose.connect (Read as mongoose dot connect) method to connect to the desired database. MongooseJS will then internally handle the connection and the connection pool as configured.

  2. Creating a Model: You create a model using the mongoose.model (Read as mongoose dot model) method.

The model represents a collection in the database.

/Lets load the mongoose module in our program

var mongoose = require('mongoose');

//Lets connect to our database using the DB server URL. mongoose.connect('mongodb://localhost/my_database_name');

/** * Lets define our Model for User entity.

This model represents a collection in the database.

* We define the possible schema of User document and data types of each field. *

*/ var User = mongoose.model('User', {name: String, roles: Array, age: Number});

/* Lets Use our Models *

*/ //Lets create a new user var user1 = new User({

name: 'modulus admin', age: 42, roles: ['admin', 'moderator', 'user']}

);

The name you provide to the model will be pluralized internally and the collection with that plural name will be created in MongoDB.

For example, if you name the model “User”, then the collection will be named “users” with lowercase. You can also define the schema for a document during this process.

We will continue our discussion on defining a schema using Mongoose in the next section.

Defining Schema Using Mongoose (contd.)

You perform the following activities using the code given below.

  1. Create a new User Object: You can create a new User object using the model. You can create as many user objects using the model as desired. Each object will represent a document in the database. In addition, you can modify and save the object.

  2. Save the user using the 'save' method: The mongoose model has many methods available on the entity objects. In this case, you will use the save method to save the document in the database.

You can also use the model to query the database. To query the database collection, you can use the model defined to do that.

 In the example given below, you try to find a user and then update it.

//Some modifications in user object

user1.name = user1.name.toUpperCase();

//Lets try to print and see it. You will see _id is assigned. console.log(user1);

//Lets save it user1.save(function (err, userObj) {

if (err) { console.log(err); } else {

console.log('saved successfully:', userObj); }

}

);

Summary

Here is a quick recap of what was covered in this lesson:

  • Capped collections in MongoDB contain collections with predefined sizes that support insert and retrieve operations.

  • When creating a capped collection, you need to specify the size, query a collection, check if the collection is capped, and convert a collection to capped.

  • The GridFS specification splits a file into different parts or chunks and stores each unit as a separate document.

  • To install the JAVA driver in MongoDB, you need to add the JAR file to your classpath.

  • You can create a collection from a Java program using the createCollection() method of the com.mongodb.DB (Read as com dot MongoDB dot DB) class.

  • You can use the db cursor instead of each or use toArray to return the full array of all documents.

  • You can use the native MongoDB module and Mongo client to connect to MongoDB from Node js.

  • You can use the db.collection method to get the reference to your collection and then use insert, find, and delete methods.

Conclusion

This concludes the lesson Developing Java and Node JS application with MongoDB. In the next lesson, we will discuss Administration of MongoDB Cluster Operations.

Find our MongoDB Developer and Administrator Online Classroom training classes in top cities:


Name Date Place
MongoDB Developer and Administrator 12 Jan -3 Feb 2019, Weekend batch Your City View Details
  • Disclaimer
  • PMP, PMI, PMBOK, CAPM, PgMP, PfMP, ACP, PBA, RMP, SP, and OPM3 are registered marks of the Project Management Institute, Inc.

Request more information

For individuals
For business
Name*
Email*
Phone Number*
Your Message (Optional)
We are looking into your query.
Our consultants will get in touch with you soon.

A Simplilearn representative will get back to you in one business day.

First Name*
Last Name*
Email*
Phone Number*
Company*
Job Title*