MongoDB - Replica Sets with Spring Data MongoDB (Part 4)

Review

In the previous section, we have introduced MongoDB Replica Sets, installed MongoDB, configured Replica Sets, and updated our Spring application. In this section, we will test our cluster to verify if we have achieved our goal.

Table of Contents

Part 1: Introduction
Part 2: Installation and configuration of Replica Sets
Part 3: Spring app configuration
Part 4: Testing the servers

Testing

Start Spring App

Kindly start our Spring application from Server 1. You may need to deploy the application first (it's up to you whether to deploy it on the cloud or on a localhost machine).

Notice the log output from Server 1. It acknowledges two connections from our Spring application, and it starts allocating space for the spring_mongodb_tutorial database:

Sat Feb 18 11:18:50 [conn14] getmore local.oplog.rs query: { ts: { $gte: new Date(5710309296642719745) } } cursorid:7697632718916849147 reslen:20 4588ms
Sat Feb 18 11:18:51 [initandlisten] connection accepted from 123.456.78.90:58023 #27
Sat Feb 18 11:18:53 [initandlisten] connection accepted from 123.456.78.90:58027 #28
Sat Feb 18 11:18:53 [conn28] CMD: drop spring_mongodb_tutorial.role
Sat Feb 18 11:18:53 [conn28] CMD: drop spring_mongodb_tutorial.user
Sat Feb 18 11:18:53 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.ns, filling with zeroes...
Sat Feb 18 11:18:54 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.ns, size: 16MB,  took 0.081 secs
Sat Feb 18 11:18:54 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.0, filling with zeroes...
Sat Feb 18 11:18:56 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.0, size: 16MB,  took 1.534 secs
Sat Feb 18 11:18:56 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.1, filling with zeroes...
Sat Feb 18 11:18:57 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.1, size: 32MB,  took 1.27 secs
Sat Feb 18 11:18:57 [conn28] build index spring_mongodb_tutorial.user { _id: 1 }
Sat Feb 18 11:18:57 [conn28] build index done 0 records 0.074 secs
Sat Feb 18 11:18:57 [conn28] insert spring_mongodb_tutorial.user 4131ms
Sat Feb 18 11:18:57 [conn28] build index spring_mongodb_tutorial.role { _id: 1 }
Sat Feb 18 11:18:57 [conn28] build index done 0 records 0 secs
Sat Feb 18 11:18:57 [conn14] getmore local.oplog.rs query: { ts: { $gte: new Date(5710309296642719745) } } cursorid:7697632718916849147 nreturned:4 reslen:1048 6948ms
Sat Feb 18 11:18:57 [conn13] getmore local.oplog.rs query: { ts: { $gte: new Date(5710309296642719745) } } cursorid:844650061822854635 nreturned:4 reslen:1048 5872ms

Watch how Server 2 synchronizes with Server 1

Sat Feb 18 11:18:51 [initandlisten] connection accepted from 123.456.78.90:43302 #20
Sat Feb 18 11:18:57 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.ns, filling with zeroes...
Sat Feb 18 11:18:57 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.ns, size: 16MB,  took 0.042 secs
Sat Feb 18 11:18:57 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.0, filling with zeroes...
Sat Feb 18 11:18:57 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.0, size: 16MB,  took 0.041 secs
Sat Feb 18 11:18:57 [rsSync] build index spring_mongodb_tutorial.user { _id: 1 }
Sat Feb 18 11:18:57 [rsSync] build index done 0 records 0 secs
Sat Feb 18 11:18:57 [rsSync] build index spring_mongodb_tutorial.role { _id: 1 }
Sat Feb 18 11:18:57 [rsSync] build index done 0 records 0 secs
Sat Feb 18 11:18:57 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.1, filling with zeroes...
Sat Feb 18 11:18:58 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.1, size: 32MB,  took 0.078 secs

Also, observe how Server 3 synchronizes with Server 1

Sat Feb 18 11:18:51 [initandlisten] connection accepted from 123.456.78.90:52987 #20
Sat Feb 18 11:18:57 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.ns, filling with zeroes...
Sat Feb 18 11:18:57 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.ns, size: 16MB,  took 0.043 secs
Sat Feb 18 11:18:57 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.0, filling with zeroes...
Sat Feb 18 11:18:57 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.0, size: 16MB,  took 0.044 secs
Sat Feb 18 11:18:57 [rsSync] build index spring_mongodb_tutorial.user { _id: 1 }
Sat Feb 18 11:18:57 [rsSync] build index done 0 records 0 secs
Sat Feb 18 11:18:57 [rsSync] build index spring_mongodb_tutorial.role { _id: 1 }
Sat Feb 18 11:18:57 [rsSync] build index done 0 records 0 secs
Sat Feb 18 11:18:57 [FileAllocator] allocating new datafile /data/db/spring_mongodb_tutorial.1, filling with zeroes...
Sat Feb 18 11:18:58 [FileAllocator] done allocating datafile /data/db/spring_mongodb_tutorial.1, size: 32MB,  took 0.091 secs

Server 4 does not synchronize because it's an arbiter-only server! Take note of that.

Start Killing

I don't mean killing people but rather MongoDB servers. Let's add a new record first. I have chosen mary, Mary, Jane, zzzzzzz, Regular as the properties of the new record. Feel free to vary.

Now, let's kill Server 1

Sat Feb 18 11:21:02 got kill or ctrl c or hup signal 2 (Interrupt), will terminate after current cmd ends
Sat Feb 18 11:21:02 [conn14] getmore local.oplog.rs query: { ts: { $gte: new Date(5710309296642719745) } } cursorid:7697632718916849147 exception: interrupted at shutdown code:11600 reslen:20 2565ms
Sat Feb 18 11:21:02 [conn13] getmore local.oplog.rs query: { ts: { $gte: new Date(5710309296642719745) } } cursorid:844650061822854635 exception: interrupted at shutdown code:11600 reslen:20 2565ms
Sat Feb 18 11:21:02 [interruptThread] now exiting
Sat Feb 18 11:21:02 Sat Feb 18 11:21:02 [conn14] got request after shutdown()
Sat Feb 18 11:21:02 [conn13] got request after shutdown()
dbexit: 
Sat Feb 18 11:21:02 [interruptThread] shutdown: going to close listening sockets...
Sat Feb 18 11:21:02 [interruptThread] closing listening socket: 5
Sat Feb 18 11:21:02 [interruptThread] closing listening socket: 6
Sat Feb 18 11:21:02 [interruptThread] closing listening socket: 8
Sat Feb 18 11:21:02 [interruptThread] removing socket file: /tmp/mongodb-27017.sock
Sat Feb 18 11:21:02 [interruptThread] shutdown: going to flush diaglog...
Sat Feb 18 11:21:02 [interruptThread] shutdown: going to close sockets...
Sat Feb 18 11:21:02 [interruptThread] shutdown: waiting for fs preallocator...
Sat Feb 18 11:21:02 [interruptThread] shutdown: closing all files...
Sat Feb 18 11:21:02 [interruptThread] closeAllFiles() finished
Sat Feb 18 11:21:02 [interruptThread] shutdown: removing fs lock...
Sat Feb 18 11:21:02 dbexit: really exiting now
Sat Feb 18 11:21:02 [conn1] end connection 127.0.0.1:39621
Logstream::get called in uninitialized state
Sat Feb 18 11:21:02 [conn38] end connection 173.204.91.83:37424

Reload our Spring application. Notice, we still have the same data. We lost nothing. The replicas are working!

Kill More

Let's kill more servers. Add new record again. I have chosen anna, Anna, Williams, zzzzzzz, Admin as the properties of the new record.

Kill Server #2. This means only Server 3 and Server 4 are running, one slave and one arbiter remains respectively. Refresh our Spring application. However the application fails to load the data. Why?

The log explains why:

Sat Feb 18 11:23:46 [rsSync] replSet syncThread: 10278 dbclient error communicating with server: 173.204.91.84:27017
Sat Feb 18 11:23:46 [conn42] end connection 123.456.78.92:53291
Sat Feb 18 11:23:47 [rsHealthPoll] couldn't connect to 123.456.78.90:27017: couldn't connect to server 173.204.91.90:27017
Sat Feb 18 11:23:48 [rsHealthPoll] DBClientCursor::init call() failed
Sat Feb 18 11:23:48 [rsHealthPoll] replSet info 123.456.78.92:27017 is down (or slow to respond): DBClientBase::findN: transport error: 123.456.78.92:27017 query: { replSetHeartbeat: "cluster1", v: 1, pv: 1, checkEmpty: false, from: "123.456.78.93:27017" }
Sat Feb 18 11:23:48 [rsHealthPoll] replSet member 123.456.78.92:27017 is now in state DOWN
Sat Feb 18 11:23:48 [rsMgr] replSet can't see a majority, will not try to elect self

It's because our servers can't elect a primary server. In general, you need two non-arbiter servers in order to have a primary server! The arbiter is designed for breaking the tie-break between equal votes. Now let's restore the other two dead servers. Run mongod again for in Server 1 and Server 2.

Refresh our Spring application again. Notice we have all four records, including Mary and Anna. Remember when we added Anna, the Server #1 was dead!

Let's kill Server 2 and see if Anna is still displayed. Yes, it's diplayed!

Let's kill Server 3 and see if Anna is still displayed. And run Server 3 again. Yes, it's still diplayed!

Conclusion

That's it! We've successfully implemented a MongoDB Replica Set. We have shown step-by-step how to configure our servers and our Spring application. We've also made some quick tests to verify the replication feature of our servers.

---

Share the joy: blogger tutorials Social Bookmarking Blogger Widget I'm reading: MongoDB - Replica Sets with Spring Data MongoDB (Part 4) ~

Subscribe by reader Subscribe by email Share