{"id":81,"date":"2010-04-25T23:31:50","date_gmt":"2010-04-26T05:31:50","guid":{"rendered":"https:\/\/www.chrisedwards.dreamhosters.com\/blog\/2010\/04\/25\/test-driving-mongodb\/"},"modified":"2011-08-21T15:08:45","modified_gmt":"2011-08-21T21:08:45","slug":"test-driving-mongodb","status":"publish","type":"post","link":"http:\/\/architester.com\/blog\/2010\/04\/25\/test-driving-mongodb\/","title":{"rendered":"Test Driving MongoDB"},"content":{"rendered":"

Lately, I have been looking into MongoDB<\/a> and other alternatives to traditional relational databases. MongoDB seems to be the best fit for the kinds of scenarios I am working with here at BancVue<\/a>.<\/p>\n

What I am looking for\u00e2\u20ac\u00a6<\/h2>\n

The project I am currently working on is using messaging to transfer large amounts of data from many remote source systems. This data then hydrates a master database that represents a consolidated view of all the data from those remote systems. This master system can then feed other systems the data they need. Based on this system, here is a list of the qualities I need:<\/p>\n

Fast Inserts<\/strong>
\nBecause there is little querying, but large amounts of inserts, they need to be fast.<\/p>\n

Handle Large Data Volumes<\/strong>
\nThe consolidated dataset can get very large. Therefore we need a system that can handle massive amounts of data.<\/p>\n

Horizontally Scalable<\/strong>
\nBecause we need to house so much data, it requires us to scale to multiple machines. Ideally, the system would easily scale to multiple machines as needed (something SQL-based databases don\u00e2\u20ac\u2122t do very well).<\/p>\n

Parallelized Queries<\/strong>
\nSince we do have a need to query the database to get the data out, it would be nice if the system could parallelize the queries so they would be more performant when scaled out.<\/p>\n

Easy to Setup and Use<\/strong>
\nI hate the thought of working on something that takes a UNIX guru to set up (because I am not that guru), and I don\u00e2\u20ac\u2122t have a whole heck of a lot of time to devote to learning a new system at the moment.<\/p>\n

MongoDB fulfills all these needs. It has a very fast insert speed, can scale to thousands of machines, can automatically shard the data across those machines to store large volumes of data, and run map\/reduce queries in parallel across those machines to produce results. Not to mention that it is easier to get running than any db I have ever used.<\/p>\n

Setting up MongoDB<\/h2>\n

Setting up an instance of MongoDB could not be much easier. Simply follow the instructions on the MongoDB Quickstart<\/a> page:<\/p>\n

    \n
  1. Download<\/a> the binaries and extract them.<\/li>\n
  2. Create a folder for the data. (C:\\data\\db<\/span> or \/data\/db<\/span> depending on OS).<\/li>\n
  3. Execute mongod.exe<\/span> in the extracted bin folder.<\/li>\n<\/ol>\n

    Now you have a running instance of MongoDB!<\/p>\n

    \"Screenshot<\/p>\n

    Connecting to the MongoDB Instance<\/h2>\n

    MongoDB comes with its own interactive shell. Run mongo.exe<\/span> from the extracted bin folder to start it up. It will automatically connect to the instance we started up on our local box. (You can use command line args to connect to instances on other machines of course).<\/p>\n

    \"image\"<\/a><\/p>\n

    From here, you can interactively execute commands against the database.<\/p>\n

    Working in the Shell<\/h2>\n

    MongoDB does not use SQL, instead, it uses JavaScript as its query language. This is all very well documented<\/a> on the MongoDB website. Lets walk through a few commands to get you started. The text in green<\/span> is the command you type in. The text in blue<\/span> is the server\u00e2\u20ac\u2122s response.<\/p>\n

    Getting a list of databases<\/h3>\n

    The show dbs<\/strong><\/span> command will display a list of all the databases on this server.<\/p>\n

    > show dbs<\/span><\/span><\/p>\n

    admin<\/span><\/p>\n

    local<\/span><\/p>\n

    mongo_session<\/span><\/p>\n

    test<\/span><\/p>\n

    > <\/span><\/p><\/blockquote>\n

    This is the list of all the databases in your mongo server.<\/p>\n

    Switching to use a different database<\/h3>\n

    To switch to another database you use the use<\/span> command just like in SQL.<\/p>\n

    > use myorders<\/span><\/span><\/p>\n

    switched to db myorders<\/span><\/p>\n

    > <\/span><\/p><\/blockquote>\n

    Notice that you may pass a new database name to the use command and it will work. Actually, MongoDB will not create the database until you actually insert something. You can see this by executing show dbs<\/span> again. The myorders<\/span> database does not show in the list.<\/p>\n

    Inserting data into a collection<\/h3>\n

    MongoDB uses the concept of collections in the same way Sql uses tables. However, since MongoDB is a document database, it does not constrain all the objects in a collection to the same structure like tables do. Each object can have its own structure (or schema). Thus, MongoDB is called a \u00e2\u20ac\u0153schema-less\u00e2\u20ac\u009d database.<\/p>\n

    Lets insert some data into a collection now.<\/p>\n

    > order1 = {orderAmount:25.00, customerName:\u00e2\u20ac\u009dBob Smith\u00e2\u20ac\u009d};<\/span><\/span><\/p>\n

    { \u00e2\u20ac\u0153orderAmount\u00e2\u20ac\u009d : 25, \u00e2\u20ac\u0153customerName\u00e2\u20ac\u009d : \u00e2\u20ac\u0153Bob Smith\u00e2\u20ac\u009d }<\/span><\/p>\n

    > db.orders.save(order1);<\/span><\/span><\/p>\n

    ><\/span><\/p><\/blockquote>\n

    So what did we just do?<\/strong>
    \nWe created a variable called order1, and assigned it a JSON object. (Did I mention it was all JavaScript based?). Then we inserted order1 into the orders collection.<\/p>\n

    Where did the orders collection come from?<\/strong>
    \nIt was created automatically by MongoDB. Likewise, the database was created at the same time. If you want to run show dbs<\/span> again now, you will see the myorders<\/span> database.<\/p>\n

    > show dbs<\/span><\/span><\/p>\n

    admin<\/span><\/p>\n

    local<\/span><\/p>\n

    mongo_session<\/span><\/p>\n

    myorders<\/span><\/p>\n

    test<\/span><\/p>\n

    > <\/span><\/p><\/blockquote>\n

    Querying the database<\/h3>\n

    Lets now look at what was inserted. First, we\u00e2\u20ac\u2122ll look at all the records in the collection. We do this by calling the find()<\/span> function on the collections with no arguments.<\/p>\n

    > db.orders.find();<\/span><\/span><\/p>\n

    { \u00e2\u20ac\u0153_id\u00e2\u20ac\u009d : ObjectId(\u00e2\u20ac\u01534bd\u00e2\u20ac\u00a6\u00e2\u20ac\u009d), \u00e2\u20ac\u0153orderAmount\u00e2\u20ac\u009d : 25, \u00e2\u20ac\u0153customerName\u00e2\u20ac\u009d : \u00e2\u20ac\u0153Bob Smith\u00e2\u20ac\u009d }<\/span><\/p>\n

    > <\/span><\/p><\/blockquote>\n

    You will notice the introduction of a new field called _id. This is an autogenerated id that serves as the primary key. You can override this key if you like, but that is beyond the scope of this test drive.<\/p>\n

    This is the simplest kind of query, we can also query by giving a prototype object to match on:<\/p>\n

    > db.orders.find( { orderAmount: 25 } );<\/span><\/span><\/p>\n

    { \u00e2\u20ac\u0153_id\u00e2\u20ac\u009d : ObjectId(\u00e2\u20ac\u01534bd\u00e2\u20ac\u00a6\u00e2\u20ac\u009d), \u00e2\u20ac\u0153orderAmount\u00e2\u20ac\u009d : 25, \u00e2\u20ac\u0153customerName\u00e2\u20ac\u009d : \u00e2\u20ac\u0153Bob Smith\u00e2\u20ac\u009d }<\/span><\/p>\n

    > <\/span><\/p><\/blockquote>\n

    Notice we got the same result\u00e2\u20ac\u00a6it found the one record. If you supply an object to the find method, it will search for all items that match all the fields provided.<\/p>\n

    Conclusion<\/h2>\n

    Well, I hope you found this test drive useful. Check out the documentation<\/a> for more information.<\/p>\n

    I am impressed with the ease of setup and use of MongoDB so far. Next time, we\u00e2\u20ac\u2122ll look at accessing MongoDB through C#.<\/p>\n

    -Chris<\/p>\n","protected":false},"excerpt":{"rendered":"

    Lately, I have been looking into MongoDB and other alternatives to traditional relational databases. MongoDB seems to be the best fit for the kinds of scenarios I am working with here at BancVue. What I am looking for\u00e2\u20ac\u00a6 The project I am currently working on is using messaging to transfer large amounts of data from […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24,23],"tags":[81,80],"class_list":["post-81","post","type-post","status-publish","format-standard","hentry","category-mongodb","category-nosql","tag-mongodb","tag-nosql"],"aioseo_notices":[],"_links":{"self":[{"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/posts\/81","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/comments?post=81"}],"version-history":[{"count":6,"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/posts\/81\/revisions"}],"predecessor-version":[{"id":211,"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/posts\/81\/revisions\/211"}],"wp:attachment":[{"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/media?parent=81"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/categories?post=81"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/architester.com\/blog\/wp-json\/wp\/v2\/tags?post=81"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}