I recently had a chance to introduce MongoDB in one of my projects. This application was a data collection system. Its main purpose was to be able to ingest large amounts of data from different data sources. These sources had different formats for data. Some were in XML, some were in CSV and some were in binary. They all had one thing in common though and that was the large size of data. The core of the application was written in a multi-threaded .NET application that was using SqlBulkCopy with a SQL Sever 2012 addition. SqlBulkCopy is one of the preferred (optimized) ways of loading data into SQL Server. The goal of the application was to be able to parse any given file with its given format definition/configuration file. Therefore, it needed to be able to store data into SQL Server that it never stored before. This provided challenges since Relational Data Management Systems require you to define your data structure before you can store data (i.e you must create your tables). Ther...
Keeping me and perhaps you up to date with focus on technology