Skip to main content

Posts

Showing posts from 2014

How to use Android Debug Bridge to copy data out of your Android device

I recently built an Android application that had the capability to export data in XML format to the devices external storage directory. External storage directory path in Android OS can be accessed using the android.os.Environment package. Following code return the path for the external storage directory:  Environment.getExternalStorageDirectory().getPath() To copy data from this storage into your local device, you can use the Android Debug Bridge command line tool which comes installed with the Android SDK platform.  The following command will pull a single file named filename.ext out of the device and copy it to your local computer: adb pull /sdcard/filename.ext ./filename.ext If you want to copy multiple files with certain extension to your local computer you can use Adb's shell command capability.  The following command will pull all files with .xml extension to your local file for example: adb shell ls /sdcard/*.xml | tr '\r' '

Useful MongoDB tools

I have been developing data driven applications for over 10 years now. I have used many different database systems to solve many different types of problems. I think it is suffice to say that good tooling is a must for any technology. I have always thought that Microsoft ecosystem provided very decent tooling for its development community. I like using Visual Studio when I am programming .NET based applications. I like using Eclipse when I am programming in Java. Sql Server Management Studio and SqlDeveloper is usually what use when I am working with Microsoft and Oracle databases. As is the case with many developers, I love my command line and shell. I have been a Mac user for a very long time and I don't know what I would do without my command shell. Before my Mac days, I was on windows and ended up using Cygwin all the time. When I started working on MongoDB, I was very happy to see MongoDB Shell. It is a very powerful tool. It can interpret JavaScript and it is very easy fo

My journey from SQL Server to MongoDb

I recently had a chance to introduce MongoDB in one of my projects. This application was a data collection system. Its main purpose was to be able to ingest large amounts of data from different data sources. These sources had different formats for data. Some were in XML, some were in CSV and some were in binary. They all had one thing in common though and that was the large size of data. The core of the application was written in a multi-threaded .NET application that was using SqlBulkCopy with a SQL Sever 2012 addition. SqlBulkCopy is one of the preferred (optimized) ways of loading data into SQL Server. The goal of the application was to be able to parse any given file with its given format definition/configuration file. Therefore, it needed to be able to store data into SQL Server that it never stored before. This provided challenges since Relational Data Management Systems require you to define your data structure before you can store data (i.e you must create your tables). Ther

Lock Escalation and Snapshot Isolation Level

I recently ran into an interesting situation with SQL Server specifically SQL Server 2012 that involves "Lock Escalation" that usually resulted in dead-lock scenarios. My application is responsible for parsing very large binary/text files and inserting data into SQL tables. My original thought was to handle this large dataset insertion in a single transaction so that data consistency would be achieved. The application has two major components. One of the components is a web application that accesses some of the tables that are used in the bulk insertion I mentioned above for basic verification and validation. When I was running some test cases, I noticed that my application became very un-responsive and it would eventually time out when the time-out values were reached. SQL Server provides some system tables that maintains information about the locks it keeps and any suspended threads due to these locks. You can use some of the following queries below to query your maste