Tuesday, July 31, 2012

Wait Cursors

Ah, cursor management, the bane of desktop programmers everywhere. Truth be told, .NET makes it quite easy to swap between cursor styles, the most common is to alert the user that the application is waiting on a long running task. Usually you see code like this:

private void cmdSave_Click(object sender, EventArgs e)
    {
        Cursor.Current = Cursors.WaitCursor;
            
        // Do Stuff Here...

        Cursor.Current = Cursors.Default;
    }

Very straight forward, but wait! What if we want to have multiple exit points in our logic? Then  we end up with something like this:

private void cmdSave_Click(object sender, EventArgs e)
    {
        Cursor.Current = Cursors.WaitCursor;
            
        // Do Stuff Here...

        if (done == true)
        {
            Cursor.Current = Cursors.Default;
            return;
        }
            
        // Do Stuff Here...

        if (done == true)
        {
            Cursor.Current = Cursors.Default;
            return;
        }

        Cursor.Current = Cursors.Default;
    }

This is obviously prone to error, if for example we forget to add an assignment to set the cursor back prior to exiting, or we hit an error and forget to put the cursor back to default in a try/catch block we can end up with the never ending wait cursor.

But Rob, you ask, what can be done? Never fear I am here with your very own WaitCursor class that will automagically fix these issues for you. Here is the code:

public class WaitCursor : IDisposable
    {
        private readonly Cursor _originalCursor = Cursors.Default;

        public WaitCursor()
        {
            _originalCursor = Cursor.Current;
            Cursor.Current = Cursors.WaitCursor;
        }

        public WaitCursor(Cursor cursor)
        {
            _originalCursor = Cursor.Current;
            Cursor.Current = cursor;
        }

        #region IDisposable Members

        private bool _disposed;

        protected virtual void Dispose(bool disposing)
        {
            if (_disposed)
                return;
            
            if (disposing)
                Cursor.Current = _originalCursor;

            _disposed = true;
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        #endregion
    }


As you can see when instantiated, we grab the current cursor state, then set the current cursor to either the wait cursor, or whatever cursor we have passed in. When the class gets disposed, we set the cursor back to the original cursor. This allows us to write code like this:

private void cmdSave_Click(object sender, EventArgs e)
    {
        using (new WaitCursor())
        {
            // Do Stuff Here...

            if (done == true)
                return;
            
            // Do Stuff Here...

            if (done == true)
                return;
        }
    }

You are welcome

Thursday, July 19, 2012

Internet Defense League

Code Refugee is proud to be a member of The Internet Defense League. Ever since the US Government tried to foist the tragically draconian SOPA/PIPA censorship laws on the world, a number of organizations (Reddit, Fark, Imgur, TechDirt) have come together to found an organization to raise awareness of threats to internet freedom. There can be no more critical fight for the future of our world than the one to keep our most important communication mechanism free from governmental interference. Today the league has officially launched and we are showing the "cat signal" in recognition.

Please take a few minutes to familiarize yourself with the league and its goals. I think you will find them worthy of your attention.

Wednesday, June 27, 2012

Submitting Fake Tracks

Now that we have generated track data and saved it to our cloud storage. We can stub out the form that will post the fake GPS data to our (not yet implemented) API.

The Goal

We want to the user to be able to select a track, set a delay interval and indicate whether or not to randomly vary that interval (for realism). Then start the process of sending the data to our API. We also want to give the user some visual feedback of the data as it is being processed.

Track Faker

Here is a mock up of the track faker tab:


As you can see, we are capturing the device, track, delay and randomization flag at the top of the screen. There is also a Start/Cancel button at the top right. The bottom portion is a rolling log of events once the track has been started. We will make a small variation to this screen as we will make the log add events at the top of the log instead of the bottom.

Multi-Threading and Tasks

This feature is a great example of a long running task that will need to be broken out into its own thread in order to keep the application from becoming unresponsive. We will use the new Task Parallel Library to kick off a background thread that will read each GPS point, build the proper API post and update the UI with its progress. The task is simple to set up, we instantiate a cancellation token source and use it when we create the task like so:

_cancellationTokenSource = new CancellationTokenSource();
            CancellationToken cancellationToken = _cancellationTokenSource.Token; 
            
            Task.Factory.StartNew(() =>
            {
            }, cancellationToken);

This allows the task to be cancelled with a simple call from anywhere outside the task:
_cancellationTokenSource.Cancel();

Retrieving Files From Storage

So far we have gone through saving a file to blob storage, now lets have a look at retrieving a file. We can have the file retrieved as a memory stream or as text. Since we are going to deserialize it into an class we will use the memory stream.

public static MemoryStream RetrieveBlobStream(string containerName, string fileName)
        {
            try
            {
                var stream = new MemoryStream();
                var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");

                CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
                CloudBlobContainer container = blobClient.GetContainerReference(containerName);

                CloudBlob blob = container.GetBlobReference(fileName);
                blob.DownloadToStream(stream);
                return stream;
            }
            catch (Exception)
            {
                //TODO: Log error here...
                return null;
            }
        }

GPX POCO Generation

Wait? What? Deserialize it into a class? What class you say? That's easy the gpx.cs class we created using the xds.exe tool. Simply give it the proper parameters and the uri to your xsd file and it will generate a class for you.
//------------------------------------------------------------------------------
// 
//     This code was generated by a tool.
//     Runtime Version:4.0.30319.17379
//
//     Changes to this file may cause incorrect behavior and will be lost if
//     the code is regenerated.
// 
//------------------------------------------------------------------------------

// 
// This source code was auto-generated by xsd, Version=4.0.30319.17379.
// 
namespace iGOR.App.GPX
{
    /// 
    [System.CodeDom.Compiler.GeneratedCodeAttribute("xsd", "4.0.30319.17379")]
    [System.SerializableAttribute()]
    [System.Diagnostics.DebuggerStepThroughAttribute()]
    [System.ComponentModel.DesignerCategoryAttribute("code")]
    [System.Xml.Serialization.XmlTypeAttribute(Namespace="http://www.topografix.com/GPX/1/1")]
    [System.Xml.Serialization.XmlRootAttribute("gpx", Namespace="http://www.topografix.com/GPX/1/1", IsNullable=false)]
    public partial class gpxType {
        
        private metadataType metadataField;
        
        private wptType[] wptField;
        
        private rteType[] rteField;
        
        private trkType[] trkField;
        
        private extensionsType extensionsField;
        
        private string versionField;
        
        private string creatorField;
        
        public gpxType() {
            this.versionField = "1.1";
        }
        
        /// 
        public metadataType metadata {
            get {
                return this.metadataField;
            }
            set {
                this.metadataField = value;
            }
        }
        
        /// 
        [System.Xml.Serialization.XmlElementAttribute("wpt")]
        public wptType[] wpt {
            get {
                return this.wptField;
            }
            set {
                this.wptField = value;
            }
        }
        
        /// 
        [System.Xml.Serialization.XmlElementAttribute("rte")]
        public rteType[] rte {
            get {
                return this.rteField;
            }
            set {
                this.rteField = value;
            }
        }

        //Snip a ton of code...
    }
}

Putting It All Together

Now we have all of the pieces, lets look at the end result. Remember this is a stub and not the complete solution for this tab yet. We still need to get our list of devices from somewhere and do the actual calls to our API.
private void cmdStart_Click(object sender, EventArgs e)
        {
            if (cboTracks.Text.Length == 0)
            {
                MessageBox.Show(
                    "Please select a demo track",
                    "Demo Track",
                    MessageBoxButtons.OK,
                    MessageBoxIcon.Error);

                return;
            }

            _cancellationTokenSource = new CancellationTokenSource();
            CancellationToken cancellationToken = _cancellationTokenSource.Token; 
            
            Task.Factory.StartNew(() =>
            {
                decimal lowerTimer;
                decimal upperTimer;

                cmdStart.Invoke((Action)delegate { cmdStart.Visible = false; });
                cmdCancel.Invoke((Action)delegate { cmdCancel.Visible = true; });

                txtLog.Invoke((Action)delegate { txtLog.Text = "Loading track " + cboTracks.Text + "..." + Environment.NewLine; });

                var timeString = "Interval between location submits will be ";

                if (chkRandomize.Checked)
                {
                    decimal randomizerPercentage = Properties.Settings.Default.IntervalRandomizerPercentage;
                    decimal variance = spnInterval.Value * (randomizerPercentage / 100);
                    lowerTimer = spnInterval.Value - variance;
                    upperTimer = spnInterval.Value + variance;

                    timeString += "between " + lowerTimer + " and " + upperTimer + " seconds." + Environment.NewLine;
                }
                else
                {
                    lowerTimer = upperTimer = spnInterval.Value;
                    timeString += "exactly " + lowerTimer + " seconds." + Environment.NewLine;
                }

                txtLog.Invoke((Action)delegate { txtLog.Text = timeString + txtLog.Text; });

                if (cancellationToken.IsCancellationRequested)
                {
                    // another thread decided to cancel
                    txtLog.Invoke((Action)delegate { txtLog.Text = "Simulation Cancelled..." + Environment.NewLine + txtLog.Text; });
                    return;
                } 

                var containerName = ConfigurationManager.AppSettings["DemoTrackContainer"];
                var stream = Blob.RetrieveBlobStream(containerName, cboTracks.Text);
                stream.Position = 0;

                var mySerializer = new XmlSerializer(typeof(GPX.gpxType));
                var track = (GPX.gpxType)mySerializer.Deserialize(stream);

                var rdm = new Random();
                int min = Convert.ToInt32(lowerTimer * 1000);
                int max = Convert.ToInt32(upperTimer * 1000);

                foreach (var point in track.rte[0].rtept)
                {
                    if (cancellationToken.IsCancellationRequested)
                    {
                        // another thread decided to cancel
                        txtLog.Invoke((Action)delegate { txtLog.Text = "Simulation Cancelled..." + Environment.NewLine + txtLog.Text; });
                        return;
                    }

                    int waitTime = rdm.Next(min, max);
                    txtLog.Invoke((Action)delegate { txtLog.Text = "Waiting " + waitTime + " milliseconds..." + Environment.NewLine + txtLog.Text; });
                    Thread.Sleep(waitTime);
                    txtLog.Invoke((Action)delegate { txtLog.Text = "Sending Lat: " + point.lat + ", Lon: " + point.lon + Environment.NewLine + txtLog.Text; });
                    // TODO: Actually send the data to the API...
                }

                txtLog.Invoke((Action)delegate { txtLog.Text = "Simulation complete..." + Environment.NewLine + txtLog.Text; });

                cmdStart.Visible = true;
                cmdCancel.Visible = false;
            }, cancellationToken);
        }



Monday, June 4, 2012

Saving GPX Files to Azure Blob Storage

Recap

In the last post we gathered the track data from Google Maps, feed it into GPS Babel and have our GPX formatted file saved to a local directory. Now we want to enable our users to put a name to the file and store it in our blob storage account for later use.

GPX Viewer

We need a way for the user to validate the output from the conversion process and specify a name for the track. We can fulfill both requirements with a single form:


A text box to display the file GPX data, one for the file name and a save button are all we need. Create a public property to hold the GPX data as a string. In the form's shown event, assign it to the large text box. One more detail, GPS Babel output the GPX in the GPX/1/0 format and we want the GPX/1/1. A simple fix is to replace the xmlns property like so:

private void Document_Shown(object sender, EventArgs e)
        {
            Doc = Doc.Replace("http://www.topografix.com/GPX/1/0", "http://www.topografix.com/GPX/1/1");

            editDocument.Text = Doc;
        }

Save That Track!

Once the user enters a file name and presses the Save button we have two tasks; first validate the file name and then save the file to our Azure storage account.

Blob file names follow the same rules as windows file names, so we can use the built in .net function (GetInvalidFileNameChars) for this:

private bool FileNameIsValid()
        {
            if (string.IsNullOrEmpty(txtFileName.Text))
            {
                MessageBox.Show(
                    "Please enter a valid file name for this track.",
                    "No File Name",
                    MessageBoxButtons.OK,
                    MessageBoxIcon.Error);

                return false;
            }

            if (txtFileName.Text.IndexOfAny(System.IO.Path.GetInvalidFileNameChars()) != -1)
            {
                MessageBox.Show(
                    "Please enter a valid file name for this track.",
                    "Invalid File Name",
                    MessageBoxButtons.OK,
                    MessageBoxIcon.Error);

                return false;
            } 

            return true;
        }

An Aside About Blobs

There are two kinds of blobs: block and page.

Block blobs allow a single blob to be broken up into smaller blocks. These blocks allow parallel upload/download thus allowing for better performance. They are limited to 200GB in size. Each block can be up to 4MB in size (allowing for 50,000 blocks). Each block must be uploaded and then the entire blob is committed into storage. That means uploading block blobs is a two-step process. You can upload the blob in a single operation when the block blob is less than 64MB.

A page blob is collection of pages. Individual pages can be up to 1 TB, but each page must be a multiple of 512 bytes. A page is a range of data that is identified by its offset from the start of the blob. Pages can be randomly uploaded and accessed. Unlike block blobs, writes to a page blob are committed immediately.

Which should you use and when?

Well that depends on your file size and your usage scenario. If you have no need to pull individual pages and your files are < 200GB then use block blobs. Otherwise you will need to use page blobs. For our purposes block will do just fine.

Now Back To Our Regularly Scheduled Post

Wire up the Save button like so:

private void cmdSave_Click(object sender, EventArgs e)
        {
            if (FileNameIsValid())
            {
                var containerName = ConfigurationManager.AppSettings["DemoTrackContainer"];
                var fileName = txtFileName.Text + ".gpx";

                if (Blob.CreateBlockBlob(containerName, fileName, editDocument.Text))
                {
                    MessageBox.Show(
                        fileName + " was sucessfully saved.",
                        "Demo Track Saved",
                        MessageBoxButtons.OK,
                        MessageBoxIcon.Information);

                    Close();
                }
                else
                {
                    MessageBox.Show(
                        "There was an error saving " + fileName + ".",
                        "Error Saving Demo Track",
                        MessageBoxButtons.OK,
                        MessageBoxIcon.Error);
                }
            }
        }

Here you can see that we test our file name, grab our container name from app settings and add the ".gpx" extension to our file name. We then call a static class called Blob and it's CreateBlockBlob method. The method creates a file in the specified container with the passed string as it content. Here is the code:

public static bool CreateBlockBlob(string containerName, string fileName, string text)
        {
            try
            {
                var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
                CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

                CloudBlobContainer container = blobClient.GetContainerReference(containerName);
                container.CreateIfNotExist();

                CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
                blob.UploadText(text);

                return true;
            }
            catch (Exception)
            {
                return false;
            }
        }


And here is the code for creating a page blob (in case you were curious)

public static bool CreatePageBlob(string containerName, string fileName, string text, long size)
        {
            try
            {
                var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
                CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

                CloudBlobContainer container = blobClient.GetContainerReference(containerName);
                container.CreateIfNotExist();

                CloudPageBlob blob = container.GetPageBlobReference(fileName);
                blob.Create(size);
                blob.UploadText(text);

                return true;
            }
            catch (Exception)
            {
                return false;
            }
        }

Now we are saving our tracks in the cloud (yea for marketing slang!) and have then available for future use. Speaking of the future, the next post we will get started on the track faker piece of the application. We are going to take advantage of Tasks to handle multi-threading so we will continue to have a responsive UI while faking a long running track.

Wednesday, May 30, 2012

Generating Fake GPS Data

Why Fake Data?

One of the more interesting bits to the project is being able to generate fake GPS tracks to feed to the API. We need to be able to demonstrate the real-time aspects of the solution without having to have someone drive around and collect the data. Also it would be nice be able to generate tracks based on who we are demoing to. Perhaps a simulated drive from the iVision office to a clients office.

So, the goal is to be able to generate fake GPS tracks that we can feed to the iGOR application while demoing the real-time updating of the maps. We also need to be able to generate these tracks from any two points, and they should follow the streets and roads available in a logical way.

Google driving directions seems like a good place to start, and I found a great tool to help. GPS Babel is a free application that can take the data generated from Google Maps (and many other sources) and generate a standard GPX file.

So we want a page in our application that looks something like this:


We will have the user enter their start and destination addresses into Google Maps hosted in a browser control. They will then copy the link generated from the directions into the text box and press the Go button.



This will save the resulting HTML to a file that can be picked up by GPS Babel and converted into GPX. A lot more straight forward than it sounds, I promise.

First thing to do is download/install GPS Babel (follow the link above). Make a note of the location you install it as you will need it later. Now in our web browser control we can set our url to http://map.google.com.

Now lets wire up our Go button event like this:
private void cmdGo_Click(object sender, EventArgs e)
        {
            try
            {
                _isBuilding = true;
                var newURI = new Uri(txtTrackURI.Text + "&output=js");
                webBrowser.ScriptErrorsSuppressed = true;
                webBrowser.Url = newURI;
            }
            catch (Exception)
            {
                MessageBox.Show("Link does not appear to be valid. Please chech the track uri and try again.",
                                "Invalid Track URI", MessageBoxButtons.OK, MessageBoxIcon.Asterisk,
                                MessageBoxDefaultButton.Button1);
                _isBuilding = false;
            }
        }

As you can see we set a form level bool to true (I will come back to this later), take the text from the textbox and append a query parameter. This parameter tells Google to output the resulting page in a special format that can be imported by GPS Babel. We then update the web browser controls uri to this new location. Lets take a look at the Document Complete event of our control:

        private void webBrowser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
        {
            if (_isBuilding)
            {
                var gpsBabelLocation = Properties.Settings.Default.GPSBabelLocation + @"\gpsbabel.exe";
                var tempFolder = Properties.Settings.Default.LocalTempFolder;

                var localMapFilePath = tempFolder + @"\map.txt";
                var localGPXFilePath = tempFolder + @"\gpx_data.gpx";

                var fs = new FileStream(localMapFilePath, FileMode.Create, FileAccess.Write, FileShare.None);
                var file = new StreamWriter(fs, Encoding.ASCII);
                file.WriteLine(webBrowser.DocumentText);
                file.Close();

                var p = new Process
                {
                    StartInfo =
                    {
                        FileName = gpsBabelLocation,
                        Arguments = @"-t -i google -f " + localMapFilePath + @" -o gpx,suppresswhite=0,logpoint=0,humminbirdextensions=0,garminextensions=0 -F " + localGPXFilePath,
                        UseShellExecute = false
                    }
                };

                p.Start();
                p.WaitForExit();

                TextReader tr = new StreamReader(localGPXFilePath);
                var viewer = new GPXViewer { Doc = tr.ReadToEnd() };
                tr.Close();
                viewer.ShowDialog();

                webBrowser.GoBack();
                txtTrackURI.Text = null;
            }

            _isBuilding = false;
        }

Ah, there is that _isBuilding again. As you can see we are using it to control whether or not we process the web controls contents on document complete. As the user is generating their track, this event gets fired several times and we only want to process it when the user has hit the Go button.

The first thing we do is load up the location of the GPS Babel exe and the location of a temp folder (these are stored via the Settings tab which I will cover in a different post). We also set up the location of the map.txt file which is the input into GPS Babel, and gpx_data.gpx which is the output from the conversion process.

Next we save the contents of the web browser control to our input file (map.txt). Now we create a new Process object to kick off the GPS Babel exe. We give it the exe location and a set of command line arguments to control how GPS Babel executes and where to store the output. Once the conversion process is complete we read in the results from the gpx_data.gpx file into a string and pass it to another form to display the output.

My next post will cover the gpx viewer form and saving our file to Azure cloud storage for later retrieval.

Thursday, May 24, 2012

iGOR Design/Architecture Philosophies


The iVision Way


Actually, we don’t have a codified ‘way’, as far a project architecture goes, but we have guiding principles that allow us to deliver quality work in volume. I am going to steal an analogy from a co-worker (thanks Brian!).

Software projects are like trying to build a tower out of blocks. The higher the tower, the more features and business value is delivered. Some will start putting bricks one on top of the other and the tower gets high very quickly. This method involves little to no strategic planning or processes to support the development cycle. Thus the tower is unstable, and seemingly small changes can topple whole thing. These kinds of projects are rarely delivered on time or on budget. Others will build a wide and stable base and only start building upwards when they are sure there are no issues with the blocks already placed. These teams create reams of documentation and analysis of the business space and all interconnected systems. Analysis paralysis, it eats at deadlines and produces little of actual value to the business. These kinds of projects never deliver. Other projects are built with a solid base, just wide enough to support the pieces above. The tower goes up at a reasonable pace, with a stable support that allows for flexibility in case of change or the unexpected. This final method is far more likely to deliver on time and on budget and is what allows iVision to succeed where others fail.

I will assume you know how to download the Azure SDK and set up a new cloud project with the correct setting. I won't bore you with those details. I also won't subject you to testing plans and UI control minutia. Let’s get right to the fun.

iGOR Specifics

We have identified four separate projects to make up the solution; a web site, an API, an Azure worker and a desktop application. As we progress we will add more as the design dictates. Our initial solution looks like this:


iGOR is the Azure project and iGOR.Portal is our web site. The rest should be self exploratory. We are going to start with the desktop app (iGOR.App). This is because we need to be able to create and submit fake data tracks to iGOR for testing and demonstration (remember this is not a real production app). Also I find it to be an interesting problem to solve.

Here is a mock up of the main layout of the app we are going for:

As you can see we are looking at eight tabs, each with a distinct role in tracking remote employee locations, solution maintenance and settings. Next post will be a deep dive (including code) into generating fake device tracks using Google Maps.

Thursday, May 17, 2012

Introducing iGOR

The Why

My current employer, iVision does some fantastic work in the line of business application development field, and I am proud to be a part of it. We are currently preparing to do an internal project that will be used for technical demos, internal training and as a new technology proving platform. We have tentatively named it iGOR (iVision GPS Online Repository) and I will be writing about its design and development as we progress through the process.

The What

The concept is pretty straight forward, we have a fictional new client that has technicians in the field and would like to have the GPS location of each employee in real-time so they can optimize routes and update schedules. They have installed GPS receivers and cellular modems into each technicians laptop. They want the devices to transmit its current location every few seconds, and if it loses connectivity, to locally store the location data until the connection is re-established and then bulk send the data. They would also like to be able view each laptops track history for a given date range as well as see it in real-time.

The How

iGOR will be hosted on Microsoft’s Azure platform and be composed of four distinct applications; a desktop application, a worker role and two web roles. The site web role will handle the user interaction with a web interface, in this case a web site written in ASP.NET MVC 4. The API web role will handle the device interaction through an exposed API. The lone worker role will handle all long running or asynchronous tasks; such as file import/export, email communications and any regularly scheduled maintenance tasks. The desktop application will allow for users to interact with the underlying data stores. Users will be able to add/edit/delete devices, view GPS data, upload data files for import and other administrative tasks

Long term storage will be handled using Azure SQL for user and device information, Blob storage for files to be imported/exported and Table storage for the actual GPS data. Queues will be used for communication between roles and applications.

High Level Concept

The next post will be about setting up the development environment and some project architecture decisions.



Wednesday, May 16, 2012

Obligatory First Post Nonsense

Ok, so this is not actually my first post ever, just the first on this new engine and first blog with a more technical bent. I will be writing about any interesting issues that I run across in my various coding adventures.

One of the things I hope to write about is a technology demo application I am putting together for my day job. A mix of MVC 4, Windows application and Windows Azure. Will be a start-to-finish look at everything involved and should be a lot of fun.

Also will pontificate on the trials and tribulations of my start up company that is developing tools for small businesses.

Questions, comments and feedback all welcome. Now, be careful out there.