Having spent 5 years taking courses in the Computing Science department at Simon Fraser University, I’ve become pretty familiar with the various web applications used to track course projects, submit assignments, etc.  Over the same five years, I’ve seen numerous technological changes, both in the “outside world” and within the University.  For example, not too long ago, SFU switched to a central authentication service aptly named “Central Authentication Service” (or CAS).  It’s a pretty nice service, with a snazzy login screen (below).

It’s not only snazzy, but a pretty useful system.  I even integrated one of my school projects (an online grade tracking application) with it without any external support.  Let’s contrast that with the current login screen for “Gradebook”, an application developed by the CS department and used in most CS courses to track grades

Ok, not as snazzy, but here’s the really frustrating thing:  My account here is NOT the same as my SFU computing account!

The “Gradebook” system also includes an electronic assignment submission server

(BTW: This is only a small snapshot of the page)

Sure, it isn’t snazzy looking, but it’s simple and to the point (+10 points).  However, it comes with a few caveats: I have to use my “Gradebook” account, not my (so-called) “universal” SFU account (-25 points), and ALL submissions must be in ZIP, GZIP or RAR format (-25 points)!  For many of my courses, the electronic submission has been a single file, and the instructor often has to email the students to remind them that they have to wrap their single PDF file (or Word Doc, etc.) in a ZIP file to upload it.  I can understand the benefits as far as storage space, and bandwidth consumption, but wouldn’t that be better achieved by simply placing a size restriction and allowing any file to be uploaded?  Plus, when I do login, I get a page with a drop-down list of courses.  Here’s the contents:

See that scrollbar?  Yeah… I’m not registered in anywhere near that many courses (the one course I’m registered in is a few scrolls away), but it would appear the submission server doesn’t know that.

It doesn’t end with the student interface.  In a course on “Web Information Systems,” we were required to develop a web-based application of some sort, and my group chose to build a grade tracking system (partly out of frustration with the current system).  Before starting the project, we organized a meeting with our instructor to find out what his frustrations were with the existing “Gradebook” system used by the department.  He mentioned that instructors had to rebuild the structure of assignments, exams and their relative contributions to the final grade every time they created a course.  He continued to show us a tool he developed which allowed him to create more detailed marks, give comments, and build an email to send off to the student.  He had to build a custom tool to perform tasks which I would consider essential to a modern grade tracking system.  In a few weeks, we hammered out a “school project quality” system running on Ruby on Rails (ASP.Net MVC wasn’t out yet :P) which supported all the features we discussed in our meeting with the prof.

Granted, I know it’s not as simple to fix as it is to complain.  But, with the tools out there, and the number of open-source and commercial courseware products, I still don’t really understand why we’re using a system with so many annoying issues…

For the CS students (or recent grads) who read my blog, is my experience atypical?  Or are your school services as out of date as these?

11:00AM – I look in my backpack to check that the Project Proposal that is due at my PHYS 192 class at 11:30 is still in there

11:01AM – I realize it isn’t there, I left it at home

11:02AM – I decide to just reprint it on campus, and relax

11:05AM – I fire up Word and open up the Print Dialog

11:06AM – I discover that the campus print server isn’t in my printers list

11:08AM – I locate the campus printing instructions and start setting up the printer

11:10AM – I discover that the printer driver I need isn’t included in Windows

11:12AM – I locate the printer driver online and start downloading it

11:15AM – Printer driver installed, I configure the campus printer

11:17AM – Still reasonably calm, I fire up Word again and send my document to the print queue.

11:20AM – After packing up my machine, I walk over to the printing station in the Laptop Lab

11:21AM – I discover a “Out of Order – Sorry for the inconvenience” sign :(

11:25AM – I walk to the other side of the building to another lab and put my printer card in the reader

11:26AM – Spending the last $0.11 on my card, I print my document

11:27AM – Calmly, I grab the printed document and see the following text:

Error: PCL Protocol Error

11:28AM – Scramble to a PC in the computer lab and log in to the machine

11:30AM – Machine finally logs in, and I fire up Internet Explorer, navigating to my Live Mesh Desktop

11:31AM – Open up the document from Live Mesh and print it

11:33AM – Go back to the printer station, and discover that my printer card is empty!

11:34AM – Walk to the nearby printer card top-up machine and see: “Out of Order – Sorry for the inconvenience”

11:35AM – After discovering I had no cash, walk to the nearby ATM and withdraw some

11:37AM – Head to the nearest cafeteria and buy a bottle of water, getting a twoonie ($2 coin, for my non-Canadian readers) in change :)

11:37AM – Head off to the library on the other side of campus (where the only other printer card top-up machine is… AFAIK)

11:40AM – Reach the library and stick my card in the machine, pop my twoonie in the slot and top-up my card

11:41AM – Go over to the library printer and release the printer job

11:42AM – Pick up the paper and dash over to the lecture hall with my printed proposal in hand

11:45AM – Arrive, late, and take the only remaining seat… at the front of the class, right next to the pile of proposals from the other students

11:46AM – Drop my proposal on the pile and sigh in relief

Seriously… need a way better campus printing system.  Only two top-up machines on campus?  Why can’t I just add funds to my student account online and submit PDFs and DOCs to be printed if I don’t want the hassle of configuring the print server?

Arg…

One of the courses I took last semester (Fall 2008), was “Software Engineering II”.  In this class, we were required to work in groups to implement a project that the professor specified.  We had to go through the whole process, design, implementation, testing (though we could choose any software process model we wanted: Waterfall, XP, Agile, etc.).  Our group’s project was an application called “Mappr” that would allow users to browse a map.  Well, it was a little more than that, but that’s all the background required by this post, I’ll post more background in future posts.

One of the necessary components in mapping software is called a “Projection”.  The Earth is round, and Latitude and Longitude co-ordinates are spherical measurements representing points on the Earth.  In order to convert those co-ordinates to (x, y) co-ordinates for displaying on a (flat) computer screen, you must project the geographical co-ordinates into screen co-ordinates.  One well-known technique for doing this is called the Mercator Projection.

A quick aside: The Mercator Projection is widely known (in geography circles) for being highly inaccurate.  However, it is the projection used by most road maps, atlases, etc., both physical and digital.

In Mappr, projection is handled by a component called a “Projection Strategy”.  A Projection Strategy is a C# class (Mappr was written in C#) with two methods: GeoToScreen and ScreenToGeo.   Here are the signatures of those methods:

public interface IProjectionStrategy {
    Point GeoToScreen(Point geographicalPoint, int zoomLevel, int tileSize);
    Point ScreenToGeo(Point screenPoint, int zoomLevel, int tileSize);
}

The purpose of each method is straight forward: To take in either Geographical (Latitude, Longitude) co-ordinates or Screen (X, Y) co-ordinates, and convert them to the other.  In order to do this, we must know the Zoom Level, which is an integer N indicating that there are 2^N tiles on the screen.  We also need the size, in pixels, of each map tile image.  This means that the size of the map, in pixels, is given by: 2^zoomLevel * tileSize.

The code to project a geographical point on to the screen is shown below:

public Point GeoToScreen(Point geographicalPoint, int zoomLevel, int tileSize) {
    // Convert to normalized mercator
    double lon = geographicalPoint.X;
    double lat = geographicalPoint.Y;

    if (lon > 180) {
        lon -= 360;
    }

    lon /= 360;
    lon += 0.5;

    lat = 0.5 - ((Math.Log(Math.Tan((Math.PI / 4) + 
                 ((0.5 * Math.PI * lat) / 180))) / Math.PI) / 2.0);

    double scale = (1 << zoomLevel) * tileSize;
    return new Point(lon * scale, lat * scale);
}

This code first normalizes the longitude (X direction) so that it is in the range 0.0 to 1.0 (where 0.0 is the left of the map and 1.0 is the right).  Then it does what I like to call “mathy stuff” (the calculations are taken from similar code written in Java) with the latitude to put it in the same range (0.0 is the top, 1.0 is the bottom).  Finally, we calculate the scale of the map (height/width in pixels, since the map is technically a square) and then we can use the normalized longitude and latitude as ratios of that scale.

The ScreenToGeo method is similar, the code is below.  I won’t describe this, but just provide it for reference.

public Point ScreenToGeo(Point screenPoint, int zoomLevel, int tileSize) {
    int pixelSpan = (1 << zoomLevel) * tileSize;
    double lngWidth = 360.0 / pixelSpan; // width in degrees longitude
    double lng = -180 + (screenPoint.X * lngWidth); // left edge in degrees longitude

    double latHeightMerc = 1.0 / pixelSpan; // height in "normalized" mercator 0,0 top left
    double latMerc = screenPoint.Y * latHeightMerc; // top edge in "normalized" mercator 0,0 top left
    
    // convert top and bottom lat in mercator to degrees
    // note that in fact the coordinates go from about -85 to +85 not -90 to 90!
    double lat = (180 / Math.PI) * ((2 * Math.Atan(Math.Exp(Math.PI * (1 - (2 * latMerc)))))
                       - (Math.PI / 2));

    return new Point(lng, lat);
}

By the way, feel free to use any of the code in this post in your own application. Consider it "Public Domain". However, I would appreciate (but not require) if you would place a comment near it indicating that this blog is the source of the original code.

Hopefully this helps those of you writing mapping applications in C#!  Please post any questions or comments in the comments section!

The Spring semester at SFU is off to a running start (after a day lost to the snow), and I’ve already got some homework to do!  One of the classes I’m taking is a course titled “Writing for Computer Scientists”. The goal is to develop writing skills through practice. Lots and lots of practice :).

Each class (3 times a week), we’re required to submit some “writing”.  I use quotes simply to emphasize the freedom of this assignment.  All I have to do is write a single word, and turn it in each week.  Of course, that’s not really the point of the assignment ;).  One option, provided by the professor, is simply to start a blog (if you don’t already have one) and post 2-3 entries per week.  Given that I already have a blog which I have had trouble maintaining, I figure this is a good kick in the behind to get myself blogging again. 

So, I’ll hopefully be blogging more often (at least while my grades depend on it :D).  I’ll probably stick to the current subject of the blog: observations from an open-source developer on the Microsoft ASP.Net platform.  Just in case I start to wander to different topics, I’ll make sure to tag appropriately, so my normal readers can subscribe just to the posts they’re interested in.

It’s time to start blogging for fun and profit grades!

This is part 2 of my VidSpeak series, where I show off an app I wrote for my multimedia course.  Check out part 1.

To recap from last time, the problem this assignment is trying to solve is to convert video frames to some kind of audio representation.  There are the steps required:

1. Extract the next frame from the video
2. Scale the frame down to 64x64 pixels
3. Make the frame a grayscale image
4. "Quantize" the grayscale frame into 4-bit colour
5. Convert the frame to sound

I covered extracting video frames in Part 1, so now we have to manipulate the image to prepare it for rendering as sound.

2. Scale the frame down to 64x64 pixels

All the transformation steps (2-4) are performed by "filters" which recieve a System.Drawing.Bitmap and return a transformed one (a filter could also modify the incoming Bitmap and then just return the same object).  Scaling is trivial with the "GetThumbnailImage" method on System.Drawing.Image:

public Bitmap ProcessImage(Bitmap input) {
    return new Bitmap(input.GetThumbnailImage(TargetWidth, TargetHeight, null, IntPtr.Zero));
}

3. Make the frame a grayscale image

There may be a function in .Net to do this, but as part of the assignment we were provided with a formula to convert RGB images to 255-shade grayscale images so I decided to do this.  Now, the Bitmap class provides GetPixel and SetPixel methods, but they aren't really very performant when you need to touch every pixel.  Fortunately, the Bitmap class provides a method called "LockPixels" which prevents the Garbage Collector from moving the Bitmap data around in memory.  With that, I made a base class for filters that process a Bitmap on a Pixel-by-Pixel basis:

BitmapData data = input.LockBits(new Rectangle(0, 0, input.Width, input.Height), 
                                 ImageLockMode.ReadWrite, 
                                 PixelFormat.Format24bppRgb);
unsafe {
    for (int col = 0; col < data.Width; col++) {
        for (int row = 0; row < data.Height; row++) {
            PixelData* pixel = (PixelData*)data.Scan0 + col + row * data.Width;
            ProcessPixel(pixel);
        }
    }
}
input.UnlockBits(data);

PixelData is a simple 3-byte struct:

[StructLayout(LayoutKind.Sequential)]
public struct PixelData {
    public byte Blue;
    public byte Green;
    public byte Red;
}

Then, my grayscale image filter just has to implement ProcessPixel

byte convertedValue = (byte) Math.Floor((0.299 * pixel->Red) + 
                                        (0.587 * pixel->Green) + 
                                        (0.114 * pixel->Blue));
pixel->Red = convertedValue;
pixel->Green = convertedValue;
pixel->Blue = convertedValue;

Kinda strange to see the "->" operator in C#, eh?

4. "Quantize" the image to 4-bit colour

Quantization is the process of "compressing a range of values to a single quantum value" (http://en.wikipedia.org/wiki/Quantization_(image_processing)).  In this case, we are quantizing an 8-bit Grayscale image to a 4-bit Grayscale image.  To do this, we take each pixel value, and assign it one of 16 partitions (4-bits can hold values 0 through 15).  The following code (in FixedPartitionQuantizationStrategy) does this (note _partitionSize is 16 in this case).

public byte Quantize(byte input) {
    return (byte)Math.Floor((double)input / _partitionSize);
}

Quantization is done by the QuantizeImageFilter, which uses a similar base class to the Grayscale conversion above.  Then, an IQuantizationStrategy is used to perform the Quantization itself:

protected override unsafe void ProcessPixel(PixelData* pixel) {
    pixel->Red = _strategy.Quantize(pixel->Red);
    pixel->Green = _strategy.Quantize(pixel->Green);
    pixel->Blue = _strategy.Quantize(pixel->Blue);
}

And thats it! We now have an image that is 64x64 pixels in 4-bit Grayscale.  Next we have to render it out to sound.  Stay tuned for Part 3 for that!

I know, its been too long since I blogged, but its pretty busy at school right now :). Anyway, I'm taking a course in Multimedia this semester, and as part of that course, I have to write a program to convert frames in a Video to short Audio clips. I thought it might be interesting to examine how that is done, in C#.  So, over the course of about 3-4 posts, I'll go over the code that I wrote.  I've attached the full project to this post, so you can take a look at it right now.  The GUI app should work, though I can't guarantee it. All I can give it is "Works on My Machine" seal of approval :) 

Here are the steps involved:

1. Extract the next frame from the video
2. Scale the frame down to 64x64 pixels
3. Make the frame a grayscale image
4. "Quantize" the grayscale frame into 4-bit colour
5. Convert the frame to sound

The Code

The Code is a Visual Studio 2008 solution, written for .Net 3.5.  It uses unsafe code for image processing and sound generation, so you can't run it without full trust (i.e. you can't run it off of a network share).

1. Extract the next frame from the video

I used a "Pipeline" (http://en.wikipedia.org/wiki/Pipeline_(software)) architecture, so this phase is handled by a component I call a "Frame Source" which is expected to return a new frame when asked (or return null to signal the end of the input). I used the DirectShow COM library "DexterLib" to do the extraction. DexterLib contains a class called MediaDet (for MediaDetector) which does most of the work. Here's the code for the function which retrieves a frame at a specified timecode (in seconds). 

FYI: "_detector" is an instance of DexterLib.MediaDetClass() ("_detector" is of type IMediaDet), "_streamLength" is the length of of the video stream in seconds, "_bufferHandle" is an IntPtr referring to an unmanaged buffer (allocated with Marshal.AllocHGlobal) to hold the bitmap, and "_bufferSize"/"_frameSize" are the size of the buffer and the size of each video frame (respectively)

// WARNING: This method will destroy the bitmap retrieved in a previous call to this method
public Bitmap GetFrameAtTime(double timeCode) {
    // Get the bitmap at this time
    Bitmap frame = null;
    unsafe {
        byte* bufferPointer = (byte*)_bufferHandle;
        _detector.GetBitmapBits(timeCode, 
                                ref _bufferSize, 
                                ref *bufferPointer, 
                                _frameSize.Width, 
                                _frameSize.Height);
        frame = new Bitmap(_frameSize.Width, // Width
                           _frameSize.Height, // Heigth
                           _frameSize.Width * 3, // Stride
                           PixelFormat.Format24bppRgb, // Pixel Format
                           new IntPtr(bufferPointer + 
                                      Marshal.SizeOf(typeof(BITMAPINFOHEADER)))); // Start of Buffer
    }

    return frame;
}

(Note: If you look at the actualy code, you will notice I snipped out some stuff from the beginning of this function to display it on the blog. The missing code just handles an (experimental) feature I added to allow me to start at any location in the video, rather than always starting at the beginning)

After loading the frame, I have to flip it, because Dexter loads the frame upside-down, fortunately the System.Drawing.Image class provides a RotateFlip method to do just that! I also rotate it 90 degrees clock-wise, so that each row of the transformed image maps to a column of the frame. This makes step 5 easier, since Bitmaps are stored in "row-major" order (http://en.wikipedia.org/wiki/Row-major_order).

To use the Frame Source, all my program has to do is call GetFrameAtTime method passing in a timecode (in seconds).  This is handled in the FrameProcessor by the GetNextFrame method

_source.GetFrameAtTime((DateTime.Now - _startTime).TotalSeconds)

Rather than going frame-by-frame, I'm extracting the next frame by time.  So, if it takes 4 seconds to process a frame, the next frame I take is approximately 4 seconds after the frame I just processed.

Here's the code: VidSpeak.zip (267.21 KB)