11:00AM – I look in my backpack to check that the Project Proposal that is due at my PHYS 192 class at 11:30 is still in there

11:01AM – I realize it isn’t there, I left it at home

11:02AM – I decide to just reprint it on campus, and relax

11:05AM – I fire up Word and open up the Print Dialog

11:06AM – I discover that the campus print server isn’t in my printers list

11:08AM – I locate the campus printing instructions and start setting up the printer

11:10AM – I discover that the printer driver I need isn’t included in Windows

11:12AM – I locate the printer driver online and start downloading it

11:15AM – Printer driver installed, I configure the campus printer

11:17AM – Still reasonably calm, I fire up Word again and send my document to the print queue.

11:20AM – After packing up my machine, I walk over to the printing station in the Laptop Lab

11:21AM – I discover a “Out of Order – Sorry for the inconvenience” sign :(

11:25AM – I walk to the other side of the building to another lab and put my printer card in the reader

11:26AM – Spending the last $0.11 on my card, I print my document

11:27AM – Calmly, I grab the printed document and see the following text:

Error: PCL Protocol Error

11:28AM – Scramble to a PC in the computer lab and log in to the machine

11:30AM – Machine finally logs in, and I fire up Internet Explorer, navigating to my Live Mesh Desktop

11:31AM – Open up the document from Live Mesh and print it

11:33AM – Go back to the printer station, and discover that my printer card is empty!

11:34AM – Walk to the nearby printer card top-up machine and see: “Out of Order – Sorry for the inconvenience”

11:35AM – After discovering I had no cash, walk to the nearby ATM and withdraw some

11:37AM – Head to the nearest cafeteria and buy a bottle of water, getting a twoonie ($2 coin, for my non-Canadian readers) in change :)

11:37AM – Head off to the library on the other side of campus (where the only other printer card top-up machine is… AFAIK)

11:40AM – Reach the library and stick my card in the machine, pop my twoonie in the slot and top-up my card

11:41AM – Go over to the library printer and release the printer job

11:42AM – Pick up the paper and dash over to the lecture hall with my printed proposal in hand

11:45AM – Arrive, late, and take the only remaining seat… at the front of the class, right next to the pile of proposals from the other students

11:46AM – Drop my proposal on the pile and sigh in relief

Seriously… need a way better campus printing system.  Only two top-up machines on campus?  Why can’t I just add funds to my student account online and submit PDFs and DOCs to be printed if I don’t want the hassle of configuring the print server?

Arg…

Before I start this post, I need to say one thing: Mark Russinovich (as well as anyone else who works on the Sysinternals stuff) is a Windows Magician.  The stuff these tools can do is just plan magic :).

Last week, my Dad’s Dell XPS laptop decided it was tired of rendering graphics and the video card just stopped working (apparently this is a known issue).  Dell support was fairly helpful and on Friday we received a box to sent the computer in for a motherboard replacement.  Now, I’m paranoid when it comes to data, I don’t trust anyone.  So, I suggested we try and wipe the hard drive of all the sensitive data on the machine, and change the password so that we wouldn’t have to give Dell the actual password.  Now, only the graphics card was damaged, the computer ran just fine and even the graphics were intermittent (at first).  We tried to get the graphics card working long enough to enable remote desktop, so we could connect from his screaming new Dell desktop (Quad Core i7 with 12GB of RAM… I’m jealous).  Unfortunately, by that time, the graphics card was done… no dice :(

After a little searching, I came across some tips for enabling remote desktop… remotely.  It turns out, there are a number of ways to connect to a machine and make changes remotely!  The first tip I found was to use the remote registry to enable it: http://www.windowsdevcenter.com/pub/a/windows/2004/05/04/serverhacks_remote.html.  Unfortunately, the Remote Registry service wasn’t running on the laptop!  That’s when I found Sysinternals “PsService” tool.  The tool allows you, as long as you have the Admin account password, to enable/disable and change the start mode of services on a remote machine.  So, first I enabled to remote registry service

psservice \\bilbo –u [UserName] start RemoteRegistry

Then, just for good measure, I set the Remote Desktop service to autostart

psservice \\bilbo –u [UserName] setconfig TermService auto

Finally, rather than fight with creating a firewall rule remotely, I just disabled the Windows Firewall service (don’t do this unless you absolutely have to :D)

psservice \\bilbo –u [UserName] setconfig MpsSvc disabled

(Btw, I got all these service names using the Services console on another machine)

Finally, I followed the instructions I mentioned earlier to set the right registry flags to allow Remote Desktop.  I killed the power to the PC (I actually tried Sysinternals “PsShutdown” tool first, but had less luck getting that working :D) and restarted it.  When I heard the Windows start up sound (as I said, the rest of the components worked just fine :), I fired up remote desktop connection and… bam!  I had a remote desktop screen.  After a quick pass over the data drive with yet another Sysinternals tool: “SDelete” and a password change, we shut the computer down, packed it up and sent it off to Dell, confident that the important data was scrubbed clean.

Like I said… black magic :).  But seriously, the Sysinternals tools rock!  If you don’t have them, get them now!

Here’s part 4 of my infinite part series on developing a compiler from scratch.  See all the posts here.

I’ve decided to change things up a bit on the compiler project.  Instead of writing the tokenizer and parser from scratch, I’m going to check out a new language that’s part of Microsoft’s “Oslo” Project.  Oslo includes a language called M, which is primarily designed for developing data models.  M is actually three languages, MGraph, MSchema and MGrammar.  MGraph is a language for defining abstract graphs, for example:

Universities {
    Canada {
        BritishColumbia {
            SFU { 
                Name { "Simon Fraser University" }, 
                Location { "Burnaby, BC" } 
            },
            UBC { 
                Name { "University of British Columbia" }, 
                Location { "Vancouver, BC" } 
            }
        }
    }
}

MSchema is a language for defining the schema for these graphs, for example

module Universities {
    type University {
        Name : Text#100,
        Location : Text#100
    }
}

Finally, MGrammar is a language for defining custom Domain-Specific Languages (DSLs) which map to MGraph graphs. This is the language that is most useful to this project. After all, a general-purpose programming language is just a DSL where the "Domain" is very broad :). So, I've spent a few hours this week whipping up an MGrammar grammar for Duh, and the results are promising.  I’ll be posting some details on that in the next post, but first we need to explore a key concept in compilers, Abstract Syntax Trees.

The end goal of the tokenization and parsing phases of a compiler is to produce an Abstract Syntax Tree (or AST) representing the source code.  This is probably best illustrated by an example, so consider this expression:

5+6

This is an addition operation, using 5 and 6 as the arguments, which we represent in an AST like this (using the MGraph syntax above):

Add { 5, 6 }

Similarly, more complex expressions can be translated to trees:

5+6*(3+4)

Becomes:

Add { 5, Multiply { 6, Add { 3, 4 } } }

Which represents the following instructions:

• Add 3 and 4
• Multiply that result by 6
• Add 5 to that result

Essentially, we are building up the expression from the bottom of the tree up.  This makes it much easier to convert the expression into code.

I’ll post the MGrammar file for the language when I finish it up in a day or so, then its on to writing the code to do the parsing.

UPDATE 2: PowerKits requires PowerShell 2.0.  If you’re seeing errors regarding the “try-catch” keywords, its because those keywords were added in PowerShell 2.0 and I use them in PowerKits.  Sorry!

UPDATE: It turns out PowerShell 2.0 already contains a feature like this: “Modules.”  See here for more info.  Since it seems to be far superior to PowerKit (which I just hacked together for my own needs), I guess that just leaves this as an interesting exercise :).

I’ve been a huge fan of Windows PowerShell since it first came out as “Monad” (aka “MSH”).  Now that its entering version 2.0, which will be built-in to Windows 7 and Windows Server 2008, I’m getting even more into it.  I’ve written a Podcast Receiver (PowerCast) and an automated process for converting videos for use on my Xbox 360 (PowerCode).

[The following paragraph is a short intro to PowerShell, feel free to skip it if you’re familiar with PoSH]

What is PowerShell?  PowerShell is an object-oriented shell which, IMHO, kicks the pants off of BASH and Perl.  PowerShell is a dynamically-typed scripting language written on the .Net Framework, which gives you full access to every class, method, and property in the framework.  You could write a GUI, Server, heck even a 3D Game in PowerShell (it just might not perform as well as another language).  PowerShell is also an interactive “Read-Eval-Print” loop application which gives you a prompt into which you type commands which are then evaluated and their results printed to the screen (i.e. a standard “Shell” similar to cmd.exe or BASH).  Finally, PowerShell is an embeddable scripting environment which lets any .Net-based application embed the PowerShell scripting language.  However, most people (myself included) are referring to the Shell itself when they talk about PowerShell

Anyway, on to the main purpose of this post.  I started to get frustrated with trying to manage all my scripts and keep them organized.  I was also downloading a lot of useful scripts off of the web, but had to manually place them somewhere in my file-system (let alone managing all the dependencies).  Other scripting environments, such as Ruby, have packaging systems, such as RubyGems, to manage libraries of functionality and publish them for others to download.  So, I thought, why doesn’t PowerShell have something like this?

Thus, PowerKits were born.  The concept is simple.  A PowerKit is a directory on your file-system which contains a manifest file “Kit.xml”.  This file specifies some information such as the paths to various script files, aliases to create, .Net assemblies to load and (in a future version) PowerShell “snap-ins” to load.  After installing a kit (by placing it in one of many, configurable, pre-defined folders), it can simply be enabled using a single PowerShell Command.  For example, to enable the “MyCompany.MyKit” PowerKit:

Enable-Kit MyCompany.MyKit

Enabling a kit loads all the assemblies, script files, etc. into your PowerShell environment, making them available for use.  A Kit can then be disabled similarly using the ”Disable-Kit” command.  Disabling the kit removes all the script files, etc. from the environment (NOTE: .Net assemblies cannot be unloaded without restarting PowerShell, due to a limitation of the .Net Framework).

PowerKit also supports packaging kits into a ZIP-like format, specifically the Open Packaging Convention developed by Microsoft for Office 2007 (since the .Net framework has built-in support for that format). These “Kit Archives” can be downloaded and installed using a single command.  Eventually, I plan to add support for installing a kit directly from the web, so you could use a command like this:

Install-Kit http://www.mycompany.com/MyCompany.MyKit.kit

Or even this:

Install-Kit MyCompany.MyKit

And the framework would go out and download the kit and install it, giving PowerShell a true “gems-esque” packaging system.

To install PowerKit, download the Install-PowerKit.ps1 script from the CodePlex site, more installation details are in the release notes on that page.

I recently used the ‘yield’ keyword in a C# method and ran into a strange issue when trying to verify arguments.  I always try to verify the arguments provided to my methods so that I can throw ArgumentException or ArgumentNullException as necessary.  However, when you use the ‘yield’ keyword to create an enumerator in C#, you can run into issues

The ‘yield’ keyword lets you easily create a method which returns an enumerator (which can then be used in a foreach loop).  For example, if I wanted to make a method that took a DirectoryInfo (representing a directory on disk) and returned file names of all the .txt files in it, I would write this:

public IEnumerable<string> GetTextFiles(DirectoryInfo dir) {
    if (dir == null) {
        throw new ArgumentNullException("dir");
    }
    foreach (FileInfo file in dir.GetFiles("*.txt")) {
        yield return file.Name;
    }
}

However, the C# compiler will do something tricky with this method, because of the 'yield' keyword.  It will create a custom implementation of IEnumerable<string> class that uses "magic" (I'm not going to go into detail as to how it works :P) to jump in and out of the code you wrote. Then, it compiles our method to the following (as decompiled with RedGate's .Net Reflector, a free tool every .Net developer must have :D):

public IEnumerable<string> GetTextFiles(DirectoryInfo dir)
{
    <GetTextFiles>d__0 d__ = new <GetTextFiles<d__0(-2);
    d__.<>4__this = this;
    d__.<>3__dir = dir;
    return d__;
}

Wait a sec, where'd our code go?  It looks like it’s gone into this <GetTextFiles>d__0 class.  If you think that name sounds compiler-generated, you’re right!  It’s the custom implementation of IEnumerable<string> that was created by the compiler.  Cool, eh?  However, there’s a side effect here.  Where did our argument checking code go?  Unless there’s something in the <GetTextFiles>d__0 constructor, the user could pass null in for the dir parameter and no exception would be thrown!  So, let’s take a look at the <GetTextFiles>d__0 constructor (using Reflector again):

[DebuggerHidden]
public <GetTextFiles>d__0(int <>1__state)
{
    this.<>1__state = <>1__state;
    this.<>l__initialThreadId = Thread.CurrentThread.ManagedThreadId;
}

Nope, nothing there.  If you keep checking around the <GetTextFiles>d__0 class, you’ll find it in the MoveNext method on that class.  That means that instead of checking the argument when you call the method, the argument won’t be checked until the first time MoveNext is called (i.e. the first iteration of a foreach loop).  If, instead, we wrap the actual “yield” code in it’s own method, and call it from our original method (after argument checks), we’ll get the desired result:

public IEnumerable<string> GetTextFiles(DirectoryInfo dir) {
    if (dir == null) {
        throw new ArgumentNullException("dir");
    }
    return InternalGetTextFiles(dir);
}
private IEnumerable<string> InternalGetTextFiles(DirectoryInfor dir) {
    foreach (FileInfo file in dir.GetFiles("*.txt")) {
        yield return file.Name;
    }
}

Now, the "magic" is occurring in InternalGetTextFiles and GetTextFiles is compiled without modification and our argument checking works just fine.