Recently I managed to add an extra certificate store to Windows by mistake, as I accidentally left out a command line argument when using makecert. Unfortunately, the Certificates MMC snap-in doesn’t seem to provide a way for you to delete a certificate store, so I had resort to a more technical approach in order to get rid of this new unwanted certificate store.

Digging around in the Windows API, I found the CertUnregisterSystemStore function that allows you to delete a certificate store programmatically. So I spun up my copy of LINQPad so that I could quickly script in C# and PInvoke that function. (Incidentally, if you don’t have a copy of LINQPad and you’re a .NET developer, you need to get yourself a copy immediately. It’s invaluable.) Unfortunately, CertUnregisterSystemStore takes a flags parameter, and the actual values of the different flags are defined in C++ .h files, which are not available from C#. So I punched out a few PowerShell commands to search the .h files in the Windows SDK for those #define lines.

Once those flag values were found, deleting the store ended up being this small C# script in LINQPad that simply calls CertUnregisterSystemStore with the appropriate flags:

void Main()
{
    int CERT_SYSTEM_STORE_LOCATION_SHIFT = 16;
    uint CERT_SYSTEM_STORE_CURRENT_USER_ID = 1;
    uint CERT_SYSTEM_STORE_LOCAL_MACHINE_ID = 2;
    
    uint CERT_STORE_DELETE_FLAG = 0x10;    
    uint CERT_SYSTEM_STORE_CURRENT_USER = CERT_SYSTEM_STORE_CURRENT_USER_ID << CERT_SYSTEM_STORE_LOCATION_SHIFT;
    uint CERT_SYSTEM_STORE_LOCAL_MACHINE = CERT_SYSTEM_STORE_LOCAL_MACHINE_ID << CERT_SYSTEM_STORE_LOCATION_SHIFT;
    
    CertUnregisterSystemStore("TestCertificate.cer", CERT_STORE_DELETE_FLAG | CERT_SYSTEM_STORE_CURRENT_USER);
}

[DllImport("crypt32.dll", CharSet = CharSet.Unicode)]
public static extern bool CertUnregisterSystemStore(string systemStore, uint flags);

I’ve also included the flags needed to delete stores from the local machine (as opposed to the current user account) in the script above, in case anyone ever needs to do that.

I used to like PowerShell a lot. It seemed like a decent scripting language that extended the .NET Framework, so anything I could do in .NET I could do in PowerShell. That's still true, except I find that every time I try to use PowerShell to quickly whip up some small solution, I spend far too long messing around getting choked by its black magic. It would have been faster to write a command-line app in full blown C#.

I guess you could say that if I knew PowerShell better, this wouldn't happen, and that would be true. However, I don't write full blown applications in PowerShell... that's not what it's for. I'm not a sys-admin and don't want to be, so I don't spend a lot of time scripting. So the few times I want to write a quick script I just want to quickly crack out a script and have done with it.

My current rage against PowerShell has been evoked by the way it handles filepaths. PowerShell lets you put wildcards, such as [0-7] or *, into your paths that you pass to its cmdlets to do things. The problem occurs when the directories you are using contain characters that are used for wildcards (the square brackets, particularly). PowerShell totally craps out. I'll run you through a short example.

Create a directory called "Files [2000-2008]" in C:, then under that create two other directories "Word Docs" and "Excel Docs from my Sister [2007]". Open up PowerShell and cd to C:\Files [2000-2008]. Now type "cd" and try to use tab completion to go into Word Docs or Excel Docs. Oh what? It's not working? Yep, broken. Okay, so you'll have to type out "cd 'Word Docs'" to move into that directory. Dodgy, but no real problem.

Okay, now "cd .." back into Files. Now try to get into Excel Docs. Maybe you'll type (since tab completion is rooted) "cd 'Excel Docs from my Sister [2007]'". Nup, doesn't work, apparently it doesn't exist! What crack is PowerShell smoking (yes, smoking!)? The wildcard crack. You need to escape the square brackets like this "cd 'Excel Docs from my Sister `[2007`]'". Yeah, what a pain, too much typing.

What's an easier way of moving into that Excel directory? You can go "(ls).Get(0) | cd" to get the first folder returned by ls and pipe that into cd. That seems pretty cool until you realise all you're trying to do is cd into a damn directory. But it doesn't end there.

Put a couple of Word docs into Word Docs and also put a .txt file in there. Now maybe you would like to get only the Word docs and filter out any other file, so you do what you normally would: "Get-Item *.docx". What? Nothing found? How can that be? You can see the documents in there! The reason is that PowerShell is getting its knickers in a knot because its performing that command on the current working directory that happens contain square brackets in it. So it's trying to do some wildcards tomfoolery even though all you're trying to do is filter on only .docx files.

So how can you work around this? A few cmdlets let you pass in a -LiteralPath instead of a -Path, which will ignore the wildcard characters in the current path. But it won't work for this because we're trying to use wildcards to filter on .docx (the *). A solution is to do this: "Get-ChildItem | Where-Object {$_.Name -like "*.docx"}". But doesn't that just strike you as crap, considering we're supposed to be able to do fancy wildcard stuff easily, instead of manually like in that command?

"But Daniel", you say, "this only happens when you have directories that contain square brackets! Just do this tomfoolery when you encounter directories with brackets in them!". Sounds good, until you want to write a PowerShell script that does something and you can just reuse it wherever. The script I was writing would rename video files from one naming format to the one I prefer. It worked okay, until I tried to run it on a directory with brackets, then it crapped itself. What's the point of having this fancy wildcard stuff, if I can't use it because it means I'm writing a script that might break depending on whether the directories contain brackets?!

Just to rub salt in the wound, try to rename a file with brackets in its filename in Word Docs. Add a file called (and these are realistic names, the latter is how I name video files) "Pure Pwnage [s01e02] Girls.avi". Maybe you want to change the name to "Pure Pwnage [s01.e02] Girls.avi". Remembering you're probably doing this in a script, and so you can't just add in backticks to escape square brackets (unless you do a find and replace (LAME!)) you'll use -LiteralPath on Move-Item: "Move-Item -LiteralPath 'Pure Pwnage [s01e02] Girls.avi' -Destination 'Pure Pwnage [s01.e02] Girls.avi'".

That'll spit back the unintelligible "Could not find a part of the path" error. What does that mean!? Using the -Debug switch doesn't produce a more useful error message. Turns out -Destination takes wildcards so it's reading the "[" and "]" in the name as wildcards. That's fine, we'll just use -LiteralDestination. Except, there is no -LiteralDestination! Nice.

So how do we get around this (notice a pattern of having to get around things)? I found changing the -Destination to: ($PWD.Path + "\" + 'Pure Pwnage [s01.e02] Girls.avi') works. Basically we're prepending the Present Working Directory path to the filename. Then the rename works.

As you can see, as soon as you want to do things with files and folders with square brackets in them, PowerShell blows a gasket. Most of the time, this isn't an issue, until you want to write a script that doesn't just fall over and die at the sight of a square bracket. Which would be all the time, wouldn't it?

Hopefully, this crap will be fixed in PowerShell 2.0, not that I've actually looked to see if it is. Certainly it's incredibly frustrating. But to finish on an up note, I did make an awesome PowerShell script recently that showed the err... power... of PowerShell. I was reading in some XML output from the Subversion command line app, loading .NET's XML DOM parser, reading the XML into that, reading the DOM using XPath and doing stuff with the data, including sending an email to myself if the script encountered an error in the XML. That was awesome and easy to do when you're backed by the full .NET Framework. So I'm not writing PowerShell off just yet.

Lately, I've been getting annoyed at the state of my music and Audiobook collection. Each Audiobook can often be made up of hundreds (one has over a thousand) of small MP3s that allow me to easily skip through the book and also easily remember where I was up to.

But unfortunately, these small MP3s are not tagged and named correctly. Often, they are in correct order on the file system (alphabetically, by their filenames), but not by MP3 ID3 tag. This makes it a pain to play in my media player, especially on my iPod where there are practically no sorting functions.

I looked at various renaming and retagging solutions out on the web and after one of them completely scrambled one of my albums by putting the tag of each song on another song, I decided I needed something that just worked and was really flexible.

I always imagined how good it would be if I could just whip up a quick program to run through those thousand MP3s and name them correctly. So today I decided to create such a solution.

I wrote a small (~90 lines) console application in C++ called ID3CL (ID3 Command-Line) that uses the open source id3lib library to edit the ID3 tags of MP3 files. It takes in command-line arguments and retags a single MP3 file. Its command-line syntax is as follows:

Usage: id3cl <mp3 filename> -set <fieldname> <value>
       [-set <field name> <value> [..]]

Fields: tracknum, artist, album, title, year
        comment, genre

You basically invoke it like this: id3cl mysong.mp3 -set artist "DJ DC" -set title "Foobar on rocks". That will set the artist and title of the mysong.mp3 file.

Of course, this one-file-retagged-per-program-execution solution doesn't seem like it'd help me with retagging over 1000 MP3s does it? That's where scripting comes in.

I've recently been going nuts over PowerShell, the newish scripting language from Microsoft which is out to get rid of batch files (yay!). Writing PowerShell scripts is kind of a cross between writing C# and writing Bash. Its got some odd things in it (like '"{0:2D}" -f 2' will format 2 to be 02) which can make it almost as incomprehensible as Bash, but most of the time its a pleasure to work with (like C# and unlike Bash).

So, by writing a script in PowerShell which invokes my little C++ app (ID3CL), I can write tiny programs that retag my MP3 files any way I want.

Here's a little PowerShell script that takes MP3 files from the folder that the script is run in (and any files in folders under that one as well (recursively)) and changes their track numbers so the first one is 1 and the second 2, and so on.

$id3cl = "& 'D:\My Documents\Visual Studio 2005\Projects\ID3CL\release\id3cl.exe' "

$mp3s = Get-ChildItem * -Include *.mp3 -Recurse

$tracknum = 1

foreach ($mp3 in $mp3s)
{
    $cmdline = '"' + $mp3.FullName + '" -set tracknum ' + $tracknum
    Invoke-Expression ($id3cl + $cmdline)
    $tracknum++
    if ($tracknum -eq 256)
    {
        $tracknum = 1
    }
}

This script is useful when I've got a two CD album, and I've got each CD from the album in its own folder. Each CD is treated like its own album with tracks starting from 1 and going on. But the thing is, I don't want to treat the album as two albums, I want one album with in-order track numbers. So that script will take CD1 and set the track numbers from 1 to X and then take CD2 and set the track numbers from X + 1 to Y. All automatically.

So you can see the power of this little system I've created. Unfortunately, only a programmer would be able to make use of this, since you've got to write scripts to do anything useful. But that's what makes it so powerful.

ID3CL is definitely me-ware. It's not user-friendly. It'll do silly things like if you get it to change the tag on a file that doesn't exist, it'll create a music-less MP3 and put your tags on it silently with no error. I can't be bothered fixing such bugs because it works perfectly when you treat it nicely and give it exactly what it expects. This initially made me not want to put it online for you guys to use, but I think I will anyway. Soon™ :). But if it errors because you did something odd with it, you'll have to figure out its unhelpful error messages.

However, I think its worth it for the power it gives you to tag your MP3 collection.

Bah, I seem to be attracting hardware failures of late. The new stick of RAM I bought for my laptop in June decided to up and die, corrupting my Windows installation along with it. Luckily it has lifetime warranty, so I didn't lose anything, except my patience with the sluggish remaining 512MB of "not enough" RAM and having to reinstall everything which sucks when you're a developer (it takes ages).

But let's move onto the more interesting things. What I've begun doing is having a folder in my bookmarks in Opera, and when I get a particularly interesting article I stick it in there to write about later. This should mean I will blog more frequently*.
* Terms and Conditions Apply. :)

So. The first item: Windows PowerShell has gone 1.0! As we all know, the standard command prompt and scripting offered in Windows blows when compared to Bash in Linux. PowerShell is here to rectify that. However, don't go jumping into it thinking that you can just run all of Windows from the shell. Windows is still a strongly GUI-centred operating system and you can't just run the OS from the command-line like you can in Linux. Certainly it has been touted to make Windows Server administrators' lives easier, but unlike Linux, most apps for Windows aren't written with command-line functionality or COM interfaces.

The PowerShell syntax is a weird amalgam of C# syntax with a little Bash and some weirdness thrown in there for good measure. I almost wish it was more C#ish; just some things like the equality operator being -eq, as opposed to the more C-style ==, seem strange when you are doing C# style foreach loops.

Where Bash is often centered around plain text hacking, PowerShell does it differently. When you "pipe" things around you are piping objects. Yes, PowerShell is weirdly object oriented. Kind of. PowerShell is built on top of the .NET Framework, and it shows through. Passing objects around instead of plain strings is better since different cmdlets (pronounced "command-lets", these are the commands in PowerShell) can act on the objects differently without the need of string hacking ala Bash. For example, instead of (in Bash) getting a list of files, using awk to rip out the filenames then throwing them into file, PowerShell does it by getting objects that represent the files and passing those objects to some other command which will extract the filename object property and write them to a file. Its a crappy example, I know, but I haven't spent a lot of time in PowerShell yet. :)

If you're interested, you can download PowerShell here, and read a rather good starter tutorial here. In my summer holidays I'm looking forward to fiddling around more in PowerShell.

The next item on today's agenda: threading in the Source Engine! If you don't know what the Source Engine is you either live under the "I don't play computer games" rock or you play way too much Starcraft. For you people, the Source Engine is the game engine that powers the bestselling Half-Life 2 game and has been licenced for other good games like Dark Messiah - Might and Magic, and Sin Episodes.

Most game engines these days don't properly take use of dual/quad core CPUs because they are not "multithreaded". A program that is multithreaded has multiple lines of execution all running concurrently. This means, on a multi-core computer, more than one thing is happening at once. If your game isn't threaded it pretty much means a whole half (or three-quarters or whatever) of your CPU is going to waste. So its an important thing for games to become multithreaded.

Valve (the makers of the Source Engine, oh uneducated ones :P) have started work on making the Source Engine multithreaded. This is difficult since threading can be a real pain in the butt and will require a large amount of the engine to be rewritten. There are three main ways that multithreading can be done in a game engine: in a coarse fashion, in a fine fashion, and in a hybrid fashion that uses elements from both coarse and fine.

The coarse fashion is where different game subsystems are put on different threads. Valve found this to be ineffective in utilising the entire CPU fulltime. The fine fashion is where low level tasks are split across cores. This method was also unsatisfactory since not all tasks are well suited to being split in this fashion. Valve settled on the hybrid method which pretty much means it uses the coarse fashion where it suits the problem and the fine fashion where it suits the problem. This way is the most complex but it scales well and maxes out the CPU.

What Valve has done is to create N-1 threads (for N cores on the CPU) with the other thread being the master controller thread. Valve uses lock-free algorithms to help remove the problem of threads sitting around blocking (doing nothing) while they wait for access to data (two threads cannot write to the same piece of data at the same time, that would be bad).

Multithreading in Source can only bring benefits to Source-based games. I know that currently half of my CPU (1 core of 2) sits around doing nothing when I play games, and last time I checked I didn't fork out good cash for it to be slacking! There is a full on article about multithreaded Source which goes into more detail and has a good focus on the technical side of the threading, which a lot of the other articles about this didn't.

A nice thing to hear is that Valve uses iterative development on the Source Engine (building and improving it piece by piece over time, rather than writing it and then rewriting it from scratch for upgrades) because my course at University likes to rave about iterative development. Wonder whether they do unit tests :).

And finally on today's show is a little something to back up my rant on Apple a few blogs ago. I will now degrade into IM-speak: LOFL, ROFLMAO, LOL.