Tuesday, February 25, 2014

From Startups to MegaCorp: Unlearning to fix everything

In the startup world when you discover an issue, whether it's a logic error or an environmental misconfiguration, being able to fix it quickly and, maybe, document the fix is a virtue.

A hard lesson that I'm learning is that in a different environment that which was a virtue in the startup world can actually be harmful. By fixing an environmental issue you can deprive the organization of the process correction necessary to handle it structurally.

This strikes me as analogous to the decision to *not* handle an error in code. Sometimes you intentionally don't handle an error even though it is anticipated because to do so masks a more fundamental problem. The "if this error occurs, something is deeply wrong in a way that can't be recovered automatically" is a widely established pattern in software construction.

The hard part is realizing that it generalizes to the organizational level. By fixing an environmental issue that another team is responsible for you may well deprive that team of a needed corrective. What happens when you're too busy working on problems that can only be handled by your team and the issue comes up again?

Saturday, February 22, 2014

Expressivity vs Trickle-Down Layering

A basic architectural principle in software engineering about dependencies across layers is that the flow of dependencies should go in a single direction. For example, in a hypothetical 3 layer system composed of top, middle and bottom layers the top layer depends on the middle layer and the middle layer depends on the bottom layer.

If the middle layer reaches back up into the top layer then you've created a circular dependency. Circular dependencies make a system harder to understand (and therefore maintain). One way in which they do this is their tendency to create chicken-egg problems. Put another way, they introduce another axis of dependency (order/time) that may not be obvious from inspecting the code. Chicken-egg problems aren't necessarily intractable, clearly *something* came first, but by adding another dimension they exponentially increase the size of the solution space making for many more blind alleys in which to get lost.

Working on a large system built up over time by many different contributors has given me a much greater appreciation for the importance of this basic architectural principle. This is a bit of a hard-learned lesson because it tends against one of my favorite design principles: the composite pattern. A layer built based on the composite pattern can reference itself as well as lower layers. I'm playing a little loose with the terminology here because layers generally refer to larger pieces of a system while the composite pattern is usually applied to smaller pieces of a large system but bear with me.

Let's take a hypothetical system called Manuals. The Manuals system is maintained by the Manuals team. Manuals depend on Chapters. Chapters depend on Sentences. Sentences are the lowest level of the system - this is where the meat of the work is done. The higher layers (Chapters and Manuals) are primarily about organizing/ordering the work of Sentences.

A few months after the Manuals system ships Developer X encounters it and, being fresh full of ideas about design patterns, thinks it would be really useful if Chapters could be composites. That is, he'd like Chapters to be able to contain Sentences and Chapters. He thinks the system would be so much more expressive that way. And it would enable reuse at the Chapter level. Who doesn't love wringing more expressivity and reusability out of a system?

Fast forward 2 years. Half a dozen developers have rotated in and out of the project each with varying levels of expertise, varying exposure to the original designs and different constituents to satisfy. None of the original developers is still working on it. The design docs, which were originally shared on the shiny, linky (e.g., non-duplicative) intranet were lost long ago somewhere between conversions to Intranet v2 then v3 and now v4 (the current version). Even if they could be found they'd be horribly out of date as the codebase is 10 times its original size. You're tasked with what should be a simple fix: change the wording of a boilerplate sentence that appears in nearly every Manual produced since the project originally shipped.

Being new to the project what do you do? You look at the manuals you can find. You see that each Manual appears to be made up of Chapters and each Chapter appears to be made up of Sentences. So you start your search in the Chapter directory and change every instance of the boilerplate sentence in each of the Chapters.

In a system that maintains the downward flow of dependencies (Manuals -> Chapters -> Sentences) there's a very good chance that this particular bug has been quashed: the boilerplate sentence is updated everywhere it will appear.

But what if Chapters were Composites in the system: they contained both Sentences and Chapters? Although originally all the Chapters were in a single directory, someone once accidentally chose a Chapter name that was already in use (without knowing it) so there was a major refactoring project that moved Chapters contained by other Chapters into other directories. A few months after that refactoring there was a big push to support the creation of chapters by another team. The Manual team just didn't have the bandwidth to handle the demand for Manuals. But the other team stored their artifacts in a different location. So to keep them productive the Manuals team added support for chapter references - pointers to chapters defined in other locations.

Now the job of changing a single boilerplate sentence everywhere it appears has gone from being a grep/findstr in a single directory to a programming task in and of itself. One that crawls Chapter references and looks for boilerplate sentences in the pointed-to Chapters. And doesn't crash on stale references. And doesn't infinitely loop when circular Chapter references are found. All because the downward flow of dependencies was broken when the elegant composite pattern was introduced into the system.

Sunday, December 22, 2013

Upgrading WiFi in a Lenovo T61 ThinkPad to a Windows 8.1 Compatible card via USB Drive

The stock WiFi card (an Intel 4965 AGN) on my ThinkPad crashes every 30 minutes under Windows 8.1 (netwlv64.sys every time).

It also seems not to like the 5Ghz channel (from an Asus RT-N66U) though this was true under Win 7. So I upgraded to an Intel 7260 which is performing flawlessly.

BEFORE Changing the WiFi card

  1. Download the Middleton BIOS iso.
  2. Download rufus from pendrivelinux.com
  3. Follow the instructions at pendrivelinux to format the usb drive as a bootable DOS disk.
  4. Copy the files from the Middleton BIOS iso to the usb drive. (either mount it or use 7-zip to extract the iso contents).
  5. Make sure your battery is near fully charged (battery icon is green not yellow on thinkpad indicator).
  6. Disable the TPM chip in the bios.
  7. Suspend BitLocker if it's on.
  8. Boot from the USB drive.
  9. Run the flash utility (lcreflsh.bat).
  10. Wait for several minutes.

Changing the WiFi card

  1. Follow the instructions here for removing the wifi card.
  2. Insert your replacement wifi card.
TPM can then be re-enabled and bitlocker resumed after rebooting.

Wednesday, December 18, 2013

Upgrading to an Intel Core i7-4770k CPU: UEFI, USB 3 and Windows 8.1

Finally got around to upgrading from an Intel Core i7-920 to a Core i7-4770k. These are roughly 3 processor generations apart (Nehalem microarchitecture to Haswell microarchitecture respectively). As has been their custom for as long as I've been building PCs, the switch necessitated a motherboard upgrade as the 920s used socket LGA1366 while the 4770k requires LGA1150.

I decided to stick with an Asus (pronounced uh-soos) motherboard and picked up their Z87-Plus. The board being replaced, the P6T, was also an Asus board. Come to think of it, so was its predecessor (the A8N-sli). Before that it was an Intel board. Even though I don't do much overclocking I love the attention to detail that Asus puts into its products. And its website is well organized - always easy to find drivers. A few years back a quick comparison of their website to the website of their competitors (Gigabyte, MSI, etc...) sent me flying into their arms. They're not the least expensive boards but I've had good experience with them.

I fallen in love with the front panel header connector they ship with their boards. You plug the front-panel connectors (power LED, reset switch, PC speaker, etc...) into the connector (pictured below) then plug the connector into the board. It's a lot easier to swap out motherboards because you don't have to reconnect the sometimes lilliputian shunts onto single pins.

Front Panel Connector
The processor itself is tiny. Back in the Pentium and Pentium 2 days the processor was huge. Size-wise I recall installing one that was somewhere between the size of an audio cassette and VHS tape (closer to the latter than the former). The processor's these days are not much bigger than a postage stamp though their fans seems to have gotten larger.

CPU and stock fan
One of the reasons I went with the Z87-Plus is that it comes with a UEFI firmware config (aka BIOS). UEFI is the next generation of computer firmware, the successor to BIOS with an emphasis on speed and security. UEFI includes a sophisticated menu system so there's no longer a need for add-on board ROMs to daisy chain prompts and increase the length of time it takes to boot. Beyond that, BIOS writers have richer libraries and greater access to machine resources. The UEFI BIOS on this thing blows me away - it's a mouse-driven modern graphical user interface instead of the standard text based interface that has been a staple of BIOS for over 2 decades.
UEFI BIOS for Asus Z87-Plus
And it displays each of the settings that have been changed in a confirmation dialog before saving them!
UEFI BIOS Confirmation Dialog (apologies for the fuzzy picture)
Performance-wise this thing is a beast. Even though it only has 8 Gigs of RAM Windows 8.1 consistently boots in under 10 seconds. The motherboard itself completes POST so quickly that I've had to turn on a 5 second pause (another nifty BIOS option) so that I have a chance to enter the BIOS if necessary before POST completes. There are all sorts of optimizations this BIOS offers; you can turn off any (or all) of the USB or SATA ports. You can disable initialization of pretty much every connected device. There's something called "hardware turbo" mode that is so fast that I had to turn it off (again because it was nearly impossible to enter the BIOS when POST completes in under a second).

As a pc hobbyist since the mid 90s it amazes me how much easier it has become to build your own PC. I haven't cut myself on an add-on card or connector in going on 10 years. :)



MSBuild: To Exec or Task?

So I needed to execute a custom task, mainly to transform some XML, as a part of the build. The build has largely migrated to msbuild. I've worked with make, ANT and other build systems before and they more or less all provide the same functionality: A way to specify a dependency relationship that ultimately ends in telling the compiler/linker what and when to compile/link.

MSBuild seems to take the object oriented metaphor a little deeper as it relates to XML. That is, both the verbs of the system (Targets and Tasks in msbuild-speak) and the nouns of the system (Properties and Items) are represented as elements. The best way to get a handle on these concepts is to peruse "MSBuild Concepts"

One of the first gotcha (there are always gotcha moments with build systems) moments I've encountered has been related to the Exec task. It's a built in task that ostensibly executes any command passed to it. Except that it seems to not like managed executables. Or parameters with whitespace (yep, 2013 and whitespace is still dangerous). It'll happily open notepad.exe (even without a fully qualified path) but chokes on a "Hello World" C# console app. But not a C++ "Hello World" app. And the exit code is usually -21474.... which to any programmer looks a lot like -(2^31) - 1 and is usually indicative of a bug lurking somewhere.

Fortunately it's pretty straightforward to roll your own Task. I love it when they provide both an interface and a handy default implementation so that you only have to override the behavior in which you're interested.

If you're doing anything other than calling cmd.exe intrinsics (e.g., echo, dir, etc...) then I highly recommend doing it inside a custom task.

Wednesday, December 11, 2013

Removing Printer Drivers in Windows

This can be done with the command:

printui /s /t2

Make sure to remove the printer from devices and printers first.

Make sure to remove the right printer (e.g., one that's using a compatability print driver will require deletion of that compatability print drivers as well as the device specific driver).

Thursday, December 5, 2013

GhostDoc, Conventions and Writing Self-Documenting Code

Given the C# method signature:

        private void CreateImageStreamAndAddImage(
            FileInfo imageInfo,
            int ImageBufferSize,
            ImageStreamInfo streamInfo,
            IPictureUploadService pictureUploadService)

GhostDoc will generate the following xml doc comment:

        /// Creates the image stream and add image.
        ///
        /// The image info.
        /// Size of the image buffer.
        /// The stream info.
        /// The picture upload service.

which, IMHO, is almost perfect! (add should be adds). This conceptually simple trick, a bit of parsing, stemming and grammar, goes a long way towards encouraging method names that clearly communicates what the method does.

This wouldn't be nearly accurate enough if it weren't for the fact that C# (well, .NET in general) ships with naming guidelines. They're illustrated in the language spec, documentation examples and the  design guidelines for class library developers.

The more team development I do the more I appreciate the benefit of conventions designed to aid readability. Team members come and go but the code is always there. Documentation comments tend to rot (easily corrected with GhostDoc but still) but method names last until changed.

Clarity in method naming strikes me as akin to clarity in writing. A well written explanation can be orders of magnitude* easier to understand than a poorly written one. It's just as easy to muddle concepts in print as it is in writing - perhaps this is why it's called writing source code. Throw in teams comprised of members from different language backgrounds and these conventions become even more important (as does one's appreciation for the hurdles being overcome by amazing devs writing in a non-native language!).