Building My Ideal Mechanical Keyboard with QMK

Photo by Florian Krumm on Unsplash

The mechanical keyboard market has exploded in the past decade. Almost every major PC peripheral vendor offers one in some capacity, and especially those targeted towards PC gamers. I discovered the more niche custom built keyboard community during my last year in college back in 2018. A classmate and good friend of mine had me receive his Happy Hacking Keyboard he had shipped from Japan at my apartment, since he would be out of town. I opened it under his instruction to make sure it arrived in one piece, and after hitting the escape key just once, I absolutely knew I had to have one.

There and Back Again, a Rubber Dome Story

My first “gaming” keyboard

My first exposure to a mechanical keyboard was likely to some crusty old beige IBM clacker from the very early 2000’s. My father would often bring home miscellaneous computer equipment from the various startups he worked for, so it is likely at least one of them was mechanical. For the remaining decade I used various run of the mill rubber dome keyboards from Dell, HP and the like and didn’t think twice of it. In 2007 I obtained my first gaming PC, and along with it I purchased a Saitek “gaming keyboard” from Fry’s Electronics. It had an overtly industrial design (to a fault), with a cheap plastic silver case, flimsy plastic wrist rest, sticky feeling rubber dome switches, and a blue backlight behind keycaps that did not even have transulscent lettering. In hindsight, it was a godawful keyboard, but oh boy did I love it. It glowed blue after all.

Fast forward to 2010, and I pre-ordered the keyboard that arguably kickstarted the mechanical keyboard craze in the PC gaming segment: the Razer Black Widow. I vividly remember the marketing making a big deal out of its Cherry MX switches, claiming that they improved your gaming performance. Every few years or so after, I would upgrade to various mechanical keyboards from Corsair and Coolermaster, until I bought a Realforce 87 with Topre switches, all because of that one experience with my friends Happy Hacking Keyboard. Ironically, Topre’s are actually rubber dome switches. They are unique however in that they are actually a combination of a spring covered with a rubber dome, which gives it a consistent actuation that doesn’t “buckle”, but with a very round and satisfying tactile bump.

Ironically, Topre’s are actually rubber dome switches.

The “Best” Keyboard

I want to take moment and acknowledge that mechanical keyboards that claim to improve gaming are effectively bulls**t. If there was a keyboard that is objectively better for gaming, it would be anything with low key travel, since that would minimize the lag between intention to press key and it registering with your game. Outside of games however, it is difficult to determine what constitutes a “good” keyboard for productivity.

As a software engineer, the keyboard is a clearly a critical tool to getting my work done. It is the main interface to which the majority of programmers write their code. The reality of software engineers though is that they do not spend most of their time writing code at all. Most of it is likely spent writing Slack messages, reading documentation, or wasting away trying to find the most idiomatic name for a function. It is hard to believe that any qualities of a keyboard would translate to any measurable improvement of work output. Even if we assume words-per-minute was a valuable metric, mechanical keyboards are worse in this category when compared to a modern Apple keyboard that famously sport low travel keys.

Form over Function

Continuing on the topic of keyboards made my Apple, the Magic Keyboard with TouchID is on paper the “best” keyboard for my workflow. It matches macOS’s idiosyncratic keymap, and TouchID can let you access the Secure Enclave for fast unlock, Apple Pay, and much more.For some inexplicable reason however, I just don’t really enjoy using one, despite acquring 4 of them across different jobs. So in effect, the “best” keyboard is subjective, and explains why qualities such as feel and sound, and visual aesthetic have taken the front seat over function. Like the car, the keyboard has become an extension of one’s personality. Custom cases, keycaps, and switches are yet another avenue for someone to express themselves.

Apart from visual aesthetics, the sound and feel of a keyboard is important factor in choosing one. “Thocky” is a new word the community has made up to describe the deep, low-pitched sounds certain keyboards make. It’s near impossible to describe in text, but the experience is satisfyingly cathartic and reminiscent of ASMR. Seriously, look it up, people publish typing videos as ASMR.

Like the car, the keyboard has become an extension of one’s personality.

I personally love “thocky” keyboards, but finding the perfect one is a difficult endeavour. I don’t know enough about the science of switch construction, keycap material, case, backplate, and way more that all interplay to register the “perfect” sounding key actuation. Impatient me resorts to listening to experts in the field, such as Taeha Types and Switch and Click to find help find the correct products.

The Build

I do not intend to dedicate too much of this article to the actual physical build of the keyboard. There are hundreds of amazing YouTuber’s and bloggers that cover this subject way better than I could ever do. That being said, the components I ended up with are:

• NovelKeys 87 Entry Edition Case+PCB (Black Polycarbonate)
• Glorious Pandas Switches (Pre-lubed)
• GMK Modo 2 Keycaps with macOS modifiers

Hastily taken photo from my desk

It has a satisfyingly “thocky” key feel, and a sleek dark gray aesthetic with bright neon green accents. These are by no means the best you can get, but to me, this build struck the right balance of having good feel without being ridiculously expensive. At the time of writing, you can simply purchase the the NovelKeys 87 and Glorious Panda Switches and actually receive them within a week. Most keyboard components, like the GMK Modo 2 Keycaps listed above, are only available via group buys and can take a very long time for the order to actually land in you hands after payment, sometimes taking up to several years (GMK Dracula, I’m looking at you). As much as a sucker I am for the look and feel, I still very much value how the keyboard performs, particularly in how the software interacts with my complicated desk setup.

Seeking Perfection

The remainder of this post is very technical in nature, so if programming isn’t your thing, the following sections might be a bit hard to follow.

KVM Woes

On my desk at home are two computers: a Windows gaming PC, and a brand new M1 Ultra Mac Studio. The peripherals are switched between the two by the excellent Level1Techs Display Port 1.4 Dual Monitor KVM. Anyone familiar with both Windows and macOS know that the keyboad layouts are ever so slightly different, with most notably the Altand Win/Cmdkeys swapped on both the left and right sides of the space bar. Normally, one would just configure in software on either the PC or Mac to translate the Win keycode to Alt and, Alt to Cmd, but the obsessive perfectionist in me cannot tolerate this solution because it only works once the software has launched. So if you are operating the computer at any time before the user has logged in (such as FileVault password screen), the keys will remain swapped. And more recently, this causes trouble with Apple’s fantastic new Universal Control feature, which does not play nicely with software that translates keycodes. The effect is the Cmd key on your primary mac with your keyboard attack may register as Alt on any other device attached with Universal Control since the translation is not performed for the remote device. The best solution to this is to configure two sets of layouts on the keyboard that can be selected on the fly, with one mimicking an Apple Keyboard. However, this now means that switching between computers on the KVM introduces a new problem: you have to be conscious of which layout your keyboard is currently operating in and flip it over manually with some special key combination. So, with everything in mind, I can attenuate my quest for perfection as solving these five problems:

1. The keyboard should support a Windows Key layout.
2. The keyboard should support a macOS layout.
3. The keyboard should be able to swap between layouts on the fly.
4. The keyboard should operate like an Apple manufactured keyboard (Media keys, Function key, etc.)
5. The keyboard should intelligently swap layouts based on which host is active in the KVM.

Points 1–3 are relatively easy to do with QMK Configurations right out of the box. You can define separate layers in QMK Configurator and define a special key to swap between them. Points 4 and 5 are much more complicated however, and will require us to edit QMK firmware beyond just swapping keymaps.

The Apple Magic Keyboard

Beyond having macOS specific modifier keys and special media keys, the Magic Keyboard operates a bit differently than a standard HID device, particuclarly when it comes to handling its Fn function key. Normally when a keyboard has a function layer, holding the function key down will cause certain key presses to send an entirely different key to the host computer. For example, if the F1 key had a function layer action to lower the volume, holding the function key and pressing F1 would send a key code that indicates to lower the volume, not the F1 keycode itself. The host OS has no idea that the function key was held at all, since it is all internal to the keyboard. On Apple Magic Keyboards, this is not the case at all. The “Fn” key is actually a discrete modifier key, and the OS will receive key-down/key-up events similar to traditional modifier keys such as Ctrland Alt. In fact, the Apple Keyboard uses up 1 of the 6 key slots explicitly just for the function key!

USB Keyboard Rollover

To make more sense of the previous statement, a quick primer of how USB keyboards work is in order. Generally, usb keyboards support pressing up to six keys at a time, which is canonically known as 6-Key Rollover (6KRO). This can be easily tested at home by pressing and holding down 7 or more keys into Notepad. Only 6 characters will show up in the editor, and any further key presses will be missed. Newer keyboards support N-Key Rollover, where each key can be scanned independently, but as far as I know, no Apple Keyboards support N-KRO. Apple Keyboard USB protocols actually deviate from the standard 6 key protocol, and reserves 1/6 of them exclusively for the “Fn” key, effectively meaning the Magic Keyboard is only 5-KRO.

Simulating an Apple Keyboard in QMK Firmware

One of the great things about building your own custom keyboard is that you can flash your own firmware to customize it any way you would like. QMK Firmware is the gold standard and basically any custom keyboard PCB worth buying supports QMK firmware. With the ability to customize firmware of a keyboard, it seems reasonable that it could be modified to mimic most of a real Magic Keyboard’s functionality.

It turns out that the macOS driver code that handles keyboard devices treat Apple Keyboards differently than the rest. If one examines the IOHIDFamily source, they could see that there are checks to see if a device is indeed an Apple Keyboard with a function key. The GitHub user fauxpark describes the situation in excellent detail in this issue thread for qmk_firmware.

If you look at an Apple keyboard in HID Explorer, you can see that right at the bottom of the list is a strange entry… Sure enough, pressing the Fn key causes the value of this usage to become 1.

…

But there’s a catch. And it’s a big one. The keyboard’s vendor and product ID need to match those of a real Apple keyboard — probably only ones with a Fn key. The product ID also seems to determine whether certain F-keys work eg. Launchpad/Mission Control and keyboard backlighting. The Manufacturer and Product strings can still be whatever you like, but this explains why there are no third-party keyboards with Apple Fn keys, and most likely rules out QMK compatible PCB/keyboard sellers from shipping boards with the key available out of the box too. I can imagine that being a bit of a dealbreaker.

That’s, right the special Fnkey entry is only respected properly if the keyboard’s vendorID and productID match that a real Apple Keyboard. The result is that QMK cannot directly support this feature, since claiming a vendor ID reserved to Apple on a non-Apple product is a big nono (That does not seem to have stopped Keychron however, whose keyboards report Apple vendor and product IDs when they are in Mac mode). Luckily, an adjacent git patch by the very same fauxpark is generally kept up to date. By applying this patch, one can easily compile a version of QMK firmware with Apple Fn key support.

Deep Dive into the Patch Set

Most of the patch set is relatively easy to understand even for newer programmers. The majority of it relates to registering a new type of key to represent the Apple Fn key.

+        /* Apple Fn */
+        case ACT_APPLE_FN:
+            if (event.pressed) {
+                register_code(KC_APPLE_FN);
+            } else {
+                unregister_code(KC_APPLE_FN);
+            }
+            break;

As a quick primer, recall from the previous section that keyboards generally only support handling six keys at once. These are communicated to the host in what is called a “keyboard report.” The type definition of it is shown below:

#define KEYBOARD_REPORT_SIZE 8
#define KEYBOARD_REPORT_KEYS 6
typedef union {
    uint8_t raw[KEYBOARD_REPORT_SIZE];
    struct {
        uint8_t mods;
        uint8_t reserved;
        uint8_t keys[KEYBOARD_REPORT_KEYS];
    }
} report_keyboard_t;

A single byte represents the pressed mods (8 bits, so up to 8 mod keys), and at the tail there is a 6 byte array representing which 6 keys were tracked as pressed.

Now, we can get to the real interesting part of the patch. It begins with some modifications how pressed keys are added to the keyboard report.

void add_key_to_report(report_keyboard_t* keyboard_report, uint8_t key) {
+#ifdef APPLE_FN_ENABLE
+    if IS_APPLE_FN(key) {
+        keyboard_report->reserved = 1;
+        return;
+    }
void del_key_from_report(report_keyboard_t* keyboard_report, uint8_t key) {
+#ifdef APPLE_FN_ENABLE
+    if IS_APPLE_FN(key) {
+        keyboard_report->reserved = 0;
+        return;
+    }

Here, we can see the implementation cleverly uses the reserved byte to store the Fn key status in the report, meaning we retain proper 6-KRO. If we wanted to match Apple Keyboard functionality exactly, we would reduce KEYBOARD_REPORT_SIZE to 5, and add an extra member as such:

#define KEYBOARD_REPORT_SIZE 8
#define KEYBOARD_REPORT_KEYS 5
typedef union {
    uint8_t raw[KEYBOARD_REPORT_SIZE];
    struct {
        uint8_t mods;
        uint8_t reserved;
        uint8_t keys[KEYBOARD_REPORT_KEYS];
        uint8_t fn;
    }
} report_keyboard_t;

USB HID Descriptors

Clearly, if the function key status can be stored an entirely different byte location in the report from normal and still operate, there must be some mechanism for the keyboard to describe to the host instructions on how to interpret the keyboard report. Enter USB HID descriptors, a protocol that allows a device to communicate how to interpret its outbound data packets, also known as “reports.”

The HID protocol makes implementation of devices very simple. Devices define their data packets and then present a “HID descriptor” to the host. The HID descriptor is a hard coded array of bytes that describes the device’s data packets. This includes: how many packets the device supports, the size of the packets, and the purpose of each byte and bit in the packet. For example, a keyboard with a calculator program button can tell the host that the button’s pressed/released state is stored as the 2nd bit in the 6th byte in data packet number 4.

— from Wikipedia on Components of the HID Protocol

const USB_Descriptor_HIDReport_Datatype_t PROGMEM KeyboardReport[] = {
         // Modifiers (8 bits)
         HID_RI_USAGE_PAGE(8, 0x07),    // Keyboard/Keypad
         HID_RI_USAGE_MINIMUM(8, 0xE0), // Keyboard Left Control
         HID_RI_USAGE_MAXIMUM(8, 0xE7), // Keyboard Right GUI
         HID_RI_LOGICAL_MINIMUM(8, 0x00),
         HID_RI_LOGICAL_MAXIMUM(8, 0x01),
         HID_RI_REPORT_COUNT(8, 0x08),
         HID_RI_REPORT_SIZE(8, 0x01),
         HID_RI_INPUT(8, HID_IOF_DATA | HID_IOF_VARIABLE | HID_IOF_ABSOLUTE),
++#ifdef APPLE_FN_ENABLE
+        HID_RI_USAGE_PAGE(8, 0xFF),    // AppleVendor Top Case
+        HID_RI_USAGE(8, 0x03),         // KeyboardFn
+        HID_RI_LOGICAL_MINIMUM(8, 0x00),
+        HID_RI_LOGICAL_MAXIMUM(8, 0x01),
+        HID_RI_REPORT_COUNT(8, 0x01),
+        HID_RI_REPORT_SIZE(8, 0x08),
+        HID_RI_INPUT(8, HID_IOF_DATA | HID_IOF_VARIABLE | HID_IOF_ABSOLUTE),
+#else
         // Reserved (1 byte)
         HID_RI_REPORT_COUNT(8, 0x01),
         HID_RI_REPORT_SIZE(8, 0x08),
         HID_RI_INPUT(8, HID_IOF_CONSTANT),
+#endif

Here we can see the macros to define the descriptor in tmk_core/protocol/usb_descriptor.c. The first argument to theHID_RI_*(size, data) macros is how many bits to emit to the array, and the second argument contains the actual data byte(s) to write. At the beginning we can see how the descriptor reports how to interpret the first byte which represents the modifier keys. We can see an Usage Page that indicates this is for a keyboard, and the minimum and maximum Usage IDs, 0xE0 and 0xE7, respectively. The logical minimum and maximums define the range of possible values in this section the report, which in this case is a single bit, 0 or 1. We can then see the report count is set to 8 instances of 1-bit data, followed by annotation flags. The next part is where things get interesting. The traditionally reserved 1 byte area is #ifdef’d out to contain the usage information for the Apple Fn key. Here we can see the Apple specific Usage Page and ID that represents the function key ( 0xFF , 0x03 ). This will instruct the host that to interpret this byte as the actual Apple function key!

Automatic Layer Switching

With the macOS layout fully working the way I wanted it to, the last item to tackle was to automatically switch keyboard layers to the proper layout depending on what host OS is currently active on the KVM switch. In order to accomplish this, there needs to be some mechanism for the keyboard and OS to communicate some form of state to each other. Thankfully QMK makes this easy with RAW HID support. With just a few changes to QMK firmware, we can easily send and receive small packets of data to and from the host. Similar mechanisms are used by keyboards that support configuration of RGB lighting from software on the host.

Receiving data is fairly simple, by just defining a user defined callback, we can start receiving packets that are sent from the host:

void raw_hid_receive(uint8_t *data, uint8_t length) {
    // Your code goes here. data is the packet received from host.
}

You can also send data back to the host using raw_hid_send(uint8_t*data, uint8_t length, but isn’t entirely useful in this situation beyond debugging.

The NovelKeys 87 allows data packets up to 32 bytes in length, which is plenty enough to build a simple message to instruct the keyboard which layer to switch to. The first byte of the payload will represent what kind of command to process, and the second byte will contain the index of the layer to move the keyboard to.

void raw_hid_receive(uint8_t *data, uint8_t length) {
        switch (data[0]) {
            case MOVE_LAYER:
                 layer_move(data[1]);
                 break;
            default: …
        }
}

(boundary checks removed for brevity)

The more complicated component is writing the software on the host to actually send this message. The general process of doing such is to enumerate over the current USB HID devices until we find a device with the VendorID, ProductID, UsagePage, and UsageID that we expect. QMK out of the box defines the UsagePage and UsageID to be 0xFF60 and 0x61, respectively. Ideally the same code base would be used for Windows and macOS, and as a huge proponent for Golang these days, it seemed like a good choice to get up and running fast.

To build this, I used the hid library from github.com/karalabe/hid, which wraps hidapi using cgo. Enumerating through the HID devices is very easy, and can be done with just a few lines:

infos := hid.Enumerate(vendorID, productID)
   for _, info := range infos {
       if info.Usage == 0x61 && info.UsagePage == 0xFF60 {
       // found raw hid for device
       }
   }

We can now simply send the correct message depending on the current host OS to instruct the keyboard to switch layers:

infos := hid.Enumerate(vendorID, productID)
   for _, info := range infos {
       if info.Usage == 0x61 && info.UsagePage == 0xFF60 {
           // found raw hid for device
           device, _ := info.Open()
           defer device.Close()
           if runtime.GOOS = “windows” {
               device.Write([]byte{0, LAYER_WIN})
           } else {
               device.Write([]byte{0, 0, LAYER_MAC}) // extra 0 byte due to some weird handling of reportID between the kb and hid lib
           }
       }
   }

Stick this in a time.Ticker and install a launchd plist and a Windows service and voila! The keyboard will now automatically switch layers based on the attached host operating system. You can go further and make your keyboard change LED state to better signify which state is in as well.

So now with everything in place, the keyboard seamlessly operates as both a macOS and Windows peripheral. Volume, media, and brightness keys all work as expected, and Universal Control works perfectly as well.