Interfacing with devices

The debrief this week is going to be a bit of a dive into the part of the Bedrock specification that I’m writing at the moment.

What do we need in a spec?

Bedrock was directly inspired by Uxn, the virtual computer project created by Hundred Rabbits. One big difference that I wanted to work towards from the beginning was that I wanted to have a complete specification for the entire system, making it possible for anyone to build their own working system from just that specification, even if every other existing implementation has been lost to time. This means that I can’t just lean on the crutch of expecting the implementer to check how other implementations handle ambiguously defined behaviours, because we can’t assume that there is another implementation to check against. The specification needs to be watertight.

As well as specifying the core system, I also wanted to specify the most important peripheral tools and formats, things like the assembler syntax and the program metadata format. They’re not strictly necessary to get an assembled Bedrock program running, but they are important for interoperability and for collaboration between programmers. It’s easier to put some effort in up-front to lock down the specification for a standard assembler than to have to deal with multiple almost-but-not-quite compatible assemblers down the line. It’ll make it easier for people to modify and reassemble old programs down the line too (because they’ll all be written in a familiar, well-supported syntax).

On top of all of this, there’s a third element of the project that I think should also be specified, which is the actual physical interfaces between the devices and the core system. The core system is more-or-less trivial to conjure up in any programming system on any hardware, it requires only a handful of small functions and a 32-branch if statement. Where the complexity really lies is the device implementations, which act as a thin translation layer between a tidy Bedrock device interface and the often unwieldy reality of how any given platform chooses to expose some capability or another. These device implementations invariably form the bulk of the effort required to bring up a working system, so by standardising the physical and virtual interfaces that are used to ‘plug’ a device into a system, we can allow these device implementations to be more easily reused across systems.

A brief digression on brevity

I’ve toyed around in the past with the concept of radically short specifications, the sort of thing where a specification is contained entirely within a single A4 page instead of taking up a whole booklet, or even cramming the whole thing on the back of a business card. It’s a great badge of honour when talking about a project, because specification length can easily be used as a measure of the ‘simplicity’ of a system, but actually pulling it off tends to involve a bit of sleight-of-hand.

What tends to happen is that a concise notation gets introduced to make it possible to shrink the specification down to the required size, but that notation itself is never fully specified — maybe you write out a couple of examples to explain it, or you lean on conventions and prior knowledge to bridge the gap. You end up with less of a ‘whole specification on a business card’, and more of a ‘quick-reference on a business card’, which is a very handy thing to have but it doesn’t quite accomplish the same goal. Would it really be possible for someone to find that business card lying on the street and then go and implement a working system by themselves, with no prior understanding of the project?

The idea of the one-page spec is still a useful tool for evaluating the brevity of a spec. It requires you to be ruthless, stripping out every non-vital word until you finally expose the two or three narrow steel pillars that have been holding the whole thing up from the start. You can’t stop here though, because you’ve probably also stripped out all of the context and clarifications that make it enjoyable to actually read. A specification isn’t designed for a computer to read, it’s designed for a human, so it still needs to flow and have a semblance of narrative structure. You need to go in and back-fill some of the stuff you took out, which will probably involve describing all of the things that the system doesn’t do, alongside all of the things that it does.

The best guideline I’ve been able to settle on so far is that a spec should be “as short as possible, but no shorter”, which really just means that you’ve gotta try out a lot of different things before you figure out what sticks. It’s pretty vague as far as guidelines go, but it’s the best I’ve got.

With all this in mind, and going back to the original thought ... I think it’s better to over-specify a system than to leave some things to chance, especially for a grandiose ‘lives for a hundred years’ kind of thing like Bedrock. It’s better that a spec takes up exactly as much space as it needs to do a good job, than to have a spec that fits some tightly constrained format just for the aesthetics of it (as tempting as that can be sometimes — what are we without aesthetics?).

Modular devices

The current revision of the Bedrock spec describes the port-based mechanism used to access a device, but it doesn’t go into any detail on how those devices are supposed to actually connect to the Bedrock system in the first place, physically or otherwise. Most often, a Bedrock system will be implemented as a program running on some computer, with the devices implemented as pieces of that program — in this case, it shouldn’t really concern the spec as to how the programmer hooks everything up, so long as they do a decent job of it. There are, however, a few other interesting set-ups to consider:

• The first and most obvious set-up is where you have a Bedrock system implemented in hardware (think of a circuit board packed densely with different chips). In this case, the board provides the core system (the program memory, the stacks, the processor), and a row of sockets or header pins is used to plug in the individual devices. These devices are each implemented as separate circuit boards, such as a screen device board with a small graphical LCD, or a file device with an SD card slot. We need to design a standard connector and protocol so that any hardware device can be safely connected to any hardware system.
• The second set-up is where you’re prototyping a new device in software — maybe you’re developing a file device that talks to a remote FTP server instead of a local file system? You don’t necessarily want to crack open the code of an existing Bedrock emulator and carve the device implementation directly into that emulator, because you might want to use a different language, or the structure of the emulator might be difficult to work with. You probably also don’t want to get distracted with writing a whole new emulator from scratch, because there are plenty of good ones available and you really just want to get straight to this new device. Instead, what if we can just implement the device as a separate program and then use some standard mechanism for hooking this program into an existing emulator at runtime? (thanks to @winduptoy for the general idea)
• The third set-up is a hybrid of the first two — what if you have a hardware device, such as a GPIO device board with a row of program-controllable pins and LEDs, and you want to connect it to a laptop and write a program that can, say, flash an LED whenever you receive an email. This is a pretty cool category of program, because it smooshes together high-level networking (traditionally only available on powerful computers) with low-level electronics (only really available to microcontrollers). The hardware interface from bullet point 1 can be used to connect the device board to the computer via a small USB adapter, and then the software interface from bullet point 2 can be used to shim this adapter into any existing emulator.

The general idea is that people will be able to develop programs and circuits against these interfaces, allowing hardware and software devices to be moved freely between different Bedrock systems whenever it takes your fancy, making it less painful to hook new functionality into existing systems. Bedrock already gives people the ability to write programs for any platform; these interfaces extend this idea to the system itself, allowing for modular hardware or software components that can be used on any platform.

I mean, imagine having a board for flashing microcontrollers that can be connected to anything from a laptop to a Game Boy Advance. Wouldn’t that be so cool?!

What do devices need to do?

The Bedrock architecture specification deals with devices at the port level — programs can freely read or write bytes to the 16 ports that make up a device’s interface. Each port represents some piece of functionality or information used by that device: the screen device, for example, contains ports for getting and setting the width and height of the screen and the position of the draw cursor, and for doing things like poking colours into the colour palette.

In addition to this explicit port-based interface, the system device specification gives us a second set of operations that each device should support, on top of the read and write operations required by the architecture spec:

• The system needs to be able to determine whether any particular slot has a device connected to it, so that programs will be able to tell which devices are available.
• The system needs to be able to fetch the name of any device connected to one of the final four device slots (these are reserved for custom user-provided devices).
• The system needs to be able to reset all of the devices each time the system is forked or reset.
• To improve performance or efficiency in a few rare situations, it’s useful for the system to be able to notify each device that the system has been put to sleep (marking the end of a ‘processing frame’). This is useful for implementing things like a double-buffered screen device (where the buffer flips on sleep) or for more efficient file flushing (where changes are only flushed to disk on sleep).

Virtual device bus interface

We’ll start with the virtual device interface, which is used to hook separate device programs into a Bedrock emulator at runtime.

Communication between the device and emulator is performed using any kind of bi-directional bytestream, which will probably be the standard operating system input/output streams in most cases. The Bedrock emulator could receive the path to a device program (launching it and taking control of the IO streams) or the ID of an existing process, the specifics don’t matter. The emulator can then send commands to the device process down the input stream, and receive responses via the output stream.

This interface is command-oriented and completely controlled by the emulator — the device just has to receive commands and respond to them. Each command is exactly three bytes long: the first byte contains the slot ID and port number (as per the Bedrock LDD and STD commands), the second byte is the command ID, and the third byte is additional data. The emulator sends out the command, pauses until the device sends back a one-byte response, and then resumes.

The commands are as follows:

Only eight commands are needed to cover all our bases. These are assigned to the first eight command IDs, which will be controlled by the spec. The remaining 248 command IDs are implementation-defined / reserved for the private use of each device, which could come in handy when debugging (especially once we get to the hardware devices below).

Most commands don’t actually need the port number or the additional data byte, but it keeps things simpler to send them all the time anyway. Likewise, only half of the commands actually return data (every odd-numbered command), but the return byte will also double as a ‘finished processing’ marker that tells the emulator to continue.

The design doesn’t require each device to be plugged directly into the emulator. Instead, sending the slot ID with each command allows us to have a whole separate device bus program sitting between the devices and the system, or to have a single program implementing more than one device (like a program for both the screen and input devices). The three-byte command can be broadcast to every available device, with a response coming only from the device that recognises the specific slot ID.

There’s a bit of a rough edge with the ‘check connection’ command, since a lone device can’t return anything if it doesn’t exist — this can probably be handled with a timeout, but I’m not convinced. More work needs to be done.

Hardware device bus interface

The hardware device interface builds on top of the virtual interface, but it’s going to have to take a lot of real-world complexities into account. We’ll keep the same list of commands as before (allowing for interchangeability between the interfaces), but the bytestreams will be replaced with a physical six-pin connector.

This design pretty much matches the ever-popular Serial Peripheral Interface (but not in a way that really matters for interoperability or anything, it’s just a good design). The power and ground pins provide each device with a constant 3.3V power source, and the other four pins are used for signalling.

To send a command, the Bedrock system brings the enable pin of all connected devices high, then starts clocking the command out over the output pin (24 bits as before, with the slot+port byte, the command ID, and the data byte). For each bit of the command, the output pin is set high or low, the clock pin is brought high, and then the clock pin is dropped low again. The slot ID is the first four bits, so the devices have some time to recognise whether the command is for them and to bring the input pin high to signal this.

The Bedrock system finishes clocking out the command, and then checks the input pin. If it’s high, the command has been received and is being processed. It waits until the input pin comes low, and then clocks eight bits out of the same input pin for the response. If the pin never came high to start with / no device responded to the command, this’ll just clock out the byte 0x00, which is the default value anyway.

There are a lot of unknowns to specify here:

• is the enable pin active-high or active-low?
• what is the maximum current that a device can draw?
• which range of voltages are tolerated by the signalling logic?
• what happens when one device pulls input high, and another device pulls low? (bus contention)
• how fast can a system clock the bus?
• is it better for throughput to use variable-length commands?
• should there be a standard connector as well as a pin-out?

Once again, lots to think about here. But the bones look good.

Thanks

Thanks for bearing with the long delays, it’s been hard finding the time to write. There’s no better way to work through the ideas in my head than to commit them to the page though. All the best, and see you next time!