(Rather Late) Surprise: Intel has "PCI Expansion ROM OS" Implementation for Computer Cluster

For some reason I stumbled upon one of Intel's patent application: Cluster computing - nic based os provision WO 2012040606 A2. The patent application cited my BUILDING A ''KERNEL'' IN PCI EXPANSION ROM article as non-patent citation.

I had that thought back then, that some companies must've built (or will build) something usable out of the principle explained there. I'm actually rather surprised why it didn't happen earlier. Of course, we have etherboot and PXE-boot eons ago. However, there's no such complete OS, even though a very small one in PCI Expansion ROM. Well, perhaps one of the limitation is the cost of the ROM.

Anyway, the use case explained in the patent application is interesting nonetheless. Head over to the patent application to see more details. It's a good read ;)

Migrating AMIBIOS 1B Module Utilities Source Code to GitHub

A long time ago (read: 10+ years) I created some small, mostly BIOS-related, utilities and dumped the source code on the web. I even forgot that they're there in my website (https://sites.google.com/site/pinczakko/source-code). Fortunately, over time, some people come across those source code and found the source code useful for them. Some of them asked question or give suggestions. Therefore, I decided to gradually move the code over to GitHub and update the code as well, so that the code would still be compile-able with present day toolchain.

Some of the utilities are small utilities to parse, extract and modifies AMIBIOS8 1B (main) module--akin to original.tmp in (legacy) Award BIOS. You need to use AMI Module Management Tool (MMTool) to obtain the 1B module from AMI BIOS binary file.

The utilities source code are now in Github: https://github.com/pinczakko/AMIBIOS8_1B_Utils. The code is POSIX compliant. I haven't test it against Mingw-w64 cross compiler to produce Windows binary--I recall that I used mingw (32-bit) on Windows to write the code back then. The code compiled just fine in Arch Linux gcc toolchain as of July 2017. Therefore, the code can produce Linux/Unix executable just fine.

I don't really remember the exact time when I wrote the utilities. But, it was wrtitten in 2010 and used in one of my project(s) in that year for one of my client(s). Those projects requires AMIBIOS8 binary modification. That's the reason I wrote the utility.

For the time being, feel free to play with it and modify it. I plan to add support for cross compilation with Mingw-w64 later on. If you tested the code on real AMIBIOS8 binary, I'd like to know the result ;-).

Anyway, the code is GPL v2-licensed (kudos to Gabor Stefanik for reminding me to put proper licensing note).

Experimental PCI Expansion ROM "OS" Code Migrated to GitHub

The code for the experimental PCI Expansion ROM "OS" explained in the Building a "Kernel" in PCI Expansion ROM article is now in GitHub: https://github.com/pinczakko/PCI-Expansion-ROM-OS. I made some changes to make it compile-able in current version of Nasm and GCC. I've only tested the compilation in Arch Linux (x86-64). I'm not sure it will work in other Linux distros. Give it a try ;-). Quick skim over the resulting binary seems to indicate the result is OK. I'm going to check it with a disassembler later on. If anyone wants to help me with that, please do so and post your result in the comment section below. 

Many of you might be aware that the code has been modified into pure GCC-only code in the Low Cost Embedded x86 Teaching Tool article. I need to migrate that code as well. But, I'm quite sure it will require special GCC version to be able to emit the correct binary, akin to the one used by Coreboot. I'll post an update once I've updated that one as well. 

Anyway, it's rather surprising to me that using Nasm + GCC is more future-proof compared to using GCC alone. It shows that you can't be really sure about the future-proof-ness of the toolset you used for software development.

IBM OpenPower Firmware Source Code Brief Analysis

First post this year ;-)

I'm taking a detour to other hardware architecture here, despite this blog is focused on x86/x86-64. As for why, it's because I was working with IBM Power 5 machine for more than a year and I found it interesting. I'm not going to talk about Power 5 here though because it's a closed system, in terms of firmware. I'm here to talk about Power 8 and its successor.

The Power 8 architecture is the first incarnation of OpenPower hardware architecture. Luckily, the firmware source code for this platform is freely available in Github: https://github.com/open-power.

Now, let's look at how we might read the code:

• The most interesting part is the Initial Program Loader (IPL) firmware: https://github.com/open-power/hostboot. This is basically the equivalent of UEFI Platform Init (PI) code, or the bulk of the BIOS code in the old days--excluding the runtime code, such as power management and SMI handlers.
• The next interesting part is skiboot: https://github.com/open-power/skiboot. This looks like the equivalent of the non-PI part of UEFI because it provides the interface that the OS can call at runtime to communicate with the platform firmware. I might be wrong tough, but all code I skimmed over indicates that.
• Last but not least is petitboot: https://github.com/open-power/petitboot. This is basically an analog of GRUB or Systemd bootloader in x86/x86-64 Linux.

The important thing that's missing from the OpenPower Github is the Baseboard Management Controller (BMC) source code. I'm not sure why IBM doesn't standardize that part as well or at least provide a reference implementation. Perhaps, it's because IBM wants to provide a point for differentiation among its partners. Also, if we think about it, the BMC code is probably the most vulnerable point should some one wants to attack an OpenPower environment because it usually provides a remote access to manage the server even before it finished booting.

I'll update this post once I've read the IPL code in more detailed manner. Hint: if you want to read it as well, try starting with the linker script (*.ld) file.

BIOS Disassembly Ninjutsu Uncovered on Play Store

Somebody has just put BIOS Disassembly Ninjutsu Uncovered "scanlation" on Play Store. Well, it's not really manga "scanlation" quality. But, I'm rather surprised someone put the effort to do that: https://play.google.com/store/apps/details?id=com.appjik.book.bios. You might want to give it a try. 

In my opinion, the PDF version in github is much more readable. Perhaps, I should make it available on Play Store, or is there anyone want to volunteer to do that? Cheers.

Down to Silicon Level Debugging

First off, I'm not a "forward" BIOS/UEFI engineer. At least not one who worked officially in a BIOS/UEFI software development company or motherboard company. I did got an official access to AMIBIOS Core8 source code and tools back then under NDA for one of my clients to customize it for a custom x86 motherboard. But, that's as far as I got into the game. This is relevant to this post as I don't know exactly the process of silicon level firmware development validation. The farthest I went was a sort of "Serial ICE" with Coreboot and also debugging via Power-On Self Test (POST) code passed over serial port in AMIBIOS Core8. That was it. I didn't even manage to do debugging via PCI POST card, but I presume it would be similar to its "redirected" cousin in the serial port. There is another way to do firmware debugging, via JTAG pins, and by using In-Circuit Emulator (ICE). Despite having been an Electrical Engineering student, I'm not yet familiar with those territories. But, AFAIK both are as good as if not more powerful firmware debugging technique compared to using Serial ICE or POST card.

Let's focus on the ICE part. There was at least one mistake I did in my BIOS book that I didn't realize due to my handicap in not having an ICE and its related skills. You can see it in the quoted errata below (it's also in the addendum part of my book over at github):

The address aliasing mentioned in Chapter 4 section 4.1.1 page 4 (the paging messed-up in the PDF) should cover both E-segment and F-Segment (E_0000h-F_FFFFh), not just the last 64-KB segment. Somebody used a sort of CPU logic analyzer to confirm this fact.

The guy who tipped me over about this was using an expensive ICE to validate the fact above. I'm not exactly sure how he tapped all of the "wires" on the chipsets and the CPU itself, but very probably similar in principle to what "Bunnie" did to the first XBox version (see: http://www.xenatera.com/bunnie/proj/anatak/xboxmod.html). IIRC, the was using one of Arium ICE products. Arium was acquired by another company, see: http://www.asset-intertech.com. However, their ICE products live on as (very probably) the ScanWorks and SourcePoint line of products from asset-intertech. These ICE products used to cost north of $20k a piece back then. I don't know about it at the moment though. With an ICE, you essentially put the CPU in a "hard" debug mode, where you can freeze it in a way ordinary debugger cannot because there is no OS or firmware required for that to happen.

Anyway, I was quite surprised to find a "low cost" version of this kind of ICE over at: http://www.loper-os.org/?p=1667. Well, I'd like to thank to whoever posted a comment about this ICE in my previous post. It's very interesting nonetheless ;-).

Anyway, for the uninitiated, a (not so) useful background is at https://en.wikipedia.org/wiki/In-circuit_emulation

Base-board Management Controller (BMC) Firmware Security

The security of the BMC firmware is very important, as compromising it means unfettered remote access to the target machine. There has been research into this area in the not too distant past:

• Exploiting Hardware Management Subsystem, by Simon Clow.
• Reversing Firmware using Radare2, by Anton Kochkov.
• IPMI: Understanding Your Server's Remote Backdoor, by Anthony J. Bonkoski.

All of them are interesting in their own right. Perhaps, Bonkoski's one is the most comprehensive? I don't know. I haven't dig into all of the papers myself.

Anyway, one of the most interesting development in BMC is OpenBMC, see: https://github.com/facebook/openbmc and https://code.facebook.com/posts/1601610310055392/introducing-openbmc-an-open-software-framework-for-next-generation-system-management/. Is it going to grant you access to Facebook-class infrastructure (from afar) if you find a flaw in it? Well, I don't think so, as it must've been protected by giant "firewall" in front of it. But, doing a code review on OpenBMC for flaws certainly a good exercise.

As a side note, let's not forget about Fujitsu, one of the most "underrated" server producer on the market. As a parting gift, let's look at what Fujitsu has in store in its BMC:

Fujitsu integrated Remote Management Controller TCP/UDP ports

Firmware/BIOS-related Patent Filings

I don't know if security researchers are used to looking at patent filings--because I'm not officially one of them. However, I found that reading and trying to understand firmware/BIOS-related patent filings is enlightening. It is also interesting because the filings are related to each other via cross-referencing, which make the activity all the more interesting, given enough time to dig into it. Among other thing, it provides a view ahead in this cat and mouse game of protecting and breaking firmware. These are some of my picks (not necessarily new ones):

• Patent 2008/7392541
• Approaches for Ensuring Data Security
• A sort of BIOS Agent
• Pre-Boot Authentication System
• Sector-Based Protection to Flash ROM (Old one)
• HP Lights-Out-Management Patent Filing

The one that interest me the most is the last one because it's a sort of insight into Baseboard Management Controller (BMC) stuff. I hope you enjoy the patent filings as much as I do ;-)

UEFI Boot from Web

I think I've been living under a rock in these last few months and not exactly following UEFI development. Nonetheless, I managed to spot this stuff over at https://firmware.intel.com/develop/server-development-kit. What's interesting is the SDK supports "Firmware Boot from Web" so to speak. This is the relevant excerpt:

The Intel® Server Board S1200RP UEFI Development Kit supports Pre-Execution Environment (PXE) boot for IPV4 and IPV6 networking using on-board and add-in networking devices. Because of added initialization time, network boot for the four onboard networking devices is disabled by default in firmware setup. Users can enable PXE boot for on-board networking by enabling the ’EFI Network’ setting in the firmware setup menu.

EDKII Menu -> Advanced -> Network Configuration

As of SDV.RP.B6, the Intel® Server Board S1200RP UEFI Development Kit supports UEFI HTTP and HTTPS boot. These features are described in whitepapers located on the Tianocore github wiki: https://github.com/tianocore/tianocore.github.io/wiki/EDK%20II%20White%20papers.

Well, it could be double-edged sword from security standpoint. It depends on who you ask and what it's being used for.

Anyway, this is my take on this:

• My "educated" guess is: This stuff emerge from Intel collaboration with the so-called Hyperscalers--Hyperscalers is what some people call them (http://www.nextplatform.com/category/hyperscale/). The Hyperscalers (Google, Facebook, Amazon, Azure, Alibaba, Baidu, Tencent) are running lots of web servers. Therefore, it makes sense for them to make it possible for their machines to boot just off of the webservers instead of preparing another "PXE Boot server" due to the prevalent web server in their bit barns. I think that Intel wants to trickle-down the same stuff into the masses, but as the first step, the Enterprise sector is what Intel targets.
• Present day flash (is it still Flash??) memory used for firmware storage is spacious enough to cram a (compressed?) HTTP/HTTPS client into it. It would hardly possible to do that just several years ago due to the space constraint in the firmware chip on the motherboard. I didn't say this is impossible years ago because I have worked with extremely limited firmware space in non-PC platform that somehow managed to cram HTTP server into space less than 1MB, along with hardware configuration software stuff. 
• This made it possible to boot from the cloud if anyone wants to implement such a stuff. But, it would entails a huge security "nightmare" if you ask me.

BIOS Disassembly Ninjutsu PDF Moved to GitHub

The primary download site for BIOS Disassembly Ninjutsu PDF (free) is now moved to https://github.com/pinczakko/BIOS-Disassembly-Ninjutsu-Uncovered  (direct download at https://github.com/pinczakko/BIOS-Disassembly-Ninjutsu-Uncovered/archive/master.zip). The previous download at 4shared is a malware-invested place, thus the change.

The addendum to the book is also included in the GitHub repository.

Moving Winflashrom code to Github

I ported Coreboot (formerly LinuxBIOS) flashrom utility to Windows a long time ago as my activity in Google Summer of Code and named it winflashrom. Because code.google.com will be shutdown this year, I moved the code to github: https://github.com/pinczakko/winflashrom.

This is old news because the code haven't been updated for years. However, it might still relevant for those who want to port flashrom or other similar utility to present day Windows. I haven't developed Windows driver anymore since Windows Server 2003. I'm not even sure if WDM-style driver is still in use in Windows. But, I might be returning to develop Windows driver this year. So, yeah, you (and I) never know.

Looking into The State of Firmware Security in Russia

I think every major industrialized country has its own policies in preventing malicious IT equipment and products to enter their premises, let alone being used within the country. In this post, we will look into one of Russian computer hardware maker, Kraftway (http://www.kraftway.ru/en/). This company might be a bit obscure to you. But, I think it serves quite a big chunk of the Russian and possibly CIS market. It was even visited by Dmitry Medvedev when he was still President.
This company is interesting for two things:

• It is not just a "box" mover. It tailors the machines it made to meet the customer requirements. Among its in-house expertise is custom firmware, including UEFI firmware. If you look at this page: http://www.kraftway.ru/en/products-and-solutions/, at the end of it, you can see that it has in-house expertise to work on UEFI security modules and Trusted BIOS (whatever that might imply). Another thing that catches my attention is this: 

In 2010 the company signed an agreement with a telecom giant Cisco establishing a special procedure for the certification of Cisco products in Obninsk manufacturing facilities. Kraftway ensures that Cisco products comply with the requirement of the Federal Technical and Export Control Service on information security. Such certified products can be used in systems processing sensitive or confidential information. In 2012 Kraftway launched the production of Fujitsu PCs with a trusted BIOS and all-in-one PCs based on the Russian processor Elbrus.

I'm not so sure what does the statement meant by "requirements". Perhaps, it includes firmware-level compliance of some sort. You can look at the whole thing over here. 

• The second thing is Kraftway also made PC based on the Russian homegrown Elbrus CPU (http://www.kraftway.ru/en/about/milestones/). Of course, in the process, it creates the firmware alongside experts from MCST. The premise for using Elbrus CPU is national security needs and "sensitive" computing needs. So, it's understandable. 

Well, I recall that Dell also did the very same thing as Kraftway with respect to firmware and hardware customization. Dell puts crypto-stuff in the firmware even before UEFI hits the market for some of its server product. Perhaps, that's not meant to be used by the masses, only certain customers.

Anyway, scrutinizing the firmware code or creating a custom ones is highly logical for "sensitive" (high-security) computing gear. Every major developed country do that. IIRC, Germany has its own Coreboot Laptop for that kind of purpose. Even China and Taiwan is doing that as well, albeit I haven't yet found writings on that.

The State of My Firmware Research

Well, I decided to post this because I've been over-promising and under-delivering for several years now.

Straight to the matter, I've been leaving my firmware research work in a state of hibernation for almost a year now due to a (some?)  product development work I'm still working on as of now (which I cannot elaborate further). It's not that I feel firmware is not interesting anymore. On the contrary, I feel it's far more interesting now than it used to be due to the raise of connected embedded systems (now re-badged as Internet of Things a.k.a "rather intelligent" data collection systems). The main problem for me is finding time to work on this research again as it's unfortunately not my day job.

As for my work on the continuation of my BIOS Disassembly book project. I will try to find time for that, but I don't want to over-promise on it. Hopefully this clears things up. 

Remote Access in Legacy BIOS

In this post I'm going to talk about Remote Access in Legacy BIOS via serial console. I aware some (or most) of you are aware that BIOS has provided management console via serial port for a long time. I have the opportunity to modify a customer custom Geode board BIOS to add support for Serial Console a few years ago. It's a quite nifty but rather buggy implementation though (I meant the serial console module). This one is from AMIBIOS Core8. This is the screen shot from minicom in Arch Linux.

As you can see, the terminal looks like how you would expect it when accessed via real keyboard. Unfortunately, some function keys are not working as expected. You can configure the serial port just like you'd expect on old BIOS with serial port support, i.e. the BAUD rate, flow control, bit-ness (8-bit), etc.. The Remote Access menu in the picture is where one would configure the serial port setting for the remote access (serial console).

I'm "dusting-off" this old board from storage because it's quite a nice board to tinker with. I almost forgot that it has remote BIOS access feature back then. Basically, it works like Linux serial terminal in most embedded Linux boards out there. But, this one is limited. I think many enterprise-class motherboard has this feature back in the day and also today because it's a very crucial feature for remote manageability especially if you have thousands of machine to work with. Keep watching this one guys ;-). It's gonna be interesting..

How Boot Firmware Development and Driver Development Differs--PCI Bus Implementation Case Study

This post is not BIOS/UEFI specific per-se. However, it has a very close relation to it because it delves deep into Windows device driver architecture.

Most of BIOS/UEFI modules are aware of the CPU architecture, motherboard chipset and all supporting logic in which it runs. However, the same assumption cannot be made for an OS, such as Windows. Therefore, BIOS/UEFI modules mostly can take for granted the CPU and bus architecture in which it will run. The same is not true for a device driver. For example, a PCI or PCIe explansion card can be used in the same operating system but runs on entirely different CPU architecture. This means device driver creator couldn't and shouldn't assume the CPU architecture and bus architecture in which it will eventually run.

This series of posts by Windows PnP subsystem developer is very enlightening in this respect:
http://blogs.msdn.com/b/doronh/archive/2010/05/05/arbitration-and-translation-part-1.aspx
http://blogs.msdn.com/b/doronh/archive/2010/05/06/translation-and-windows.aspx
http://blogs.msdn.com/b/doronh/archive/2010/05/06/arbitration-and-translation-part-3.aspx

This hopefully shed some light in system software developments :)

(Cross) Compiling My Sample PCI Expansion ROM Code

My sample PCI Expansion ROM code over at Low Cost Embedded x86 Teaching Tool is no longer compile-able on recent x64 Linux distributions. This is due to the fact that the default GCC toolchain in those Linux distros doesn't support output in the form of the particular ELF32 i386 required by the source code. Other possible problem is the GCC toolchain doesn't support 16-bit code output anymore, as required by the linker script.

For trouble free source code compilation, you're advised to download and build Coreboot cross compiler, see: http://www.coreboot.org/Build_HOWTO#Known_issues. Just invoke:

make crossgcc

in the directory where you download Coreboot (Coreboot root directory) to build the cross compiler. You'll also need to alter the Makefile of the PCI Expansion ROM code to use the newly built cross compiler. The toolchain components that you need to change in the Makefile is pretty much everyone of them, i.e: CC, LD, AS and OBJCOPY. The cross compiler should be located in the /util/crossgcc/xgcc/bin directory, once you have completed building the cross compiler. If you are still unsure about the path of the cross compiler toolchain, open the .xcompile file in Coreboot root directory.

Gentle Introduction to Coreboot

I've just stumbled upon this series of articles by  Lennart Benschop about Coreboot. It's a very smooth introduction to Coreboot that explains Coreboot components from a high level view and proceed toward more specific parts of it. You can read it over here. Note that Coreboot specific stuff starts at "Column" no.4. Have a nice reading :-)

NSA BIOS Backdoor Article Part 2: BULLDOZER is up

You can read the complete article at: NSA Backdoor Part 2, BULLDOZER: And, Learn How to DIY a NSA Hardware Implant

This is the excerpt:
This article is the second part of a series on NSA BIOS Backdoor internals. This part focuses on BULLDOZER, a hardware implant acting as malware dropper and wireless communication “hub” for NSA covert operations.

NSA BIOS Backdoor Article Part 1

OK. This is part one of my controversial article series of the year :P

NSA BIOS Backdoor a.k.a. God Mode Malware Part 1: DEITYBOUNCE
This first part focuses on the DEITYBOUNCE malware described in the NSA ANT Server document.
I won't spoil the details here. Head to the link above for the details :-).

System Address Map Initialization Part 2 Article

Finally, System Address Map Initialization Part 2 article is up. Check out :  http://resources.infosecinstitute.com/system-address-map-initialization-x86x64-architecture-part-2-pci-express-based-systems/. It's a lengthy article. There could be minor error coz I didn't have enough time to review some parts of the detail close to the end of it.