Solving the dreaded Gigabyte "Won't boot from USB" problem

As mentioned in this post, I just did a fresh install of Ubuntu 10.04 "Maverick Meerkat". I mentioned two fairly major problems I had with the Ubuntu installer in that post. I also had another problem, but this one isn't Ubuntu related.
I had chosen to install from a USB key, but what I had forgotten is that the motherboard in the PC is a Gigabyte motherboard, and it doesn't like booting from USB. This is a common problem with Gigabyte motherboards - just Google for Gigabyte and "boot from USB", and you'll see many variations of "can't boot from USB", "won't boot from USB", "unable to boot from USB", etc, etc. What happens is that the motherboard just completely ignores the USB key; effectively, it's like it isn't there.

Unfortunately, there isn't a solution to be found in any of those posts. Or at least I ran out of patience before I found one, although it's clear that the problem is to some extent USB drive dependent - Gigabyte motherboards seem to like some drives, others not. So rather than continue to go through endless posts that don't offer solutions, I started playing around. What I found was:

With any other drive in the machine, even if the boot from the drive was disabled in BIOS, the Gigabyte motherboard would still ignore the USB key and boot from the other drive;
With all other drives physically unplugged, boot would fail with a "Insert boot disk and press enter" message. The light on the USB key didn't even flicker at any point in the process, indicating that the USB key wasn't even being seen;
However, that was where things got interesting. What I found was that if I then unplugged the USB key, and plugged it in again, then pressed enter, the USB key was recognized, and the machine booted.

Now this is a step in the right direction. If you don't have any other bootable drives in the machine, then you can just follow that procedure. Problem is what if you do have other drives in the machine? In my case, the intention was to install Ubuntu on a dual booted drive, so having a bootable drive in the machine was required, and the there wasn't any way to get to the "Insert boot disk" prompt - the machine just booted from the drive I wanted to install Ubuntu to.

The Solution

Thinking about this, the USB key only being recognized if it was unplugged and then plugged in again suggests that somehow, if the USB key is inserted at the time the machine is powered up, the motherboard gets into a mode where it doesn't recognize the key and it gets ignored until a operating system driver starts up. But later in the boot cycle, the motherboard does seem to be able to recognize it. So what I did was simple:

I started the machine with the USB key unplugged;
Once the machine had got through the first parts of its boot sequence, I plugged the key in, then hit the F12 key to bring up the boot menu. And on the third try, there the USB key was, on the list of hard drives. And it booted fine!

Two thing to note here - what I found worked, for me anyway, was to plug the USB key in just after the BIOS displayed the list of PCI devices, and just as it gave its "Verifying DMI pool" message. Secondly, note that the USB key appears on the list of hard drives.

Now I must admit, plugging the USB key in during the boot sequence is neither elegant or reliable - you'll probably have to try it a few times - but it worked for me.

Good news in a difficult year - all the macOS apps that I support - AccuRaw EXR, AccuRaw Monochrome, pcdMagic, CornerFix, dcpTool (both the GUI and command line version), DNG cleaner and pcdtojpeg - now are all available with native Apple Silicon versions for blazing fast performance on Apple's new "M1" processor. Download are in all the usual places.

Enjoy.

There's a whole slew of new camera support, including for Canon's new CR3 format.

Updates of AccuRaw EXR, AccuRaw Monochrome, PhotoRaw are now available on the Apple App Store. These updates add support for these new cameras: Canon EOS-1D X Mark III, Canon EOS 90D, Canon EOS M50, Canon EOS M6 Mark II, Canon EOS Rebel SL3 (EOS 250D), Canon EOS R, Canon EOS RP, Canon EOS R5, Canon EOS R6, Canon PowerShot G5 X Mark II, Fujifilm XF10, Fujifilm GFX 50R, Fujifilm GFX 100, Fujifilm X-100V, Fujifilm X-A7, Fujifilm X-Pro3, Fujifilm X-T30, Fujifilm X-T4, Fujifilm X-T200, Leica Q2, Leica SL2, Nikon Coolpix P950, Nikon Coolpix P1000, Nikon D780, Nikon Z 6, Nikon Z 7, Nikon Z50, Panasonic DC-G90, Panasonic DC-G100, Panasonic DC-G110, Panasonic Lumix DC-G95, Panasonic DC-LX100 M2, Panasonic DC-S1, Panasonic DC-S1R, Panasonic DC-S5, Panasonic Lumix DC-FZ1000 II, Olympus OM-D E-M1X, Olympus OM-D E-M1 Mark III, Olympus E-PL10, Olympus OM-D E-M5, Olympus OM-D E-M10 Mark IV, Olympus TG-6, Pentax K1 II, Ricoh GR III, Sony DSC-RX100 VII, Sony A7 III (ILCE-7M3), Sony A7C (ILCE-7C), Sony A7R Mark IV (ILCE-7RM4), Sony A7S III (ILCE-7SM3), Sony a6100 (ILCE-6100), Sony a6400 (ILCE-6400), Sony a6600 (ILCE-6600),Sony HX99.

pcdMagic for Windows - the only currently available app that can convert Kodak Photo CD images with correct color and at full resolution - is now available on the Microsoft Windows Store.

This great news for users:

The Windows Store version has a free trial mode that allows the app to be tested without any commitment.
The Windows store handles all updates automatically.
There's no need to keep track of license codes.

The Windows store only supports the latest versions of Windows 10. However, for users of earlier versions of Windows, you can still purchase pcdMagic from the FastSpring webstore.

As usual when new Leica cameras come out, I took a quick look inside a DNG from one of Leica's new Leica CL cameras:

The camera name shows as "LEICA CL"
The image data is 14-bit. There is no compression used in the DNG I looked at.
The DNG version is 1.4, with a "backward version" of 1.3. There is a reason for this - DNG 1.3 allows for opcodes, which Leica use for lens correction.
In the DNG I looked at, which was shot with a "Summicron TL 1:2 23 ASPH." lens, lens correction is done by a single "WarpRectilinear" operation in the DNG.
In addition to the lens correction op code, there is also a "FixBadPixelsConstant" opcode, whose function is exactly as the name states. This is the same as in the Leica Q, SL, etc.
Unusually, the DNGs all contain 2 different JPEG preview images in addition to the main raw image, one of 1620x1080, and finally a full sized preview of 6000x4000. Having the full sized preview is particularly odd, as it takes up a lot of space. In the approximately 45 MB files I've seen, the full size preview typically takes up about 1.8 MB.

So in summary, the DNGs appear to be a bit of a hybrid of the "SL", "Q", etc style of DNG, with a single lens correction opcode, and a bad pixels correction opcode, and of the "M10" style of DNG, with a full sized preview. As I noted in my analysis of the M10 DNG, the full size preview is probably there to support viewing on mobile devices that often don't have raw decoding capability built in.

Finally, it's notable that the DNGs don't contain the bizarre Lightroom XMP metadata that the M10 DNGs have embedded in them.

So finally, after many years of searching, I have an answer to the question that torments all who go down the digital color rabbit hole.

This is from XKCD, brought to my attention via an article on the Digital Transitions website about the Phase One IQ3 100mp Trichromatic.

Back in January, when the new Leica M10 was introduced, there were claims that the improvement in dynamic range from the Leica M240 to the M10 was of the order of 1.5 to 2 stops. At the time, I wrote that just by eyeballing the published images, I believed the improvement to be "closer to 0.5 stops than 1.5-2".

Much to my surprise, given what I had thought to be just a basic explanation of why dynamic range is a tricky concept, the post generated a lot of push-back. And I mean a LOT.

Well, now there is actually an independent third-party measurement available, from photonstophotos.net:

And....(drum roll)....the improvement in DR between the M240 and the M10 is 0.55 stops. As I predicted back in January.

You can a find whole lot more information, including a useful interactive graphing tool that allows you to compare cameras, on the photonstophotos.net site.

dcpTool has been available for quite a while as a command line application for Windows and the Mac. But now it available in the form of an easy to use Mac app, with powerful batch processing capabilities. dcpTool for the Mac is available from the App Store.

dcpTool allows you to:

Decompile DNG Camera Profile (DCP) files in XML. The XML can then be read and edited with a simple text editor.
Recompile edited XML into DCP files
Remove "Hue twists" from camera profiles.

For more information on hue twists, see these posts:

Many cameras embed lens corrections into raw the raw files that they produce. Generally, that's a good thing - straight lines stay straight, etc. For an example of lens corrections in practice, take a look at this post about the Leica SL.

But, as the saying goes, "there is no free lunch". Lens corrections also have some downsides:

Lens corrections result in a small reduction in sharpness. Sean Reid, at Reid Reviews has done extensive testing on this as part of his various lens reviews, and his conclusions are clear - there is a measurable loss in sharpness.
Lens corrections result in some reduction in image size - the corrections inevitably result in the edges of the image curving, and the curved parts need to be trimmed off to get back to a straight edge.

So sometimes, it's useful to be able to get an uncorrected image. Now there are some raw developer apps that allow you get to uncorrected images (AccuRaw EXR is one of them), but most mainstream apps such as Lightroom and Photoshop don't allow correction to be disabled.

Fortunately though, there is a "simple trick" that will allow you to disable lens corrections for nearly any camera that produces raw images. What you need to do is a simple two step process:

Firstly, convert the raw image to a DNG image, using Adobe's DNG converter.
Then use DNG Cleaner (macOS only) to remove any opcodes.

The resulting DNG will not have any lens correction, and you can load it into Lightroom, Photoshop, or any other app that supports DNGs.

Why does this work? This works because whenever DNG Converter converts an image that needs lens correction, it embeds the required correction as an "opcode" into the DNG image. Lens correction opcodes as usually things like "WarpRectilinear". DNG Cleaner knows where to find these opcodes, and simply removes them, as long as you have the "Remove opcodes" checkbox selected:

Note that in order for this to work, you need DNG converter to be using it's default settings. Specifically, Compatibility should be set to the most recent version of Camera Raw. You should definitely not be using Custom Settings with the "Linear" checkbox selected. "Linear", otherwise known as LinearRaw, bakes any corrections into the actual data in the DNG, making it impossible to remove.

There's more information on the DNG Cleaner website.

Those of you that have read the Leica M10 raw file analysis post will know that M10 DNGs have more baggage in them that is typically the case for a Leica DNG. I've put together a little app to clean them up, called DNG Cleaner (Mac only for the moment). For M10 DNGs this will remove:

The full size image preview - that will save about 3MB
The various Lightroom/Photoshop adjustments embedded in the XMP portion of the DNG - ISO dependent noise reduction settings, lens profiles, etc

DNG cleaner will also optionally remove opcodes, and apply lossless compression. This isn't relevant to the M10, but in many other Leica cameras, e.g., the Q, the SL, etc, opcodes are used for lens correction. For an example of the SL's with and without lens correction, see my post on the subject. DNG Cleaner is what Sean Reid used to remove opcodes for his SL 50 review, which some of you will have seen.

As ever, back up your files before using any app that is designed to modify them.

For more information on DNG Cleaner, see the website.

AccuRaw users might have gotten a bit of a surprise recently - AccuRaw has turned into AccuRaw EXR, and is now at version 3.

The reason for the name change is that the focus of the AccuRaw product has changed a bit, based on what users were actually doing with it. AccuRaw has always been focussed on having highly linear color response, with no "hue twists" or other surprises in the color rendering. As it turns out, that is a very useful feature to have if you're composting stills into video - it reduces the amount of work required for color matching.

In the world of video, especially in the professional cinematography world, the "gold standard" is to use a format such as EXR, which is a floating point format. That allows huge flexibility in the way that post processing is done. So the new version of AccuRaw, now named AccuRaw EXR, supports EXR output.

However, it's more than just a name change. In order to really support EXR output, AccuRaw has seen major changes internally - it now has a fully floating point, non-clipping workflow. For more on why that is important, see this post.

Solving the dreaded Gigabyte "Won't boot from USB" problem

View comments

ChromaSoft

Digital imaging and building the software that makes digital imaging happen

All apps now natively support Apple Silicon

Add a comment

Major updates for AccuRaw EXR, AccuRaw Monochrome and PhotoRaw

Add a comment

pcdMagic for Windows in now available on the Windows Store

View comments

Leica CL raw file (DNG) analysis

View comments

What is color?

Add a comment

The Leica M10 and dynamic range revisited

Add a comment

dcpTool for the Mac is now available

View comments

How to remove the lens corrections from any raw image

Add a comment

Cleaning Leica M10 DNGs

Add a comment

AccuRaw has become AccuRaw EXR

View comments