Adobe to bring Lightroom-style photo editing to tablets

Yesterday the web lit up with variations on "Adobe to bring Lightroom-style photo editing to tablets". Originally from Scott Kelby's "The Grid", and then picked up by CNET and multiple others. The big deal being the ability to process raw images on the iPad.

Guess what guys? - it may be news that Adobe is doing this, but the technology isn't new at all. Actually, Adobe's 18 months to two years behind the curve. There are at least two apps on the App Store ready for purchase that have been doing most of what Adobe demo'd. They've been available for years now. PhotoRaw, my app, being the most popular of them.

In fact, Adobe may be more that two years behind the curve here. PhotoRaw works on a first generation iPad while, so far as I can tell, the Adobe demo needs the latest fourth  (corrected - see comment below) second generation iPad. Very different.

Maybe a bit of research rather than just parroting the press release.......just saying......

Lightroom 4.4RC and Capture One versus the X-Trans sensor

Over the past few weeks, support for Fuji's X-Trans sensor, as found in the X-Pro, X-E1 and the new X100s has gone from minimal to quite extensive. Phase One have for the the first time shipped a product with support (Capture One V7.1) and Adobe (with Lightroom 4.4RC and Adobe Camera Raw 7.4) have improved their X-Trans support. In addition, AccuRaw is close to release as a commercial product.

In previous blog posts (here, here, here, here and here) I've compared the then-current raw developer options, and discussed why demosaicing the X-Trans sensor is difficult. On various photo forums, Capture One has had a largely enthusiastic reception, while Adobe's offering has had a more mixed reception, some praising it, but other complaining of softness, blown blue channels and various artifacts.

How do they really stack up? Let's see:

Important Disclaimer: For those that don't know, AccuRaw is my product. So I'm biased.

Lightroom 4.4RC versus X-Trans

The previous blog posts have extensive detail about the older versions of Lightroom and Adobe Camera Raw's problems with the X-Trans sensor. Using the same section of the image I used in the previous blog posts, here's LR 4.4RC versus the previous version (all images processed with default settings unless otherwise specified):

The latest: Adobe Camera Raw V7.4RC, 400% crop

The old version: Adobe Camera Raw V7.1 beta, 400% crop

So, an interesting result. Compared to the previous generation, the new Adobe algorithm has much less obvious chroma smearing, so it certainly is much improved. Taking a closer look, where previously the smearing was really bright and intrusive, in the new version the smearing is a lot less bright. However, there's actually more smeared pixels - in effect, the smearing now has a wider radius. In addition, the image is noticeably softer than the previous version.

Capture One V7.1 versus X-Trans

Phase One's Capture One is a new player on the X-Trans block. Here's what it delivers:

Capture One V7.1, 400% crop

Another interesting result. Still some smearing, notably inside the letter "A", but not nearly the level seen in the Adobe results. Saturation is good, expect at the tips of the letters - e.g., look at the tips of the "F", and compare to the AccuRaw "maximum resolution" image below. Image sharpness is "interesting"; the way the image looks to me is as if what Phase One have done is to add some extra sharpening on top of an image that probably wasn't too sharp to start with.

SILKYPIX versus X-Trans

We saw SILKYPIX in the previous posts:

SILKYPIX conversion, 400% crop

Previously the best of the breed, some chroma smearing, saturation down, resolution appears slightly reduced

AccuRaw versus X-Trans

AccuRaw, now in its Release Candidate form, delivers the following:

AccuRaw RC1: Maximum resolution settings, 400% crop

AccuRaw RC1: 60% luma and chroma artifact suppression, 

20% post demosaic filtering, 400% crop

In the first crop, set for maximum resolution, AccuRaw gives very good results on the red letters, but has some artifacts. However, with AccuRaw, you can tune the result to what you want. The second crop shows moderate artifact suppression settings - still not as much chroma smearing as the other raw developers, but much reduced artifacts.

Conclusion - How do they stack up?

Firstly, Adobe's products, even in the new LR 4.4RC/ACR7.4 form, still don't stack up. Although much improved over the previous generation, they still have excessive chroma smearing relative to image resolution.  If you were to select a raw processor purely on the basis of getting the maximum out of your X-Trans based camera, Lightroom wouldn't be it.

The other products are much more evenly matched - in my view, technically they're at the same level, just making slightly different choices as to the trade-off between chroma smearing and resolution that the X-Trans sensor brings with it. AccuRaw has the advantage that you can adjust that trade-off to suit yourself and the nature of the image - e.g., for landscapes you can generally use the "maximum resolution" setting because artifacts won't show. However, AccuRaw doesn't have the features that either Capture One or SILKYPIX have. 

So really, it's a choice, and that's a big win for users, and a massive step forward from just a few weeks ago. Nine months ago, my comment on the X-Trans was that with a good raw developer, it was almost as good as a conventional sensor. And "almost as good" is actually mostly enough - practically, with the new raw developers, the difference between a conventional sensor and a X-Trans sensor is small enough to get lost in differences in lens performance, etc. There are now enough good raw developers that most users will be able to find one that works for them.

If the users are the winners here, who are the losers? Adobe certainly haven't covered themselves with glory - they have huge reserves of money and probably the best engineering talent in the business, but don't seem to have been able to apply it. Fuji is also a loser. It's ten months since I first blogged about the X-Trans processor, and so far it's delivered nothing to justify the "greater resolution than conventional sensors" hype. Finally, the really big losers are the many camera "reviewers" out there that uncritically repeated Fuji's claims about the X-Trans sensor's greater resolution. To their credit, some reviewers did raise warning flags - Sean Reid and Thom Hogan to mention two, but they were the exceptions. So next time you read a camera review, here's a suggestion - take look at what they wrote about the X-Pro when it was introduced, and judge accordingly.

X-Pro raw conversions from AccuRaw Beta 5

In a previous post, I has mentioned the existence of a "new product". Well, AccuRaw is now in a closed beta. AccuRaw isn't of course aimed at the X-Pro specifically. AccuRaw is, as its name suggests, intended to deliver technically accurate raw conversion rather than the "Hollywood colors" conversions that most current raw developers deliver by default. But one part of what AccuRaw does to to give very fined grained control over the internal operation of the demosaic process. Specifically, it has sliders that control artifact suppression in luminance and chrominance, and post-demosaic chroma filtration. So you can tune the demosaic to suit your camera, the nature of the subject, etc, rather than have the one-size-fits-all of the mainstream raw developers.

Of course, this makes AccuRaw potentially useful to owners of camera with X-Trans sensors. So here's a quick comparison showing AccuRaw vs the other guys:

ACR and SILKYPIX versus AccuRaw

Just for reference, here are the various contenders from previous posts:

Adobe Camera Raw V7.1 beta

ACR beta 7.1  - Lots of chroma smearing, and the letters are quite desaturated.

SILKYPIX conversion

SILKYPIX - best of the breed so far, some chroma smearing, saturation down, resolution appears slightly reduced

AccuRaw Beta 5: Maximum resolution settings, 400% crop

AccuRaw Beta 5: 60% luma and chroma artifact suppression, 

20% post demosaic filtering, 400% crop

In the first crop, set for maximum resolution, AccuRaw gives very good results on the red letters, but has some artifacts. However, with AccuRaw, you can tune the result to what you want. The second crop shows moderate artifact suppression settings - still nowhere near as much chroma smearing as the other raw developers, but much reduced artifacts.

PhotoRaw makes the New York Times

PhotoRaw was one of the apps mentioned in an article in the New York Time's Personal Tech column entitled "The iPad as a Hand-Held Darkroom". Sadly, they didn't provide a direct link to PhotoRaw, but hey, it's still good.

dcpTool and "V4" camera profiles

dcpTool V1.4 is out, with support for the DNG 1.4 specification, and Adobe's "V4" profiles.

When Adobe's V4 DNG Camera Profiles (DCP files) came out, I blogged about the new fields in this post. Well, the good news is that Adobe did eventually document the new fields, and now dcpTool is updated to support them.

For reference, the three new fields are:

1. Exif 0xc7a4 : ProfileLookTableEncoding, which defines whether the LookTables are in linear or sRGB encoding, 
2. Exif 0xc7a5 : BaselineExposureOffset, which allows the profile to specify an exposure offset, and
3. Exif 0xc7a6 : DefaultBlackRender, which is a "hint" to the raw converter as to whether or not to perform black level subtraction. (Black level subtraction can interfere with LookTable operation. Not that I think that a "hint" has any place in a specification, but the Adobe folks don't agree with me on that.)

Demosaicing the Fuji X-Pro1 Part 4

Well, I wasn't expecting to come back to the topic of Fuji, the X-Pro1 and its X-Trans sensor. However, I have been putting a lot of work into the suppression of artifacts when demosaicing. A lot more work than I had intended to, but that's another story. This is for a new product that I hope to release in a few weeks time (several months later than I'd hoped). But I did stumble into a better understanding of the nature of the chroma smearing (or watercolor effect, as it has also become known). The previous posts about Demosaicing the Fuji X-Pro1 are here, here and here.

In previous posts, I compared renderings from Adobe Camera Raw, SILKYPIX and Fuji's in-camera JPEG processing, as well as DCRAW and RPP. Finally, I compared those renderings to renderings from PhotoRaw, both in its "retail" configuration, and in modified form with post demosaic filtering. Practically, DCRAW and RPP were pretty much outclassed -- they use VNG algorithms that generate substantial zipper effects.

In post three, I hypothesized that the chroma smearing effect that you see very visibly in the ACR conversion, and to a lesser extent in the SILKYPIX conversion, was due to filtering, possibly mean filtering post demosaic. I now think that I was probably wrong, or at least partially wrong - the effect is due to filtering, but not mean filtering post demosaic. Rather, it's as a result of filtering during the demosaic process itself.

PhotoRaw, ACR and SILKYPIX

Just for reference, here are the various contenders from previous posts:

PhotoRaw 3.5.4, 400% crop

PhotoRaw does pretty well, but has those pesky artifacts on the paperclips.....

Adobe Camera Raw V7.1 beta

ACR beta 7.1  - Lots of chroma smearing, and the letters are quite desaturated.

SILKYPIX conversion

SILKYPIX - some chroma smearing, saturation down, resolution appears slightly lower than PhotoRaw or ACR

PhotoRaw 3.5.4 plus mean filter, 5 pass, 400% crop

The "PhotoRaw plus mean filter" showed a similar pattern of chroma smearing to SILKYPIX and ACR, but some parts didn't quite match. For both SILKYPIX and ACR, the chroma smearing is more pronounced in specific areas, e.g., in between the upper parts of the "Y", and inside the "A".

Multipass Demosaic Algorithms

As mentioned in the introduction, I've been working on artifact suppression for another product. Now the easiest way to suppress artifacts is not to generate them in the first place, so one of the things I looked at are demosaicing methods that have inherently low levels of chroma artifacts. There are a number of demosaicing algorithms that I'll call "multipass", for lack of a better term. The best known of these is LMMSE (see reference 1 below). LMMSE works, in simplified form, by first performing an initial demosaic in both vertical and horizontal directions, then filtering in the chroma domain, then using the filtered chromas to perform what is in effect a second higher quality demosaic. In this case, higher quality in the sense of fewer visible chroma errors, as the second demosaic is partially based on filtered chroma.

The results of using such a algorithm in a beta version of the new product are shown below. The first crop shows the algorithm using a filter width as it would be for a conventional two by two Bayer array. The second uses exactly the same algorithm, but with double the filter width. Note that unlike the crops above, these two are unsharpened, due to limitations of the beta.

"New Product" beta, multipass demosaic, single filter width, 400% crop, unsharpened

"New Product" beta, multipass demosaic, double filter width, 400% crop, unsharpened

Several thing stand out here (note - you may want to double-click on the crops to get a better view):

• The multipass algorithm shows fewer artifacts than the "AHD class" algorithm, even with a "normal Bayer" filter width.
• However, with a wider filter two things happen. Firstly, artifacts are reduced, as we'd expect - the X-Trans sensor has sparse colored photo sites, so its logical that we'd have to filter more heavily to get this particular algorithm to perform. Secondly, the crop starts to show the characteristic chroma smear pattern that the SILKYPIX conversion shows, as well as the slight loss of resolution and saturation that the SILKYPIX conversion shows.  In fact, the conversion starts to look uncannily like SILKYPIX, a lot more so that was the case for the mean filter I originally suspected was involved.
• Extrapolating, you can easily see how, with an ever wider filter, ACR would generate the rendering it does.

Conclusion

It's highly likely that both SILKYPIX and ACR use some form of multipass algorithm, and that the chroma smearing is as a result of the need to adapt the algorithm to the sparse colored photo sites in the X-Trans sensor. ACR appears to be using a very wide filter, probably indication the whatever methods are being used in the initial and secondary demosaic are not well suited to the X-Trans sensor. Note that it's highly unlikely that either ACR or SILKYPIX are using LMMSE in anywhere near it's text book form (neither is the "New Product" beta used above, btw). The text book form depends on using color data to improve the initial green demosaic; given the X-Trans sensors relatively large number of green photo sites and low number of red or blue photo sites, that would not work well.

References:

1. L. Zhang and X. Wu, Color demosaicking via directional linear minimum mean square estimation, IEEE Trans. on Image Processing, vol. 14, pp. 2167-2178, Dec. 2005.

Lightroom 4.3 rc and the Fuji X-Pro

Just took a look at Fuji X-Pro files using Lightroom 4.3 release candidate.

In short, same chroma smearing problem as the previous versions. Oh dear. Again(!)

Updated: Adobe now have improved support.