Olympus just announced the E-M5 Mark II, an updated version of its popular micro Four Thirds E-M5 model, with an interesting new feature: its 16MegaPixel sensor, presumably similar to the one in other E-Mx bodies, has a high resolution mode where it gets shifted around by the image stabilization servos during exposure to capture, as they say in their press release
‘resolution that goes beyond full-frame DSLR cameras. 8 images are captured with 16-megapixel image information while moving the sensor by 0.5 pixel steps between each shot. The data from the 8 shots are then combined to produce a single, super-high resolution image, equivalent to the one captured with a 40-megapixel image sensor.’
A great idea that could give a welcome boost to the ‘sharpness’ of this handy system. This preliminary test shows that the E-M5 mk II 64MP High-Res mode gives in this case a 10-12% advantage in MTF50 linear spatial resolution compared to the Standard Shot 16MP mode. Plus it apparently virtually eliminates the possibility of aliasing and moiré. Great stuff, Olympus.
Shake it Baby
Moving the sensor around to good effect is not new. The Pentax K-3 for instance uses its in-camera image stabilizer to jiggle the sensor during exposure in order to provide anti aliasing action, therefore giving the photographer the ability to turn the AA on or off at will depending on the scene, trading off aliasing for resolution. The difference in the E-M5mkII implementation is most likely additional control and half-pixel precision in sensor positioning, resulting in AA action AND additional resolution. Clever, and the first time in a DSC I believe.
Since in High-Res Shot mode spatial information is captured at every half (original 16MP) pixel position the resulting raw file represents twice as many pixels horizontally and vertically, resulting in a 64MP file. In order to accomplish this feat 8 images are captured and assembled sequentially. It is therefore imperative that the subject be static, shot with good technique and with a totally vibration-free setup.
How does the 64MP HR Shot mode work compared to the 16MP Standard, all other things including f-number equal? It works well, in this case apparently adding 10-12% to MTF50 grayscale linear measurements at f/5.6 – at the cost of a 1 second or so capture, 3 second delay between captures and almost 8 times the raw file size: 103MB vs 14MB at base ISO.
It turns out that somewhat dated dcraw 9.22 is able to open the High-Res .ORF raw files in grayscale document mode, so these results are based on the raw, unprocessed data provided by the camera according to the procedure described here. Hopefully ol’ dcraw opens them properly with the -d -4 -T -w switches, if not ignore the rest of the post. [Edit: I get similar results by opening the HR raw files with the latest version of RawDigger which does support them, so we are most likely good to go.]
One High-Res EM5II raw image is twice the linear size of one captured in Standard Shot mode: 6938×9280 pixels, that’s 64MP, weighing in at 103MB at base ISO – versus 3472*4640 and 14MB or so, the same 16MP resolution as earlier E-Mx models.
10-12% MTF50 Advantage
Open Source MTF Mapper produced the following MTF50 results from the slanted edges in the center of DPR’s New Studio Scene raw files at f/5.6. The E-M5II is wearing a Zuiko 45mm:1.8 at f/5.6. A Sony a6000+55:1.8ZA (APS-C) and Nikon D810+85:1.8G (Full Frame) were thrown in for reference only, both also at f/5.6:
The blue curves show the relative performance of the Olympus E-M5 II in its 64MP High-Res Shot mode (dotted line) and 16MP Standard Shot mode (solid line). The right side of the diagram shows results from DPR’s ‘daylight’ Studio Scene raw files, the left side from the ‘Low light’ raw files. The first capture of each Low light or Daylight series is at base ISO, the last one of each series is at ISO 1600 (MTF curves start becoming unreliable at the noise level of this lofty ISO).
DPR’s E-M5 mark II’s ‘Daylight’ shots appear to suffer from some form of vibration. As mentioned in earlier articles shutter speeds in the 1/10 to 1/320s range are sometimes susceptible to vibration because of the multitude of small masses and mechanisms that start and stop motion to accomplish a capture. It could also be poor technique, or simply bad luck (heavy traffic nearby for instance) – spatial resolution measurements are famously finicky.
But the first three EM5II ‘Low Light’ shots with shutter speeds slower than 1/10s I think give us an idea of how well the new High-Res feature performs. In Standard 16MP shot mode, with it off, the E-M5II achieves on average MTF50 readings of around 900 lp/ph in the center of the frame, as would be expected of an AA-less sensor of this pixel size; in 64MP High Res Shot mode, about 1000 lp/ph. That’s quite an improvement, perhaps in this preliminary test not quite generically ‘beyond full-frame DSLR cameras’ as the marketing talk goes – but it’s an impressive, perceivable change nonetheless, besting the 16MP D4+85mm:1.8G at f/5.6 for example. Bravo Olympus.
Keep in mind that with decent home technique and the same exact setup, results should be repeatable within about +/-5% from capture to capture. And that although a 5% change in MTF50 readings may be hard to spot by eye in two images set side by side, 10% is definitely noticeable.
Excellent Bonus: Reduced Aliasing and Moiré
Perhaps more important than the MTF50 resolution improvement, though, is the positive effect of the High-Res Shot on aliasing and moiré: it looks like it minimizes frequencies above Nyquist, effectively eliminating both. You get better resolution AND no aliasing. That’s brilliant if you need to shoot non-movable man-made subjects. This is how the MTF curves from the raw green channels alone look like according to MTF Mapper at base ISO:
Note how in Standard 16MP Shot mode the Modulation Transfer Function (black line) crosses Nyquist’s frequency with a lot of energy still, indicating that the sensor has a weak or (most likely) non existent Anti Aliasing filter. All that energy past Nyquist can and does show up in images under the guise of aliasing and moiré.
But in the 64MP High Res Shot the MTF curve (green line) stays well away from Nyquist, virtually leaving no spatial frequency information above it. Unless I am missing something the raw files should therefore be free of aliasing and moirè in this mode, truly excellent news.
Of course if you were to pixel peep the two images side by side, the one from the 16MP raw file with an MTF50 of 0.27 cy/px would look significantly ‘sharper’ than the 64MP image at 0.15 cy/px (see here for an explanation of the units used to measure spatial resolution). But when comparing different systems we know better than to look at images at 100% – so we quickly and figuratively fit both of them to our monitor and evaluate them based on image height instead. This is what the performance of the two modes looks like in comparable units of line pairs per image height:
Looking just at the green channels as in the previous figure, 64MP mode wins by about 11% over 16MP (1041 lp/ph vs 937 lp/ph ) -that’s definitely noticeable on a displayed image. Plus it gets the no aliasing and moiré advantage. Minus the 1 second capture time, 3 second recycle time and 103MB raw file – not necessarily an issue in studio conditions.
A Little MTF Theory
So how do you model a sensor that captures and assembles ‘8 images […] with 16-megapixel image information while moving the sensor by 0.5 pixel steps between each shot’, as far as grayscale spatial resolution is concerned?
The key variables in the simple frequency domain model of spatial resolution (see this series of articles) are sampling pitch, pixel aperture (pixel size), f-number, filters and aberrations.
I think we start by recognizing that these are 8 separate captures with the same lens at the same f-number, therefore the imaging plane is sampled 8 times with the same pixel aperture, filter and aberratons as in 16MP Standard Shot mode. So if the lens was not ‘ouresolved’ at good ol’ 16MP, it will still not be outresolved when in 64MP High-Res Shot mode.
Assuming that two pixels of data are recorded every physical pixel-width as suggested by current literature, linear sampling pitch for the 64MP ORF raw files is effectively doubled. The combination of an unchanged pixel aperture and doubled pitch is a relatively wider integrating function. The top hat hangs over neighbouring pixels mixing spatial information from adjacent photosites as if the lens were moderately out of focus. And in fact the 64MP High Res MTF green curve can be modeled fairly accurately by a defocus function with about a 0.7 wavelength Optical Path Difference:
The green solid line is the measured MTF curve, the black solid line is the prediction of the model and the product of the three main components: f/5.6 diffraction, 3.74 micro pixel aperture/pitch and 0.7 wavelength OPD lens blur/defocus (dashed lines). Recall that Lord Rayleigh’s in-focus criterion was an optical path difference of less than 1/4 wavelength. And note that there is no antialiasing component, as I believe by looking at these graphs that the E-M5II does not have an AA filter.
The no aliasing and moiré bonus is simply due to the fact that twice the sampling frequency means that Nyquist is shifted twice as far out, near diffraction extinction, where there is virtually no incoming energy.
So if the working assumption holds, as far as modeling grayscale spatial resolution of the HR ORF file goes, we can actually think of High Res mode as producing a moderately out of focus 64MP capture.
All in all a very interesting feature. Good job Olympus.