Panasonic promises high sensitivity sensors using ‘Micro Color Splitters’










Share:








Panasonic has devised a method to increase the sensitivity of image sensors, by replacing the near-universal colour filter array with prism-like ‘Micro Color Splitters’ to generate colour information. The key advantage is that all of the incoming light is directed to the sensor, instead of half or more being absorbed by the colour filter dyes. This promises to deliver images with less noise in low light. The development is published in the journal Nature Photonics, and outlined in a press release on Panasonic’s website.

According to Panasonic, the dyes used in conventional colour filter arrays absorb at least half of the incoming light before it reaches the sensor’s photosites. In contrast, the company’s ‘Micro Color Splitter’ technology passes all of the incoming light to the sensor. In practical terms, this corresponds to a sensitivity increase of a whole stop, which should in turn result in a stop improvement in noise performance. In other words, a ‘Micro Color Splitter’-based sensor should give similar noise at ISO 3200 as a conventional Bayer sensor does at ISO 1600, for example.

Perhaps the most interesting aspect of this development is that it may be the first serious alternative to colour filter array-based sensors since the Foveon X3 appeared over 10 years ago. Like any new technology there’ll be plenty of development to do before it appears in a real product, but the prospect of a stop better high ISO performance is certainly a tantalising one.  

Press release:

Panasonic Develops Technology for Highly Sensitive Image Sensors Using Micro Color Splitters

Constitution and features compared with the conventional method

Osaka, Japan – Panasonic Corporation has developed unique “micro color splitters”, which separate the light that falls on image sensors by exploiting light’s wavelike properties. Applying them to actual image sensors allows bright color images to be achieved even under low-light conditions. This development makes color filters unnecessary by using the micro color splitters that control the diffraction of light at a microscopic level. Panasonic has achieved approximately double the color sensitivity in comparison with conventional sensors that use color filters.

Image sensors are used in devices like smartphones, digital still cameras and video cameras, as well in security, vehicle parking, office, and healthcare applications – anywhere, in fact, that digital imaging is needed. Conventional color image sensors use a Bayer array, in which a red, green, or blue light-transmitting filter is placed above each sensor. These filters block 50 – 70% of the incoming light before it even reaches the sensor. Progress is being made in increasing the resolution of image sensors used in mobile and other devices by reducing pixel size, but demand for higher-sensitivity cameras is also increasing. Panasonic has developed a new technology that can be applied to existing or future sensors to enable them to capture uniquely vivid color images.

The developed technology has the following features.

  1. Using color alignment, which can use light more efficiently, instead of color filters, vivid color photographs can be taken at half the light levels needed by conventional sensors.
  2. Micro color splitters can simply replace the color filters in conventional image sensors, and are not dependent on the type of image sensor (CCD or CMOS) underneath.
  3. Micro color splitters can be fabricated using inorganic materials and existing semiconductor fabrication processes.

This development is based on the following new technology.

  1. A unique method of analysis and design based on wave optics that permits fast and precise computation of wave-optics phenomena.
  2. Device optimization technologies for creating micro color splitters that control the phase of the light passing through a transparent and highly-refractive plate-like structure to separate colors at a microscopic scale using diffraction.
  3. Layout technologies and unique algorithms that allow highly sensitive and precise color reproduction by combining the light that falls on detectors separated by the micro color splitters and processing the detected signals.

Panasonic holds 21 Japanese patents and 16 overseas patents, including pending applications, for this development.

This development is described in general terms in the Advance Online Publication version of Nature Photonics issued on February 3, 2013.

More on the Technology

 Conventional method using a color filter  Developed method using a micro color splitter

1. Unique method of analysis and design based on wave optics permitting fast and precise computation of wave-optics phenomena

FDTD is widely used to analyze light in wave form, but its heavy computation workload has up to now made it impractical for designing micro color splitters. On the other hand, BPM is an effective method of fast computation, but it has lower precision than FDTD and cannot accurately simulate color splitting. This prompted Panasonic to develop a practical and original design method that permits fast and precise computation of wave-optics phenomena. This technology allows the precise modeling of optical phenomena such as reflection, refraction, and diffraction by modeling spaces in regions with different optical constants and applying BPM to the spaces. This method can be applied not only to the design of micro color splitters, but can be extended to the design of other nano-scale optical processing systems.

2. Device optimization technologies leading to the creation of micro color splitters that control the phase of the light passing through a transparent and highly-refractive plate-like structure and use diffraction to separate colors on a microscopic scale

Color separation of light in micro color splitters is caused by a difference in refractive index between a) the plate-like high refractive material that is thinner than the wavelength of the light and b) the surrounding material. Controlling the phase of traveling light by optimizing the shape parameters causes diffraction phenomena that are seen only on a microscopic scale and which cause color separation. Micro color splitters are fabricated using a conventional semiconductor manufacturing process. Fine-tuning their shapes causes the efficient separation of certain colors and their complementary colors, or the splitting of white light into blue, green, and red like a prism, with almost no loss of light.

3. Layout technologies and unique algorithms that enable highly sensitive and precise color reproduction by overlapping diffracted light on detectors separated by micro color splitters and processing the detected signals

Since light separated by micro color splitters falls on the detectors in an overlapping manner, a new pixel layout and design algorithm are needed. The layout scheme is combined and optimized using an arithmetic processing technique designed specifically for mixed color signals. The result is highly sensitive and precise color reproduction. For example, if the structure separates light into a certain color and its complementary color, color pixels of white + red, white – red, white + blue, and white – blue are obtained and, using the arithmetic processing technique, are translated into normal color images without any loss of resolution.

Notes:

1. Diffraction
 
Behavior of light as a wave on the wavelength (nanometer) scale. Various phenomena occur when a wave encounters an obstacle.
 
2. Bayer array
 
The arrangement of color filters used in most single-chip digital imaging sensors used in digital cameras, camcorders, and scanners to create a color image. The filter pattern is 50% green, 25% red and 25% blue.
 
3. Charge Coupled Device Image Sensor (CCD sensor)
 
A type of solid-state image sensing device for digital imaging, used in digital video cameras of all types. It has higher sensitivity and lower noise than other sensing devices.
 
4. Complementary Metal Oxide Semiconductor Image Sensor (CMOS sensor)
 
A solid-state image sensing device for digital imaging using CMOS.
 
5. Finite-Difference Time-Domain method (FDTD)
 
FDTD is a versatile modeling technique used to solve Maxwell’s equations by spatial and temporal discretization.
 
6. Beam Propagation Method (BPM)
 
A numerical analysis technique in electromagnetics for solving the Helmholtz equation under conditions of a time-harmonic wave.


Comments


Lan

So, in summary, it’s basically the 3CCD camcorder idea, but on a pixel level.


John Summers

A Lumix teaser to go head on with the Sigma Merrill Fovern all color sensor.


migus

Makes most sense iff integrated within the u-lens, to also compensate for the incident angle delta (center – edge). But it’s a lab prototype, not mass product yet.


schirmer

Some mathematical details from nature article:
http://www.nature.com/nphoton/journal/vaop/ncurrent/extref/nphoton.2012.345-s1.pdf

Looks like a pixel pitch of about 1µ is required and the micro lenses have to ensure less than 10° angle of light at splitter. But this is only my clueless interpretation.


ttnewton

>>> “…pixels of white + red, white – red, white + blue, and white – blue are obtained and, using the arithmetic processing technique, are translated into normal color images without any loss of resolution.”

I wonder if no resolution is lost only providing the pixels are ~1/4 the area as for the conventional filter approach, because the final light entering a pixel actually derives from that pixel plus its adjacent pixels. For the same size pixels, it seems that sharing the light between pixels should result in a loss of resolution.


Andy Crowe

I wonder if this approach will have the same colour problems as CYM filter arrays? It’s a shame no-one managed to perfect that technology as it would have been an easy improvement over RGB arrays.


Peter Heckert2

It is to consider, light does not come in from an recticular angle as shown in the advertisement images above.
It comes from all angles with bright lenses and if aperture or focal length (zoom factor) changes, the angles will change. So it must be much more complicated and sophisticated, if this should be an universal sensor for ILS cameras.

I suspect it is for webcams and cellphones only.


stevens37y

Actually the microlenses (top lense on the diagram) collect the light from the different angles and direct it to the sensor area.


Peter Heckert2

I would want to hope so, but no lens does change the angle of a light beam that goes straight through the center of the lens.

This is already a problem with Bayer sensors and wide angle lenses, false colors in corners can occur.

Sorry, I am a little bit pessimistic to see this soon in a Camera 😉


OttoVonChriek

Four thirds standards require telecentric lenses


6x9

OK, it is so nice… What is the trade-off?
I guess, this technology will not work equally well for all sensor sizes, as there must be limitations on the photocells (and/or deflector) dimensions.


ThePhilips

If it brings us even a step closer to the Foveon-like true-color sensor, I’m all for it.

Actually, this is a first time something what appears to be easy to manufacture is proposed.

But I’d love to hear how that “fine-tuning” of splitter supposed to work though. And the “white + red, white – red, white + blue, and white – blue” thing sounds to be quite prone to clipping in both directions.


schirmer

If one uses different capacity on ‘sum’ pixels than on ‘sub’ pixels the electrical signal wouldn’t differ too much. But I think we’ll see Panasonic patents quite soon. Otherwise they wouldn’t have published this work now.


Peter Heckert2

Possibly this must be calibrated to the lens in use and so could prevent usage of arbitrary lenses. So is this for fixed lenses only?


schirmer

Not sure. It just replaces the color filter in the stack, so below the micro lenses so the micro splitters should behave similar.


BJN

Why? The sensor should be agnostic to the light it receives.


OttoVonChriek

What an excellent idea. And like all the best ideas, so incredibly simple!

I suspect the reason we are seeing this approach being applied first on four thirds is because it is easier to do with highly telecentric optics which characterise four thirds systems.


Andy Crowe

I wouldn’t call this a simple idea, it’s probably very complex to manufacture tiny beam splitters like that.

Also they don’t mention four thirds so it’s still to be seen whether this technology will apply to large sensors or (like BSI) only compact sensors.


Cane

I am sure there must something to complain about with this, just not sure what.


stevens37y

Interesting that nobody invented it earlier. The splitter can be a kind of microprism.


OldArrow

Like Polaroid instant slidefilm… Possible.


schirmer

It’s much smaller than a microprism: Size below the wavelength of light. So the dimension will be approx. in 100nm range. Not sure about available diffraction angle. If only small angle is possible small sensor pixel have to be used, otherwise the splitted wavelength will end on same pixel again.


plasnu

It is 2 TIMES better. The color filter is removed and the complementary color now reaches the sensor. Copmplementary color was blocked at the color filter on Bayer.


plasnu

Breakthrough technology!!!!


JEROME NOLAS

When? How much…?


Roland Karlsson

Interesting. Hmmmmm …. of course no one knows when or if this is going to materialise into real world sensors.


Michael_13

Excellent concept!
Would be great to have it in a camera, soon.
And please not only in µ43, but also in compacts like XZ-3 or LX8!

Hope they won’t run into moiré problems…


Hugo808

I bet my photo’s will still be crap…


Allan

Lol, mine too!


BJN

Like your punctuation 😉


HDF2

Ya gotta love honesty.


Michael_13

… but crap with much less noise!

😉


ManuelVilardeMacedo

Crap with incredibly accurate colours. I’m in.


Emacs23

Two lenses in the light path? That doesn’t sound too good.


markusm

These are micro-lenses. At this stage, you do not care too much about distortions, chromatic aberrations or any such thing. The only important thing is that the light travels in roughly the correct direction.


Andy Crowe

Most (all?) large sensors already have microlenses.


kb2zuz

My first concern would be that it might lead to color reproduction problems. Foveon has poor color reproduction with certain colors as it makes a hard delineation between where red stops and where green starts and where green stops and where blue starts. The human eye over-laps these colors so there’s some green in many reds and some blue in many greens, bayer filters mimic this overlap and have fewer problems with color reproductions. Without seeing specifics I can’t say for certain that this technology will have problems, but I have a feeling that it is quite a possible concern.


BJN

I agree. All I’m seeing here is white+red, white, and white-red. I don’t pretend to understand how visible colors could be reconstructed using pink(?), cyan, and white but it does sound like it’s the long way around to get green, the color we’re most sensitive to. One stop of sensitivity is good, but only if color rendition doesn’t suffer.


MarkInSF

They’re only showing one row. In current Bayer sensors one row of photosites is r,g,r,g and the next is b,g,b,g. The row they’re showing would replace one row, the next would instead split out the blue, so would be w-r,w+r,w-r,w+r and the next would be w+b,w-b,w+b,w-b. Each pixel consists of four photosites. In Bayer they are r, g on one row and g, b on the next.

In this the photosites would be w-r,w+r, then w+b, w-b. It takes slightly more computation to turn that into an rgb value (assuming you want one) , but what you need is there. Many video cameras have worked with analogous principles, measuring white, red, and blue. This is a bit more sophisticated. I’ve been hoping someone would come up with something similar. It’s potentially a lot simpler to make than a stacked sensor, though it doesn’t have all the advantages. It would still need an aa filter, though only in one dimension. Not a pro, so could be wrong about that.


jl123

Foveon disadvantages too. Also If fuji works out then 6×6 random could be used with this. no aa. j


Everdog

I wonder how long it will take to get from lab to production.


plasnu

It wouldn’t take a long time.

No no new manufacturing technology required.


vodanh1982

When’s it supposed to come out? Next year?


ageha

Most probably not.


mandophoto

Well, this is very annoying. I promised myself after last year’s camera acquisitions that those would be the last for at least 10 years. Now this?! Well this is just great, just great.


b33g33

This seems to have all the benefits of Foveon with none of it’s weaknesses.

Wouldn’t this also improve color rendering and resolution over Bayer?

(From the press piece: “The result is highly sensitive and precise color reproduction. For example, if the structure separates light into a certain color and its complementary color, color pixels of white + red, white – red, white + blue, and white – blue are obtained and, using the arithmetic processing technique, are translated into normal color images without any loss of resolution.”)

Hopefully they can get it to actually work outside the lab.


plasnu

Wrong. You’re misunderstanding.

It’s just different color pixels than Bayer and will have the same (kind of) drawback as Bayer. This one also will require AA filter.


ThePhilips

@plasnu. I wonder if that Panny sensor can be somehow combined with the Fuji X-Trans…


Stefan Zeiger

I wouldn’t jump to any conclusions before seeing results from a production model. Fuji’s XTrans design does away with the AA filter but it still needs demosaicing, so you get higher resolution and different demosaicing artifacts than from a Bayer sensor. This one might be similar (with yet another tradeoff, and new challenges to makers of raw converter software).


3systermuser

so will new Oly OM-D successor use it?


ageha

No.


Kim Letkeman

I’m sure that Panny can’t wait to outfit the next generation Olympus m4/3 camera with the latest breakthrough in sensor technology :-\


Surefoot

Well their sensor division sells its production, to whoever. It may be Panasonic themselves, or an external company, in which case they do make a quick profit on it, instead of having to actually sell the cameras that use these sensors.


agentul

also, no full-frame sensors either. Panasonic knows what “some people” have been saying about their cameras and lenses.


brunobarolo

With a one stop improvement, Panasonic may finally bring their sensors to the level where Sony sensors are today. Nice :-)


3systermuser

it is not just a stop better , according to Pana site , it will have 2 times better light sensitivity to any other kind of sensor tech available now including you mentioned Sony ones.
and Sony sensors are not really the best, I mean the new Toshiba sensors in the latest Nikons easily beat the Sonys.

I believe the D800 sensor is a Toshiba , this is why Sony has no access to it and using silly old 24.3mp junk used in the A99v and D600.
I have both A99v and D800E and know that the D800E sensor is much better , though as a whole camera I prefer the A99v.


Matthew Miller

“2 times better” = 1 stop.


ryansholl

http://nikonrumors.com/2012/08/29/confirmed-the-sensor-inside-the-nikon-d800-is-made-by-sony.aspx/

Not that it matters much, but for the record and all…


OniMirage

I think the biggest detail improvement will occur with low light and high ISO performance. Being able to accurately represent color in low light is a huge plus especially if your lifting shadows. In addition high ISO performance receiving a boost may also receive the ability to resist color shifts and retain fine detail. If these two things are possible by this technology alone, it could be a big deal.


Emacs23

3system: d800 has sony sensor. BTW, distribution noise figures of 24 “Toshiba” sensor are too close to Sony (other sensors differ much). And I was told the Nikon D3200 uses Sony sensor. This can only mean Toshiba licensed production of Sony.


marike6

@3systermuser According to Chipworks, the D800’s 36 mp sensor is made by Sony. The Nikon D5200’s APS-C sensor is made by Toshiba, and like the D800 sensor, it’s absolutely fantastic, topping all other APS-C sensors tested by DxOMark.


Kim Letkeman

They are already close enough for practical purposes in RAW and pretty close in JPEG too. This has the potential to leap ahead of everyone by increasing sensitivity, resolution and color accuracy all at the same time. Can’t wait to see it in a body …


plasnu

1 to 1.5 stop better.

Source Article from http://www.dpreview.com/news/2013/02/04/panasonic-promises-high-sensitivity-sensors-using-micro-color-splitters

Share on FacebookTweet about this on TwitterShare on Google+Pin on PinterestShare on TumblrShare on RedditShare on StumbleUponShare on LinkedInEmail this to someone