|
Did they start the revolution without you? 02/21/07 UPDATE: |
The Revolution is Now If you've been sitting around waiting for new ideas to come forth from the various What's New columns (2,137 last count), then you may find this a tad boring. It's about an image chip for compact digital cameras that merely combines a bunch of existing ideas in a new package with implications that will remove your shoes, peel your socks off, then replace your shoes. Sony took the idea of A to D and lifted it to a new plateau. Someone asked the question, "What would the effect be of making a little A>D circuit for every line of pixels in an image chip?" Several other people around him in the break room slapped their heads and shouted, "High Speeeeeed! That was months ago. After many months of banging heads, squashing problems and pumping up the designs, a new imaging chip has surfaced. Technically it's the IMXX017CQE, but you can call it Speedy Gonzalez if you wish. I just call it the X017. Here's why it matters:
Although the whole chip is 6.4MP, speed increases when only a portion of the pixel map is used to create an image. It's a technique that has been with us since the 1990s when live viewing became available through the monitor on the back of your camera. None of those chips were full-rez video rate. The monitor view was a compromise. If you only viewed the results of every third pixel on every third row of sensors, you would effectively reduce processing time to 11% of the chip's previous performance. In other words, it would still make a picture--cruder by a significant degree--but speed up by 888%. If you ignored 4 out of 5 pixels and 4 out of 5 rows, the live image would become 4% of its full resolution, but could speed up as much as 2500%. Believe it or else, this was actually the case in every digital still camera you've been peering at since 1997 that showed a live picture on its monitor. That's why the "live" images looked so steppy with generous amounts of poor edge continuity, but who cares--it's only a viewfinder, right? The so-called "video" output of these cameras was similarly flawed. You can't get a perfect image out of every ninth or twenty-fifth pixel. Recent cameras have gotten better, but still not pixel-perfect video. The reason for the odd divisions is that with any sort of Beyer pattern--that color checkerboard overlay that teases full color separations out of a single chip--even divisions would keep repeating the same colors, but odd divisions kept the Beyer effects coherent. If Sony's X017 chip is a logical step forward. There's "logical" and then there's "Holy Spumoni: are you telling me THAT'S LOGICAL???" This appears to fall into the latter category. The full frame rate of the whole chip is 60 shots per second. I'll let that settle in for a moment. Of course it means very high speed motor-drive like still camera sequences, but it goes way beyond that. You see, the 60 frames per second thing has no limit. The chip can pump out these pictures forever, given a really, really high capacity battery. So it's not actually a still camera chip, now, is it? The 6.4MP part of the idea says it is, but the 60 shot per second part shouts "Video!" and both would be right. Chip size is important. I mean the dimensions of the image. This new chip falls into the common 3:4 aspect, 1:1.8" package size with a picture receiving area that has a diagonal of 9.1mm. And that means that many existing optical packages--some with truly extraordinary ranges and capabilities--can be adapted to the X017 chip. What does all this give to the world? More than you might think. 1. Viewfinders may have real-time, full-resolution images. At 60 full frames per second, the viewfinder of the future will be a glorious full resolution view. So full, so fast and so high in resolution that the need for optical DSLR-style viewing may fade as the next generation of killer cameras appear with these--and similar--chips. OLED viewing is on the rise. pixel counts are already above the camera monitor pixel densities of current gear and the day in which a 3-inch screen will hold a full-resolution, multi-MP image is landing soon at a camera store near you. 2. HDTV cameras the size of your cell phone will become common. With a 6MP image, the chip shoots a superior quality 60 full frame 10-bit image stream of shots. Processing that into a high quality MPEG or JPEG stream could result in massive storage requirements, or it could result in HDV-like compressed data stream easily absorbed by any technology (chips, tape, Wi-Fi, 802.11n) that can breathe quickly enough. 3. Super high-speed frame rate video for motion analysis. Add a version of the pixel-skipping that was needed in the past for seeing a live picture at all, and suddenly you have the ability to see the world at 300 fps. Like the water balloon above. Not full rez, mind, but still video quality. What happens to a world that can see at high speed for the price of a pocket camera? You could do a lot with that capability. Here are some blue-sky notions: HDX Movie Cameras. At 60 fps, the X017 spits out a 10-bit image with every frame. Saving this into a high-speed disk drive is an already solved suite of technologies. Now you have every shot 2916 pixels wide by up to 2178 pixels high. Out of that you can carve a 2916 x 1576 pixel frame in post production yielding a 1.85:1, movie screen format image. Or perhaps do it in the camera to save some storage space. Reduce the frame rate to 24 (itself a 60% data savings) and you have a Panavision, Arriflex and Mitchell cine-camera killer for the price of a Coolpix. Move over, Camera Red.
Obfuscated on the Showscan reference? Allow me to eschew: Showscan was a movie format with a 2-inch wide image area. Meaning about 50mm of picture. At 30 lines of resolution per millimeter, those cameras made a picture with around 1800 lines of detail per frame at 60 frames per second. Of course if you were to resolve higher detail than that per frame--and cinematographers claim higher resolution than God--the numbers would be higher, but I digress. The point is that with 2916 pixels across the shot, 60fps on the world's best HD projector will turn your frames into Showscan At Home. Or, it could, if all the follow-up technologies align just right. You are going to need a bigger HD TV. Super Slow Mo for the Masses. Normal video makes an interlaced image every 1/60 sec. The X017 can do that full frame, full rez. But cut out the need for full resolution and the frame rate goes up exponentially. At 1/3 resolution, the camera still shoots 972 x 726 pixels, delivering a potentially half-HD image with 888% higher frame rate. Meaning perhaps as much as 533 fps. Am I dreaming? Yes, but it's such a good dream. Sony is rating the chip using pixel-skipping readouts at 300 fps, but the math suggests that this is a conservative number. Dreaming forward, a 1/5 rez image, still 538 x 435, VHS-ish in dimension and acuity, could speed up 2,500% for a 1500 fps view of the world. And the data recording rates for these would be no higher than 60 fps full-frame shooting. Fingersnap! The current readout format design isn't following my dream, but that may be just a software opportunity waiting to happen. The standard 300 fps output is a whack in the head, producing a 2880 x 432 pixel map that gets reformatted as a 3:4 image on the way out. In the current chip design, this was an easy compromise to make, but I hear that the guy who asked the original question is back in the break room at Sony snapping his fingers, too. Keep in mind, this is only the first chip to clump all the required technologies together. Evolution will evolve.
Mentos and Diet Coke anyone? But just the same, your golf swing, bat swing, power dive, and every other sports technique will come under unprecedented scrutiny. Boxing match video will never be the same. And just wait till NASCAR gets ahold of one of these. Black Box Video. With a 60 fps huge picture feeding an ever-recycling data storage unit, super-quality security camera systems could shoot and constantly update the last hour or more of activity in real time, only stopping to preserve the data when some significant event were to occur. Most of the cop car video that a camera captures is insignificant, but if a crash, traffic stop or arrest were flagged by the officer, or by sensors automatically, the stream of way-above HD images could be of immense value. Using the 300 fps mode, this enters other new realms. Conceivably, a continuous stream of 300 fps video could be recorded and recycled until something significant happens as detected by sensor or decision-switch. Huge chunks of time could be preserved at previously impossible frame rates in great temporal detail. Anything that fails in critical systems can afford to have a fabulously rich visual slow-motion monitor tracking it continously. VSLRs. I've invented yet another new term. The VSLR. It's not an optical SLR, and yet it might as well be. The X017 has a 1/3 rez mode that creates a 959 x 719 pixel image three times faster than phlegm and with an OLED view screen (over 100 MHz in today's lab versions) and fast support electronics, the live displayed view could be well under 1/30-second behind real time. You aren't under 1/30 second behind real time, for comparison. The eye/brain system we've all inherited is slow, compared to that of a cat, spider, bee or bird. So much for thinking you were Number One. But I digress again. Where this is Big Time Significant is that the major definitive difference between compact digital cameras and DSLRs is that the viewfinder is full-resolution (photons) and has no lag at all in a DSLR (photons at the speed of themselves). Compact digital cameras show a picture that is from 1/10 second to 1/4 second in the past. It's all that processing in the back room that slows the image down. The lower the lag time, the easier it is to view a display screen and anticipate the critical moment for shutter release. By the same token, the higher the resolution of the display, the better to focus and frame, and the more you feel connected to the world through your camera. HDTV live viewfinders are WAY more transparent than the views you've been seeing on compact digital cameras. Ask me about mine. As each frame takes 1/60 second to capture, that time will always be part of an electronic lag, unless, of course, improvements to collecting the image speed up on the next version of one of these chips. But even with this 16.7 millisecond amount of lag, fast image distribution to the viewfinder would result in so little perceived lag that you wouldn't feel cheated. Wirelessness hasn't hit digital cameras much, but here it comes. Separate the viewfinding from the camera part and magic happens. X-tooth anyone? You could wear the viewfinder on your sunglasses, flipping it into viewing position while your hands positioned the camera head elsewhere. No need to keep the pieces together any more. Isn't wireless fun? I'm getting a sense of deja view. Back in the 70's I drew pictures predicting where the evolving world of image chips could lead. I called my anywhere-mounted micro-cam the Vistor and the head-mounted direct view playback, solid state display chip the Pictor. Ah, misspent youth. For decades the world has been evolving those in the familiar forms available in every video camera, but the X017 expands that vision. Hopefully, Sony will introduce a tour-de-force camera with still, video, hyper-video, super high speed slow motion and introduce it soon. I'm waiting... 08/31/07 Interesting. Casio seems to be making the first camera with this chip. Here's an announcement. on DPReview. | |
you don't know what that all means, it's okay, you could get one of our eBooks and dig it out, or move on to the more interesting stuff in the next paragraph.
With in-camera compression cranked up, HD images in HDV format can become everybody's business at superior quality. Switching among frame rates now becomes business as usual for all the HD formats including 1080/60p (60 full frames per second) with near-
Where this becomes significant is when you throw the switch and shoot pictures of anything speedy. Hummingbirds get flappy, bees pump air at a reasonable rate, water drops do liquid coronets, kitchen matches fluff into fire in macro close ups and Christmas tree bulbs crash into gleaming shards of sparkles right before your lens. Anything that goes boom , crash or splat becomes a ballet. Oh, the worlds of mass destruction this chip will unleash!