Brighter, more stable, smarter: how smartphone cameras will improve in 2021

Artificial intelligence (AI) algorithms have improved many facets of image processing. And they haven’t finished. A slew of changes are coming in 2022, as smartphone makers push the boundaries.

What are the trends that will lead to this change? In a recent webinar, Global Mobile Camera Trends 2022, hosted by Counterpoint Research, experts from TECNO, Samsung and DXOMARK Image Labs discussed some of the key trends we’re likely to see over the coming year. – and what we will organize soon. in our hands.

Double OIS: stay stableNot getting the crisp images you wanted from your vacation views? Blurry and shaky photos are a common experience with most smartphone users and are especially pronounced in low light conditions as the shutter stays open longer to capture more light.

Smartphone makers solve this problem with Optical Image Stabilization (OIS) technology, where the lens “floats” inside the entire camera. The OIS system stabilizes the image or video by “shifting” the lens to counter shake.

OIS technology improves every year, with greater degrees of movement supported by the camera stabilizer. According to Jimmy Hsu, Senior Product Manager, Imaging Technology, TECNO, prior to 2019 the trend was for OIS to be around a degree of stabilization. According to Hsu, in 2021, the standard stabilization was increased to three degrees thanks to the OIS pan-tilt.

In 2022, the company aims to introduce the first Android phone with sensor displacement. “Compared to conventional optical imaging stabilization, TECNO’s sensor shift technology can achieve a response rate of 5,000 adjustments per second,” Hsu said during the webinar. By combining the lens shift seen in conventional OIS systems and new sensor shift technology, TECNO aims for stabilization of up to five degrees.

High-resolution sensors with pixel binning: Balancing resolution and light sensitivityShowcasing camera resolution trends, Tarun Pathak, director of research, smartphone, Counterpoint Research, said that about 40% of all smartphones in the second quarter of 2021 had 48-64 megapixel sensors ( MP) and 3% of all smartphones had 108 MP sensors. According to Pathak, smartphones with 200 MP sensors are expected to launch in 2022.

Sensors with a high pixel count tend to capture more detail of the scene. However, the pixel size on such sensors is generally smaller. “The smaller the pixel, the less light you can capture in each pixel, and the harder it is to achieve good image quality in darker environments,” said Pan Xuebao, vice president and head of R&D, Samsung Electronics.

However, sensor makers like Samsung are deploying an algorithm-based technique called pixel binning that allows high-resolution sensors (with a small pixel size) to perform better in low-light conditions. “Just as our human eye expands when we are in a darker environment to let in more light, likewise, in low light conditions, our algorithm will automatically put the sensor into high sensitivity mode,” Xuebao said. In this mode, the smaller individual pixels “combine” and act like larger pixels. This mode is disabled in brighter conditions.

Continuous optical zoom: closer to the actionPhones are getting more and more stylish, but that aesthetic can be a technological challenge. This trend presents a greater challenge for manufacturers trying to incorporate bulky telephoto lenses. Such technologies, such as periscopic lenses which use innovative positioning to enable 5x or 10x optical zoom, began to appear in smartphones in 2021.

The next step is to offer continuous optical zoom, which will allow users to switch smoothly between focal lengths. This requires precise movement of the lenses in small spaces without affecting the quality of the image.

During the webinar, Hsu predicted that the market may see continuous optical zoom in the second half of 2022, allowing users to switch from 3x optical zoom to 5x. Driven by advancements like the development of long-stroke motors and active optical alignment, we might even see 3x to 10x zoom in 2023, he said.

RGBW sensors: let there be (more) lightAre the cat photos not up to par? This could be due to the lighting. Indoor environments have less than ideal lighting conditions. Yet, this is where people typically take most of their photographs.

“[A solution to] low light is the holy grail of mobile photography, ”said Hervé Macudzinski, director of image science and product owner of DXOMARK Image Labs, which tests the quality of the camera, audio, display and battery. “It’s a [aspect] this can lead to disappointing quality. In low light, users capture portraits, photos of family, pets, and more. So there are a lot of challenges in this use case.

While smartphone manufacturers use different techniques to improve low-light performance, RGBW technology solves the problem at the sensor level.

The camera’s sensor captures color information through a mechanism called the Color Filter Array (CFA). Traditional camera sensors use CFAs arranged in the Bayer format, which includes a pattern of red (R), green (V), and blue (B) filters. RGBW sensors add white filters (W) to the mix.

According to Hsu, adding white pixels increases light sensitivity by 60%, which means the sensor should be able to absorb more light. Smartphone manufacturers are also developing glass + plastic lens technology – many smartphone lenses are constructed of plastic, but the introduction of glass elements into the lens construction improves light transmission. “With the combination of the two, the result is an overall 200% increase in light output,” Hsu said.

Smartphone performance in low light is expected to improve as these two technologies appear on smartphones in 2022.

Localized algorithms: using AI to personalize portraitsAs the cameras improve, we can see each other more clearly. New technologies allow our portraits to be more precise, especially for people of color.

While hardware enhancements are essential, smartphone software plays a bigger role in determining the final image or video quality of these portraits. “We have to have a really good processing, which is to understand what the user’s intention was and to modify the image adequately,” Macudzinski said.

That’s why some smartphone manufacturers are using big data and deep neural networks to drive personalized, localized image processing. For example, TECNO’s TAIVOS signal processing technology accurately recognizes and reproduces dark skin elements. According to Hsu, the company uses a large dark skin database to “produce over 6,120 assessment scenarios covering 115 countries.”

For portrait photography, the AI ​​engine recognizes and ranks different elements such as faces and hair color to determine the subject’s age, gender, and skin tone. The photo is then optimized for better clarity, color and light, allowing for more accurate reproduction of darker skin tones, even in low-light environments.

Scene, light and color enhancement: Behind every subject hides a background

It’s not just the subject of a photo that matters, it’s what’s behind it. Whether it’s the sun or a beautiful view, smartphone cameras adapt to better capture these details.

With traditional photography, professional photographers use their experience and training to make adjustments that highlight not only the subject, but elements of the background scene as well. They also usually spend a lot of time in post-production which brings the footage to life.

“Smartphone cameras have to provide all of these capabilities in the default mode,” Macudzinski said. “This means that the end user of the smartphone doesn’t have to think about everything. They just have to push the button.

This is where AI-driven image processing comes in. According to the Counterpoint Research white paper, AI is becoming advanced enough to recognize dynamic scene elements such as trees or the sky in the background. Once identified, the image processing system optimizes their color, contrast and exposure.

The AI ​​will also help interpret the light information. For example, if a user takes a photo against the sun, the algorithm works to underexpose the sun and make the subject brighter. Smartphone makers are also working on software solutions to correct distortions caused by wide-angle lenses.

Time-of-flight and dynamic vision sensors: getting a feel for depth

Photos with a sharp foreground and blurred background can give a nice effect, especially when taking portraits. However, to do this, the smartphone must recognize the depth and contours of the subject.

With improved time of flight (ToF) and dynamic vision sensors (DVS), smartphones will be better able to generate 3D models of the scene and detect contours. This will allow more precise measurements and provide users with features such as. enhanced and progressive blur of backgrounds.

“ToF and DVS could be the opportunities where cell phone cameras start to outperform (DSLR) cameras,” Hsu said.

A brighter and smarter 2022These trends indicate that the technology is becoming more complex and adapting to a variety of use cases, whether it’s custom portrait rendering, stable imagery, or allowing users to zoom in on the action.

However, advancements won’t be defined by cutting edge features on expensive smartphones alone. It’s a win across the budget – the developments are driving not only technological advancements but also premium features trickling down to smartphones in the low and mid-price range. With the way smartphone cameras evolve, the best moments of your life will be remembered more clearly than ever.

Source link

Comments are closed.