Researchers have developed a fiber that functions as pixels and senses touch at the same time, integrating 54 of them into a sleeve to create the world’s first touchscreen built into a garment.

The sleeve displays images and video, responds to finger presses and connects wirelessly to other devices, yet stays as washable, breathable, and flexible as ordinary clothing. The team’s invention enables the creation of garments that act as a direct interface to a wearer’s health data and their digital devices, with no rigid screen involved.

The screens that we are familiar with, such as those on phones and laptops, typically present graphical user interfaces (GUIs) on flat, rigid panels that display information and enable interaction. Textiles, woven from one-dimensional fibers, have never matched this capability. A full-color, touch-sensitive display that can be threaded like yarn was, until now, out of reach.

“What made this possible was our ability to construct the GUI system from electronic building blocks in fibre form—similar to traditional yarn, but with the sophisticated electronic functionality built directly into the fibers,” says Assistant Professor Tural Khudiyev of the the materials science and engineering department at the National University of Singapore’s College of Design and Engineering (CDE).

The findings appear in Nature.

To build a fiber-enabled garment display, the NUS team developed a programmable method that creates connection points along tiny, insulated wires exactly where each microchip needs them. A laser exposes these locations, while controlled surface wetting guides molten solder to bond only at the programmed points. This enabled the team to integrate more than 5,000 chips through over 20,000 electrical connections.

The strip of interconnected digital LEDs was then integrated into a multilayer fibre using a thermal drawing approach, which allows precise formation of the touch-sensitive and pixel layers within a thread just a millimeter thick and 10 meters long. A layer beneath the LEDs senses touch—pressing the fiber completes a circuit, and the signal reveals exactly where your finger landed so the nearest pixel lights up.

“A fiber has a one-dimensional, axially symmetric geometry, which makes it difficult to build complex circuitry using approaches developed for conventional two-dimensional electronics,” adds Khudiyev. “We overcame this limitation through a new interconnection architecture, which enables thousands of microelectronic devices to be integrated and interconnected along a fibre.”

In the team’s prototype, 54 pixel fibers, each carrying 96 LEDs, were threaded into the forearm of a sleeve to form a 96 by 54-pixel display measuring 168 mm by 105 mm, refreshing 30 frames per second with 8-bit color depth. Each fibre has its own control channel, so a fault in one line leaves its neighbors working normally, and the system can register several touches at once. Alongside the pixel fibers, the team integrated peripheral fibers made the same way, carrying a microphone, a speaker and sensors that track movement, find magnetic north, measure temperature, and gauge how bright the surroundings are.

Together, the fibers support the functions early generations of smartphones could perform: run a calculator, show a photo gallery, play a video at 30 frames per second (25 frames per second with synchronized audio), work as a compass, respond to voice commands, and receive images over Bluetooth. They can even rotate the screen and adjust brightness automatically. When positioned under the arm, the temperature-sensing fibre lets the sleeve read and display body temperature.

Importantly, the garment still behaves like fabric. Light output and touch response were nearly unchanged after 50 detergent washes in a typical washing machine, and after 1,000 bends around a 5 mm radius. In addition, the built display is as breathable as cotton, polyester and denim, and stays within established safety limits for skin contact.

Smaller microchips and thinner fibers, both compatible with the team’s method, offer a route to sharper and larger displays. Furthermore, as the display and sensing elements sit along the length of the fibre, the thinner fibers can be knitted, weaved, and embroidered using standard industrial machines. The same technology could one day perform other functions such as tracking health signs and delivering touch-based feedback.

“Health care is one particularly promising application,” says Khudiyev. “A garment could continuously collect physiological information through fibre-based sensors and allow the wearer to access and interact with that information directly through their clothing. Combined with artificial intelligence (AI), this could enable personalized health monitoring, feedback and assistance. Just as smartphones enabled applications that were difficult to imagine when they first appeared, garment GUIs could ultimately enable entirely new applications that do not yet exist.”

The research team’s next step is to push the textile touchscreen further—towards higher resolution, integrated sensing, and on-device AI, bringing display, touch, sensing, and intelligence together within the same textile platform. This convergence could take the technology far beyond a wearable screen, towards clothing that understands both the wearer and their surroundings, providing information or interaction when and where needed.

The researchers are now developing a new interface between people and digital intelligence—one that extends beyond a device users carry to become a natural part of what they wear.

Source: National University of Singapore

Related

New display tech creates graphics you can feel

‘BrushLens’ smartphone case boosts touchscreen accessibility

Sensor turns almost anything into a touch screen

A new sensing system can turn couches, tables, sleeves, and the human body into a high-fidelity input device for computers.

The sensing system repurposes technology from new bone-conduction microphones, known as Voice Pickup Units (VPUs), which detect only those acoustic waves that travel along the surface of objects.

It works in noisy environments, along odd geometries such as toys and arms, and on soft fabrics such as clothing and furniture.

Called SAWSense, for the surface acoustic waves it relies on, the system recognizes different inputs, such as taps, scratches, and swipes, with 97% accuracy. In one demonstration, the team used a normal table to replace a laptop’s trackpad.

“This technology will enable you to treat, for example, the whole surface of your body like an interactive surface,” says Yasha Iravantchi, a doctoral candidate at the University of Michigan. “If you put the device on your wrist, you can do gestures on your own skin. We have preliminary findings that demonstrate this is entirely feasible.”

Taps, swipes, and other gestures send acoustic waves along the surfaces of materials. The sensing system then classifies these waves with machine learning to turn all touch into a robust set of inputs. The researchers presented the system at the 2023 Conference on Human Factors in Computing Systems, where it received a best paper award.

As more objects continue to incorporate smart or connected technology, designers are faced with a number of challenges when trying to give them intuitive input mechanisms. This results in a lot of clunky incorporation of input methods such as touch screens, as well as mechanical and capacitive buttons, Iravantchi says.

Touch screens may be too costly to enable gesture inputs across large surfaces like counters and refrigerators, while buttons only allow one kind of input at predefined locations.

Past approaches to overcome these limitations have included the use of microphones and cameras for audio- and gesture-based inputs, but the authors say techniques like these have limited practicality in the real world.

“When there’s a lot of background noise, or something comes between the user and the camera, audio and visual gesture inputs don’t work well,” Iravantchi says.

To overcome these limitations, the sensors powering SAWSense are housed in a hermetically sealed chamber that completely blocks even very loud ambient noise. The only entryway is through a mass-spring system that conducts the surface-acoustic waves inside the housing without ever coming in contact with sounds in the surrounding environment.

When combined with the team’s signal processing software, which generates features from the data before feeding it into the machine learning model, the system can record and classify the events along an object’s surface.

“There are other ways you could detect vibrations or surface-acoustic waves, like piezo-electric sensors or accelerometers,” says Alanson Sample, associate professor of electrical engineering and computer science, “but they can’t capture the broad range of frequencies that we need to tell the difference between a swipe and a scratch, for instance.”

The high fidelity of the VPUs allows SAWSense to identify a wide range of activities on a surface beyond user touch events. For instance, a VPU on a kitchen countertop can detect chopping, stirring, blending, or whisking, as well as identifying electronic devices in use such as a blender or microwave.

“VPUs do a good job of sensing activities and events happening in a well-defined area,” Iravantchi says. “This allows the functionality that comes with a smart object without the privacy concerns of a standard microphone that senses the whole room, for example.”

When multiple VPUs are used in combination, SAWSense could enable more specific and sensitive inputs, especially those that require a sense of space and distance like the keys on a keyboard or buttons on a remote.

In addition, the researchers are exploring the use of VPUs for medical sensing, including picking up delicate noises such as the sounds of joints and connective tissues as they move. The high-fidelity audio data VPUs provide could enable real-time analytics about a person’s health, Sample says.

The research is partially funded by Meta Platforms Inc. The team has applied for patent protection with the assistance of University of Michigan Innovation Partnerships and is seeking partners to bring the technology to market.

Source: Zachary Champion for University of Michigan

Related

What are the ethics of brain-computer interfaces?

Tech prints sensors right onto human skin

Paint transforms walls into interactive touchpads