Neurons and Pixels

by James Cavuoto, editor

The goal of restoring vision to blind people using visual neuroprosthetics has been one of the most confounding challenges that has faced neurotechnology researchers and startup firms. The graveyard of commercial failures in this space—Optobionics, Retina Implant, Second Sight, Pixium Vision, and others—is testament to the difficulty of building a viable and sustainable solution.

So when Neuralink founder Elon Musk began to make claims that his Blindsight cortical implant could eventually outperform human vision, it rightfully raised expectations—and eyebrows—from divergent constituencies. Many in the blind community saw cause for hope, perhaps in part because of what this business leader has accomplished in other engineering endeavors. Others, mindful of the many unfilled promises in this area, saw cause for skepticism, perhaps in part because of some of the ridiculous statements coming from Musk in recent days.

Still others within the scientific community saw an opportunity to explore the question of what is possible with stimulation of the visual cortex of blind people. One researcher in particular, Ione Fine from the University of Washington, took to the literature, publishing in the journal Scientific Reports a model for simulating the visual percept produced by neurostimulation of V1. Her model casts serious doubt on Musk’s claims, at least for the foreseeable future. The journal article displays the extremely fuzzy images that one would perceive even with tens of thousands of stimulating electrodes implanted in V1.

The mistake that Musk and other naive players in this space make is presuming that there is a one-to-one relationship between neurons in V1 and phosphenes produced by stimulation, she says. “Engineers often think of electrodes as producing pixels,” Fine said, “but that is simply not how biology works. We hope that our simulations based on a simple model of the visual system can give insight into how these implants are going to perform. These simulations are very different from the intuition an engineer might have if they are thinking in terms of a pixels on a computer screen.”

Instead of bombarding the cortex with thousands of electrodes, it might be more effective to understand the receptive field of each neuron in V1 and to learn the code by which these neurons represent complex visual images, Ione contends.

The model described in the Scientific Reports article is elegant and the authors’ skepticism of Musk’s claims appears to be justified. Still, there is reason to be hopeful that cortical stimulation could become an effective strategy for visual neuroprosthetics.

Even with just a few electrodes—and just a few phosphenes they produce—the brain is a powerful instrument that can impart meaningful information to a blind person. We’re reminded of an early user of Second Sight’s original Argus implant—with just 16 electrodes—who was able to shoot hoops while using the device. This she accomplished by moving her head slowly and scanning the basketball backboard. It was stationary so each gaze, coupled with visual memory, multiplied the usefulness of those 16 electrodes.

We also recall from our time in the digital imaging industry how engineers were able to enhance the apparent resolution of low-res devices using techniques such as stochastic imaging (random dots) or increased pixel depth (dynamic range). And if we’ve learned anything from the early experiments with implanted BCIs, it’s that cortical neurons are inherently plastic, and can be trained to modify their response to input from a nearby electrode. Which is one reason why BrainGate implants have continued to perform even after many of the original 100 probes stop working.

In the end, perhaps the best strategy might be to devise a flexible implant and a training regimen that enables each individual user to modify the spatial and temporal pattern of stimulation of however many electrodes are useful. The result might not be super-human or even normal human vision. But it might just enable blind users to regain lost functions and perform activities they couldn’t perform without the device.