Visually impaired persons can benefit from the use of RSA (remote sighted assistance) technology, that connects visually challenged people with human agents via a live video chat on their cellphones. Even if current computer vision technology can read the directions on a prescription bottle and recognise flight information on a digital screen at an airport, what happens when an agent needs further help?
Challenges which can be solved with current computer vision approaches have been identified by researchers at Penn State’s College of Information Science & Technology. These issues may be better handled if humans and artificial intelligence (AI) worked in tandem to advance the technology for visually impaired people and for the agents who serve them.
There are five developing issues with RSA that need to be addressed through human-AI collaboration. John M. Carroll, prominent professor of information sciences & technology say that solving these issues might help develop computer vision research and kickstart the very next generation of RSA services.
To build this paradigm, we want to work with both sighted and non-sighted humans, and we also want to incorporate computer vision skills,” Carroll stated. Human-to-human, human-technology, and human-technological interaction and innovation all feature prominently in our framework.”
Free apps which connect visually impaired people of sighted volunteers or commercial services that connect them to sighted agents are presently available. As a result, the agent perceives the world through the eyes of the user, assisting them in completing their tasks.
It’s difficult for agents to create a perspective through the lens of a camera, Billah explained. “The good news is that computers that run a 3D reconstruction algorithms can outsource part of this effort.”
When an agent helps a vision challenged user find their way around the parking lot or reads the label on the bottle of medication, there are bigger stakes involved.
Billah believes that existing computer vision technology has potential for advancement in order to overcome these issues.
During their investigation, the researchers looked at the current RSA technologies and spoke to users to learn about their technical and navigational issues. Next, a subset of difficulties was identified that could be handled using currently available computer vision technology, and design concepts were provided for doing so Computer vision approaches are not capable of solving five developing challenges because of their complexity.
By addressing these issues, experts feel that the RSA’s design and experience might be improved.
Recognizing that items that smartphone cameras identify as impediments may really be beneficial resources for visually impaired people. Even while a wall that borders the sidewalk may seem as a barrier in most navigational applications, visually impaired people who use a cane may use it as a guide to help them navigate their path.
Live camera feeds can be lost when there is limited cellular connectivity, which is common in interior environments.
Digital LCD displays like temperature control panels inside a hotel room or flight information at an airport , can be read by people with visual impairments.
The ability to read words printed on non-standard or uneven surfaces. People with visual impairments may have trouble reading labels on products because they are printed in a way that makes it difficult for the human agents assigned to help them to do so.
Predicting the movement of individuals or things that are not in the frame. A user’s safety and the safety of other people depends on agents being able to immediately transmit environmental information in the user’s public surrounds, such as the presence of other people or a moving automobile. However, the researchers observed that agents are currently unable to follow and anticipate the movements of these other persons and things.
To help both visually challenged users and agents, researchers are conducting a study.
As a doctorate student at IST, Rui Yu envisions a future where computer vision may be used to immerse the agent in a virtual world and equip them with mixed reality technologies. To enable the users receive some basic information from the environment, we will deploy computer vision technologies.”
It was supported by the National Institutes of Health as well as the National Library of Medicine, with contributions from a former doctoral student in the College of Information Science and Technology and current research fellow at the Rochester Institute of Technology and Jingyi Xie, a doctoral student in informatics.