FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:   || 2 | 3 |

«A Fast and Flexible Computer Vision System for Implanted Visual Prostheses Wai Ho Li Monash Vision Group, Monash University, Australia Abstract. ...»

-- [ Page 1 ] --

A Fast and Flexible Computer Vision System for

Implanted Visual Prostheses

Wai Ho Li

Monash Vision Group, Monash University, Australia

Abstract. Implanted visual prostheses generate visual percepts by electrically stimulating the human visual pathway using an array of electrodes. The resulting bionic vision consists of a spatial-temporal pattern

of bright dots called phosphenes. This patient-specific phosphene pattern

has low resolution, limited dynamic range and is spatially irregular. This paper presents a computer vision system designed to deal with these limitations, especially spatial irregularity. The system uses a new mapping called the Camera Map to decouple the flexible spatial layout of image processing from the inflexible layout of phosphenes experienced by a patient. Detailed simulations of a cortical prosthesis currently in preclinical testing were performed to create phosphene patterns for testing.

The system was tested on a wearable prototype of the cortical prosthesis.

Despite having limited computational resources, the system operated in real time, taking only a few milliseconds to perform image processing and visualisations of simulated prosthetic vision.

Keywords: Visual Prosthesis, Bionic Eye, Cortical implant, Simulated Prosthetic Vision, Wearable computer vision, Integral Images, Irregular, Camera Maps, Real Time, Phosphene Maps, Image processing 1 Introduction According to the World Health Organization, visual impairment and blindness affect nearly 300 million people worldwide1. Some causes of vision loss, such as cataracts, can already be treated using existing medical technology. Implanted Visual Prostheses (IVP) attempt to address currently incurable diseases, such as Retinitis Pigmentosa (RP), by electrically stimulating the still-healthy parts of a patient’s visual pathway to produce prosthetic vision.

Prosthetic vision has many limitations, which are further detailed in Section 2. These limitations severely restrict the bandwidth of visual information that can be provided to a patient. Computer Vision provides a promising way to improve the usefulness of prosthetic vision despite its limitations. This paper presents a computer vision system for implanted visual prostheses. The system can be flexibly tailored in a patient-specific manner and operates in real time on a computationally limited wearable prototype. The research contributions, design and testing of the system are detailed in Section 3.

http://www.who.int/mediacentre/factsheets/fs282/en/ 2 ECCV-14 submission ID W22-54 Ever since 1755, when LeRoy discharged a Leyden Jar to cause a blind patient to see “flames passing rapidly downward” [15], numerous experiments have confirmed that electrical stimulation of the human visual pathway can result in visual percepts. Modern implanted visual prosthesis (IVP) operate using the same fundamental principle. Controlled electrical stimulation is applied using small implanted electrodes to produce a bright visual percept called a phosphene.

By apply temporally varying stimuli using an array of electrodes, the patient sees spatial-temporal patterns of phosphenes similar to a low resolution dot display.

In the late 1960’s, Brindley and Lewin [2] developed the first IVP. The system used an array of electrodes on the visual cortex to elicit multiple phosphenes at different locations of a patient’s visual field. However, the IVP was only suitable for laboratory use as the stimulation electronics were not portable. The IVP also did not include a portable camera.

From the 1970’s to the early 2000’s, Dobelle developed several IVP devices that used implanted cortical electrode arrays, including systems that generate electrical stimuli based on imagery captured with a headworn camera [9]. Despite a range of problems including the heaviness of the portable electronics and the use of wired transcranial connections, a patient’s biography suggests that the device did provide useful vision [21].

Recent research and development have focused on IVP that electrically stimulate either the retina or the visual cortex2. The reason for the focus on retinal and cortical stimulation is that electrical stimulation at these two anatomical locations can give reliable spatial patterns of phosphenes. Retinal prostheses, such as the Argus II device from Second Sight, have already been implanted into several tens of human patients in clinical trials [11]. Cortical implants, such as the Monash Vision Group’s Gennaris device3, are still in the preclinical phase.

However, cortical implants may be able to treat additional causes of blindness as the cortex is further downstream along the visual pathway. The cortex also has a larger surface area than the retina, which may allow vision with higher spatial resolution.

For a survey of IVP research and development, including many concepts in this paper, please refer to the extensive book edited by Dagnelie [8].

2 Limitations of Implanted Visual Prostheses At a fundamental level, implanted visual prostheses operate by converting imagery from a headworn camera into spatial-temporal patterns of electrical stimulation applied to a patient’s visual pathway. This is true for both cortical and retinal prostheses. The conversion process is usually performed on portable computational hardware, which is externally worn by the patient.

Figure 1 is a system overview of Monash Vision Group’s cortical visual prosthesis [17], which contains stereotypical sub-systems shared by many other prostheses. Images are captured by a headworn camera and sent to a portable comhttp://www.eye-tuebingen.de/zrenner/retimplantlist/ http://www.monash.edu.au/bioniceye/resources.html (Annual report 2013) ECCV-14 submission ID W22-54 3 puter, the Pocket Processor. In real time, the pocket processor converts camera images into spatial-temporal patterns of electrical stimulation, which are conveyed over a wireless link. The implanted electrodes receives electrical power and signal from the wireless coil, which it uses to apply electrical stimulation to the visual cortex. A conceptual walkthrough of how the MVG device operates is available online: http://youtu.be/v9Ip8j3eca8.

Fig. 1: Overview of the Monash Vision Group (MVG) Cortical Visual Prosthesis.

2.1 Limited Spatial and Intensity Resolutions The conversion from headworn sensor imagery to electrical stimuli is an ongoing research problem. While state-of-the-art stimulation regimes are able to reliably elicit phosphenes (bright visual percepts), the elicited phosphenes have poor dynamic range and can only be packed at low spatial resolutions. Figure 2 illustrates this using Simulated Prosthetic Vision (SPV), a technique pioneered in the early 1990’s to simulate what an implanted patient may see [4]. The input image is converted into prosthetic vision using an adaptive thresholding approach [22] where a corresponding phosphene is enabled for bright regions of the input image. The SPV assumes the ability to generate 625 binary phosphenes, which is similar to the expected capabilities of the Monash Vision Group prosthesis [16].

The SPV image in Figure 2b clearly illustrates the severe information loss due to the limited spatial and intensity resolution of prosthetic vision. As the number of phosphenes generally corresponds to the number of implanted electrodes4, the spatial resolution of prosthetic vision is limited by the factors such as the spread coordinated activation of many electrodes may increase future phosphenes counts 4 ECCV-14 submission ID W22-54

–  –  –

Fig. 2: Simulated Prosthetic Vision (SPV) from an implanted visual prostheses.

of electrical charge, surgical safety of implantation and electrode fabrication technology. Improvement in these areas are slow as they often require lengthy preclinical and clinical trials.

Clinical studies of retinal prostheses [10] suggest that multiple levels of phosphene brightness can be achieved but brightness ratings are likely to vary substantially across sessions and across subjects. There is also evidence that phosphenes of multiple levels of intensity can be produced by varying stimulation currents [20], but changes in phosphene brightness may be coupled with changes in phosphene shape and size. There is little evidence that phosphene brightness can be varied consistently with a cortical prosthesis. As such, the work presented below assumes the worst case of binary phosphenes.

Arguably, Dobelle was the first to consider the use of computer vision to improve the usefulness of prosthetic vision [9]. More recently, simple IVP computer vision algorithms were developed to run on wearable devices with embedded processors [29, 23]. More sophisticated IVP vision algorithms have also been investigated using less portable computers. Transformative Reality [18] uses multiple sensing modalities to better render a pattern of phosphenes representing models of the world. The substantial body of work on Vision Processing for prosthetic vision [1] applies computer vision algorithms to improve simulated multi-intensity phosphene patterns for reading text, navigation and other tasks.

2.2 Irregular Phosphene Maps

A patient’s Phosphene Map contains all the phosphenes that can be elicited by electrical stimulation. Older IVP research, including work on image processing and computer vision, generally assumes regular phosphene maps similar to the map shown in Figure 3a. However, there is strong clinical and biological evidence to suggest that phosphene maps are irregular and patient-specific [3, 19, 27]. An example of an irregular phosphene map is shown in Figure 3b.

Apart from irregular locations and sizes, there is also evidence that phosphenes can exhibit irregular shapes. Studies from Bionic Vision Australia5 and Second http://goo.gl/LwcGwO ECCV-14 submission ID W22-54 5

–  –  –

Fig. 3: Example of Regular and Irregular phosphene maps.

Sight [19] show that the shape of phosphenes can be anisotropic and the shape of phosphenes may vary depending on electrical stimulation.

The computer vision system presented in this paper has the potential to accommodate all three aspects of phosphene map irregularity: location, size and shape. However, as the MVG device is still in the preclinical stage, the system is only tested on phosphene maps simulated based on electrical, surgical and cortical models. These models only generate irregularities in phosphene locations and sizes. The simulation assumes that phosphenes appear as isotropic Gaussians as recommended by the survey of simulated prosthetic vision by Chen et al [5].

Details of the simulation are available in Section 3.1.

3 Computer Vision System for IVP Despite the reality that Implanted Visual Prostheses (IVP) produce irregular phosphene maps, very little research has been done to address the problem in full. Research that attempts to deal with irregular phosphene maps generally only do so for near-regular mappings where small spatial shifts in phosphene locations and electrode dropouts are modelled [25] or only irregular phosphene shapes are considered over a regular grid [14].

More importantly, many systems do not run in real time on an embedded processor suitable for a wearable medical device. Clinical trials of retinal implants [28] and cortical implants [21] suggest that prosthetic vision may have refresh rates as high as 10Hz. Simulated Prosthetic Vision trials show that low refresh rates may reduce task performance [13]. Therefore, a practical IVP requires a fast and flexible computer vision system.

Given the background above, this paper provides the following contributions:

1. Section 3.1 describes a detailed simulation of a cortical IVP device

2. Section 3.2 details a computer vision system that deals with irregular phosphene maps using a second mapping called the Camera Map to provide flexibility.

3. Section 3.3 details a fast image processing method for the vision system.

4. Section 3.4 details a simulated prosthetic vision visualisation that shows the phosphenes seen by a patient in real time.

5. Section 3.5 summarises the real time performance of the system.

6 ECCV-14 submission ID W22-54

3.1 Simulating a Cortical Implanted Visual Prosthesis

Phosphene maps were simulated in order to test the computer vision system.

The simulation is based on the Monash Vision Group (MVG) cortical Implanted Visual Prothesis (IVP), which is currently undergoing preclinical trials. Parameters of the MVG IVP system were obtained from published sources [17, 16].

The main components of the simulation are detailed on the left of Figure 4 (in red). The simulation starts with the definition of the spatial layout of an implanted electrode array. The array is also known as a tile. The MVG IVP uses multiple identical tiles. A tile contains 43 active electrodes. The MVG Tile Layout is shown at the top left of Figure 4 with blue dots representing electrodes.

Next, the simulation places multiple tiles onto the surface of the visual cortex. Coordinates on the visual cortex are defined on a Cortical Plane, which represents a flattened cortical surface. Tile Locations are defined using 2D affine transforms. This results in a list of Ideal Electrode Locations on the cortical plane.

A surgical scenario proposed by a MVG Neurosurgeon is shown at the middleleft of Figure 4. The four-tile wedge-shaped arrangement avoids the Calcarine Sulcus, which is a large crevice on the visual cortex.

The simulation then applies two sources of irregularities that simulate real world issues: Electrode dropouts and the imprecise placement of electrodes. The Dropout Rate models implanted electrodes that fail to elicit a phosphene when stimulated electrically. For example, a dropout rate of 50% means that half of all implanted electrodes cannot be used to elicit a phosphene. Electrode dropouts have been reported in multiple IVP clinical trials [11, 32], but generally at rates lower than 50%.

Spatial Error models several issues by approximating their combined effect as a normally distributed 2D random offset defined on the cortical plane. For example, electrode deformation during surgical insertion and variations in cortical anatomy are both factors that can be approximated as spatial error. The application of dropouts and spatial error results in Irregular Electrode Locations, an example of which can be seen at the bottom-left of Figure 4.

Pages:   || 2 | 3 |

Similar works:

«Milan BExpo 2015: A behavioural study on food choices and eating habits Final report Specific Contract n° 2014 85 09 Written by: Dr. Millie Elsen (CentERdata) Dr. Roxanne van Giesen (CentERdata) Dr. Jorna Leenheer (CentERdata) Date: 12 October 2015 Justice and Consumers EUROPEAN COMMISSION Produced by Consumers, Health, Agriculture and Food Executive Agency (Chafea) on behalf of Directorate-General for Justice and Consumers Directorate E — Consumers Unit E.1 (Consumer Markets) European...»

«Journal of Data Science 9(2011), 23-41 Maximum Likelihood Estimation for Ascertainment Bias in Sampling Siblings Balgobin Nandram1, Jai-Won Choi2 and Hongyan Xu2 1 Worcester Polytechnic Institute and 2 Medical College of Georgia Abstract: When there is a rare disease in a population, it is inefficient to take a random sample to estimate a parameter. Instead one takes a random sample of all nuclear families with the disease by ascertaining at least one affected sibling (proband) of each...»

«Iosr Journal Of Pharmacy (e)-ISSN: 2250-3013, (p)-ISSN: 2319-4219 Www.Iosrphr.Org Volume 3, Issue 3 (April 2013), Pp 31-43 Anxiolytics, Antidepressants and Autogenous training in the therapy of burning mouth syndrome Josipa Sanja Gruden Pokupec 1, Lidija Gavić 2, Mladen Klemenčić 3,Helena Matić4 1) Dental Clinic, Perkovceva 3, Zagreb, University of Medicine, Soltanska 2 in Split, Croatia 2) University of Medicine, Soltanska 2 in Split, Croatia 3) Dental Clinic, Perkovceva 3, Zagreb,Croatia...»

«GROUP LEADERS’ GUIDE I. RI Group Meeting Policies and Procedures II. RI Group Leadership III. Appendix A – Recovery International Meetings IV. Appendix B – RI Discovery Meetings V. Appendix C – Recovery International Phone Meetings VI. Appendix D – Recovery International Online Meetings REVISED EDITION © Copyright 2010 by Abraham Low Self-Help Systems ************* Neither this booklet, nor any portion thereof, may be reproduced in any form without the written consent of Abraham Low...»

«1 2 Dr. Amy K. LeBlanc, University of Tennessee, D14CA-002 Total Study Cost: $160,782 Lymphoma is one of the most commonly occurring malignant tumors in dogs. Though treatable, the disease often recurs and spreads. This study will determine the safety, efficacy and prognostic factors of a cancer-killing virus developed by researchers at the Mayo Clinic. Researchers at the University of Tennessee have already determined a safe dosing protocol for this virus in healthy dogs, and this clinical...»

«SPA TREATMENT MENU The Atlantic Spa, a perfect combination of relaxation and renewal focusing on the healing powers of the Atlantic Ocean and the vast Namib Desert. The Atlantic Spa, a perfect combination of relaxation and renewal focusing on the healing powers of the Atlantic Ocean and the vast Namib Desert. Our core philosophy of natural health is based on inspiration from the Atlantic Ocean and the use of Mother Earth’s ingredients. For centuries natural ingredients have been used to...»

«Basic Plan Preferred Provider Organization Evidence of Coverage Effective January 1, 2016 – December 31, 2016 A Self-Funded Plan Administered Under the Public Employees’ Medical & Hospital Care Act (PEMHCA) HOW TO REACH US Important: For all members outside of the United States, contact the operator in the country you are in to assist you in making a toll-free number call.CUSTOMER SERVICE For information regarding Protected Health Information: For medical claims status, claim forms,...»

«Printed Items in the Baring-Gould collection of Popular Literature Listing of items, by their order in the volumes, in the collection of Popular Literature formerly with Baring-Gould’s Books in the library of Killerton House, Nr Exeter and now held in the Devon Record Office. Ballads – Volume 1 O'Connell's health, to which is added, the pearl of Lucantown. Dublin: printed in the present year, 1820? 8p. Erin's lament, together with the Peeler and goat, and Molly Astore. Printed in the...»

«Weevil for Canada Thistle control Kennedy Creek Riparian Health Assessment CARA Shelterbelt Weed control in shelterbelt with chickens Browning of evergreens Chinook Applied Research Association 2012 Annual Report 81 Riparian Health Assessment of Kennedy Creek Background Riparian health is critical to water quality, quantity, stream stability and habitat for fish and wildlife. A Riparian Zone is the interface between the upland area and the aquatic zone. Riparian communities usually include or...»

«HOW HEALTHY IS YOUR CHURCH?VITAL SIGNS OF A HEALTHY CHURCH • Has a functional dependency on Christ and HIS Gospel • Has a clear plan for members to serve the body according to the instructions of THE HEAD • Has a clear process for increasing spiritual health in each member and especially in each leader • Has a process for engaging each member in kingdom focused prayer— especially for leaders • Has leaders who are known for regular and unhurried prayer for the church and its people...»

«PACKAGE LEAFLET 1 Package leaflet: Information for the user /./ 70 mg tablets Alendronic acid Read all of this leaflet carefully before you start taking this medicine because it contains important information for you.Keep this leaflet. You may need to read it again.If you have any further questions, ask your doctor or pharmacist.This medicine has been prescribed for you only. Do not pass it on to others. It may harm them, even if their signs of illness are the same as yours. If you get any side...»

«Mental Health Chaplaincy: A Buddhist Perspective Peter Goble We are all pioneers of a sort – rough frontiersmen and women. The so-called helping professions are at the beginning of an exciting journey that attempts to explore and enlarge human awareness and establish new disciplines – echoing some of the experiences of sixteenth-century physics. It is a fresh form of alchemy. There is that same sense of arrant nonsense and profound wisdom jostling side by side – often very difficult to...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.