ITRA

Information Technology Research Academy

Virtual Assistant for Mobile Devices using Voice and Gesture Technologies 

Current interfaces on a mobile phone are based on alpha-numeric keys and touch technologies, which expect a human being to operate over a tiny area. These interfaces are extremely sub-optimal and restricted for literate/illiterate and differently-abled. Our proposal aims to overcome this limitation of and build a rich interface using voice and gesture based technologies for a multi-modal mobile interaction and computing.

The project would like to address how to seamlessly integrate voice and gesture technologies for building a rich multi-modal conversational interface, in order to understand the issues of fusion and interplay of voice and gesture technologies on a mobile device, so that literate, illiterate, and differently abled persons can use this rich multimodal interface for better communication, information access for services related to health care and entertainment.

 The specific research issues of interest include:

  • Multimodal grammars integrating voice and gestures for task completion.
  • Active learning of pronunciation and acoustic models for scalability and adaptability,and
  • Exploitation of syllabic and postpositional nature of Indian languages for speech recognition and synthesis.

                                        The Proposed Deliverables of this Project

Task 0-6 months 7-12 months 2nd Year 3rd Year
Data collection Conversational Hand-written Multimodal data
Differently-abled datasets socionity datasets
Languages   two one
Virtual assistant Voice mode Multimodal Tuning
Workshops 0 1 1
Courses 1 0 1
 Principal Investigators
Dr. Kishore Prahallad, IIIT Hyderabad kishore@iiit.ac.in Link to homepage
Dr. Y. Padma Sai, VNRVGIET padmasai_y@vnrvjiet.in Link to homepage
Dr. A. Jagan, BVRIT jagan.amgoth@bvrit.ac.in Link to homepage

Virtual Assistant for Mobile Devices using Voice and Gesture Technologies 


Posted in ITRA-Mobile Projects, Project, Virtual Assistant

|

Comments Off on Virtual Assistant for Mobile Devices using Voice and Gesture Technologies 

PhD Students

Virtual Assistant for Mobile Devices using Voice and Gesture Technologies 


Posted in ITRA-Mobile Projects, Project, Virtual Assistant

|

Comments Off on Virtual Assistant for Mobile Devices using Voice and Gesture Technologies 

 

Collaborators

Virtual Assistant for Mobile Devices using Voice and Gesture Technologies 


Posted in ITRA-Mobile Projects, Project, Virtual Assistant

|

Comments Off on Virtual Assistant for Mobile Devices using Voice and Gesture Technologies 

 

Institutions

IIIT HyderabadVNRVJIET HyderabadBVRIT Medak District

virtual assistant

For more details, check the project website.

                                                   Project Status
PhD Student(s) 4
Publications   8
Number of Workshop/Training(s)  0
Labs Created 0

 

Publications

Virtual Assistant for Mobile Devices using Voice and Gesture Technologies 


Posted in ITRA-Mobile Projects, Project, Virtual Assistant

|

Comments Off on Virtual Assistant for Mobile Devices using Voice and Gesture Technologies