Evaluating approaches to improve upon a Leap Motion-based hand-gesture recognition system
AuthorsChase, Stephen Ricardo
Subject Categories::G440 Human-computer Interaction
MetadataShow full item record
AbstractThe research proposed in this thesis aims to utilise feedback gained from user testing to evaluate and present viable approaches to improve the self-made Marionette Project while maintaining its core principles. These improvements should primarily allow for users to identify hand gestures efficiently while minimising the number of incorrect hand gestures performed. Analysing various studies throughout the period of the initial Marionette Projects conception and implementation. It was noted that research in human machine and human computer interaction focused on a few prevalent topics, the most common being sign language, and generic gesture recognition. The implementation of these predominate topics focused typically on one of two implementation types; vision or image-based approaches as seen in Cho et al.’s research which looked into implementing a low-cost vision-based gesture recognition system based on the FPGA approach (Cho et al. 2012) or the device-based approaches utilised in Khambaty et al.’s conference paper into “Cost Effective portable system for sign language gesture recognition”. While both studies look at implementing “low-cost” or “cost-effective” solutions they both tackle it from different standpoints. Khambaty et al.’s study looks at it from a monetary perspective presenting the developed system as a cheaper means of providing daily communication as opposed to the cost of hiring an interpreter (Khambaty et al. 2008), which while successful, still puts the product out of range of the general consumer. While Cho et al.’s study focuses on the reduction of computation costs by process the recognition of gestures through the use of the FPGA approach (Cho et al. 2012). This focus on one section of “cost” lowering in implementations has left room for research that provides both a monetary reduction allowing for the implementation to be consumer-friendly and a computational reduction allowing for faster and quicker recognition while still maintain accuracy. Additionally, with a larger focus being placed on visual based implementation, but solely in the realms of sign language and generic gesture recognition as a means for human to human communication it provides a gap in the field to test the plausibility of these implementation types for other uses, like machine control. As such, the initial Marionette Project aimed to find a cost-effective means of producing firstly, an effective but innovate hand gesture recognition system that could be utilised to control a range of robotics but in particular a mechatronic hand. The Leap Motion controller was implemented into the project, to test the viability of a low-cost consumer-grade product as a means to manipulate robotics. Utilising the Leap Motion controller also provided notable innovation as most published studies incorporating the Leap Motion Controller focused almost exclusively on the identification of various forms of sign language. In the thesis, three crucial feedback points garnered from the external testing process in the original Marionette Project, and are presented and utilised to shape the work implemented throughout this research in the form of minimum viable requirements listed below: 1) improving upon the accuracy of the hand gestures recognised by users through the use of real-time gesture confirmation system. 2) mitigating the amount of incorrect hand gestures performed when stopping the system. 3) Allowing for the support of more dexterous robotics though more complexed gestures In the thesis four approaches main approaches are presented, the first of which looks into the first minimum viable required, while the second, third and fourth approaches are created and evaluated as a means to fulfill the second and third minimum requirement points. By utilising the Spiral methodology, the implementation of each approach primarily followed the pattern of: Planning, Risk Analysis, Engineering and finally an Evaluation phase. The planning phase looked at the original feedback provided by the user as well as any relevant iterations implemented prior, to detail aims and requirements for the current iteration/spiral. After which, additional research was then carried out into hardware, software components as well as, additional published research papers. After this stage, the implementation or engineering phase would then be carried out. This primarily would look to implement each identified requirement for the iteration. Once these requirements were implemented the evaluation step would then be performed. The evaluation process consisted of two parts; an internal evaluation and then an external evaluation. Internal evaluations focused on developer testing, and consisted of standard logic, user flow and selected edge case testing. If no issues were found in this testing process the second stage of evaluations, external evaluations would then take place. External evaluations saw the implemented work tested by volunteer users under given scenarios, to generate more user results and feedback. However, in the event that problems were found during the internal evaluation process depending on the severity of the problem an additional mini-iteration could be added to an existing iteration as seen in section 4.3.4 or in the case were larger problems are identified and the approach needs to be altered an entirely new iteration with the listed spiral approach steps would be carried out as seen with approaches, three and four in the thesis. From the approaches detailed in the thesis, the first approach provided a viable means to improve upon the accuracy of hand gestures recognised by users, though the incorporation of real-time textual confirmation appearing onscreen while the user interacts with the system. The incorporation of this implementation showed an average increase of 28% in gestures recognised and identified by the user. Additionally, the fourth approach detailed in the thesis provided a means to improve upon limiting the amount of incorrect gestures performed by the user showing an overall 20% decrease in the number of incorrect reported hand gestures. It was concluded that the first approach presented an ideal implementation for clear improvements for the recognition of performed hand gestures and showed a 28% average increase in hand gesture recognised and identified by users. Additionally, the fourth approach was seemingly well-received by testing participants as a means to limit the number of incorrect gestures being performed when users removed their hand from the Leap Motions field of vision lowering reports of incorrect gesturing cause by the system by 20%. However, while both approaches implemented within this research was viable for the completion of the first two feedback points provided by users from initial testing, the third point was not achieved. As such, it would be ideal to provide extra research to find a plausible and novel solution.
CitationChase, S.R. (2020) 'Evaluating approaches to improve upon a Leap Motion-based hand-gesture recognition system'. MSc By Research thesis. University of Bedfordshire.
PublisherUniversity of Bedfordshire
TypeThesis or dissertation
DescriptionA thesis submitted to the University of Bedfordshire, in fulfilment of the requirements for the degree of MSc by Research thesis
The following license files are associated with this item:
- Creative Commons
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivatives 4.0 International