A theory-based content-based image retrieval approach for capturing user preferences during query formulation
Authors
Artemi, Mahmoud Abolgasem AliIssue Date
2021-09Subjects
content-based image retrievaluser interface design
active learning paradigm
Vakkari’s theory
eye tracking
query formulation
Subject Categories::G440 Human-computer Interaction
Metadata
Show full item recordAbstract
Most image search systems rely on text-based retrieval. It is often challenging for users to describe their search intents by describing images using keywords; these may lead to unsatisfactory retrieval results containing irrelevant images with respect to the users’ search intents. To preserve the users’ intent visually and improve search performance, contentbased image retrieval (CBIR) has emerged. Since CBIR search uses the representation of visual features (such as colour, shape, and texture), it is built for users to express their intents more precisely. Although CBIR helps to cope with the ambiguity in text-based image search systems, sustained attention has been made to cope with two essential challenges in the CBIR system called the intention gap and semantic gap. The intention gap lies between user search intent and desired query, whilst the semantic gap refers to the difficulty of mapping high-level concepts to low-level image features. In an effort to solve both problems, most existing studies of CBIR system design focus on learning users’ information needs through relevance feedback at the result assessment stage only. However, in many CBIR systems, the underlying machine learning mechanisms need the users’ feedback at the query formulation stage to capture user intents for better training and search performance, which unfortunately is often not supported by the search interface design. The lack of support for the users’ query formulation through an effective CBIR interface has been a drawback for system performance and the users’ search satisfaction and experiences. We propose a new CBIR system design approach based on a multistagelinear model of information seeking named Vakkari’s model of the three-stage search process, which uses as a guide of interface design to capture user preferences in an early stage of the search process. The computational complexity is important since the process of user feedback and image matching is executed online, so to improve the system response to user’s feedback, we have applied a colour quantization algorithm called Improved Gray Scale (IGS) to reduce the colour intensities of images which aims to accelerate the image matching process. The designed interface support users to provide feedback at the query formulation stage through a user-centred interface. This interface enables the user to form and express their information needs by involving them to participate in the training phase of the active learning mechanism-based search model. The research focuses to investigate whether users find search functionalities or search mechanisms of the interface more efficient which influences user behaviour and perceptions. To evaluate the proposed CBIR system, two user studies were conducted in a lab-based setting using a screen-based eye tracker (Tobii Pro Nano) and Galvanic Skin Response (GSR) on the iMotions platform. The findings of both studies highlight the importance for the users to engage in all search stages of the search process, especially at the query formulation stage when the considered mechanism requires a training process, through a user-centred interaction design. To the best of our knowledge, this is the first research study in the context of content-based image retrieval (CBIR) that has investigated whether the multistage-linear model of information seeking can be employed as both an evaluation mechanism for a lab-based controlled experiment, and a guide of interface design. To this end, the overall aim of this research is to provide a principled investigation in exploring the synergistic effect between user involvements in system training over the exploratory search task and sense-making during the result assessment stage of CBIR.Citation
Artemi, M. (2021) 'A Theory-based Content-based Image Retrieval Approach for Capturing User Preferences during Query Formulation'. PhD thesis. University of Bedfordshire.Publisher
University of BedfordshireType
Thesis or dissertationLanguage
enDescription
A thesis submitted to the University of Bedfordshire, in partial fulfilment of the requirements for the degree of Doctor of PhilosophyCollections
The following license files are associated with this item:
- Creative Commons
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivatives 4.0 International