Bias Evaluation in Search Platforms through Rank and Relevance Based Measures

Presenter: Gizem Gezici, new research fellow at KDD Lab

Abstract: Search engines decide what we see for a given search query. Since many people are exposed to information through search engines, it is fair to expect that search engines are neutral. However, search engine results do not necessarily cover all the viewpoints of a search query topic, and they can be biased towards a specific view since search engine results are returned based on relevance, which is calculated using many features and sophisticated algorithms where search neutrality is not necessarily the focal point. Therefore, it is important to evaluate the search engine results with respect to bias. In this seminar, we will firstly examine the stance (in support or against), as well as the ideological bias (conservative or liberal) in search results of two popular search engines with respect to controversial query topics such as abortion, medical marijuana, and gay marriage.In the second part of this seminar, we will investigate gender bias in online education. Students are increasingly using online materials to learn new subjects or to supplement their learning process in educational institutions. Nonetheless, online educational materials in terms of possible gender bias and stereotypes which may appear in different forms are yet to be investigated in the context of search bias in a widely-used search platform. Thus, as a first step towards measuring possible gender bias, we will analyse online educational videos of YouTube in terms of the perceived, i.e. from the viewer’s perspective, gender of their narrators. Then, we will evaluate possible perceived gender bias in ranked educational video search results returned by YouTube in response to queries related to STEM (Science, Technology, Engineering, and Mathematics) and NON-STEM fields of education. Lastly, we will also discuss our attempts for bias mitigation in the scope of perceived gender bias in YouTube.