Domain Expert Discovery Using Web Scraping from GitHub

Abstract

This paper presents a comprehensive system for Domain Expert Discovery using web scraping techniques to automatically identify, collect, and visualize expert profiles from GitHub. The proposed system extracts information such as GitHub usernames, repositories, skills, and follower counts, and detects LinkedIn URLs from user bios or repositories to enrich professional data. Extracted profiles are stored in MongoDB and displayed on a responsive web interface built using Node.js and React.js. The resulting system helps organizations, recruiters, and researchers find qualified experts in specific domains. Future work includes AI-based skill tagging and intelligent expert ranking.