Multimodal Knowledge Extraction and Question Answering in Farming

Student Team: Brian Mak, Ignacy Tymoteusz Debicki, Juan Sebastian Reyes, and Sriram Mahesh

Project Mentor: Professor Yi Zhang (UCSC), Dr. Yueqi Li (X) and Dr. Kezhen Chen (X)

We propose a novel visual long-form question answering system for the farming domain, an area of research with no previous baselines or significant work. We scrape and curate our datasets from openly available farming information available through reliable farming forums and agricultural encyclopedias. We propose two methods to design our visual question answering agent with one being a Knowledge graph-based solution and another being an extractive question answering system designed using a DensePhrase retrieval based reading comprehension agent.