Visual Analysis of Printed Illustrations using Computer Vision

Workshop ID:

WT-07

Workshop Title:

Visual Analysis of Printed Illustrations using Computer Vision

Organizers:

Giles Bergel, Abhishek Dutta

Time (in JST and UTC):

July 25 19:00-22:30 (JST)

July 25 10:00-13:30 (UTC)

expected no. of participants:

50

Description:

This tutorial will address the use of computer vision in studying printed illustrations, based on over a decade of collaborations between Oxford’s Visual Geometry Group and book and art historians. The tutorial will present a complete processing pipeline for printed illustrations, covering illustration detection, matching, comparison and classification using machine learning. No prior knowledge of computer vision or programming experience is assumed, but the tutorial will also support technically capable users. Participants will gain an understanding of the state of the art in computer vision in this domain, and learn how to make their own images searchable.

Aim of the workshop/tutorial:

While digital humanities researchers have many tools for extracting and processing text from printed documents, there are fewer options for the computational analysis of their visual elements. This half-day tutorial is designed to address the needs of such researchers. Participants will step through four applications using case studies based on early Western printed books, with reference also to nineteenth-century Indian print. No knowledge of these materials is assumed. The tutorial will also address critical and operational issues such as data capture; bias in machine learning; user experience; and good practice in research reproducibility, software citation and accreditation.

Outline:

The tutorial will present a four-stage pipeline for analysing printed illustrations.

1. Illustration Detection using a pre-trained object detector model, in conjunction with the List Annotator (LISA) tool, which is used to review and refine the detections.

2. Visual image search and grouping, using VGG Image Search Engine (VISE) which allows visual search of collections of images using images as search queries.

3. Image Comparison, using software which allows researchers to investigate fine differences between illustrations

4. Visual classification using VGG Image Classifier (VIC), to semantically classify images in a dataset, either through keyword searches or by images as queries