What We Learned:
We collaborated with research groups and institutions on new ways to provide fact-checkers with better tools to debunk image manipulation. Here are some challenges and learnings we’ve discovered during our testing, thanks to the valuable feedback from our testing partners.
The Challenge:
To develop machine learning models that are capable of detecting certain types of detection, you need to train the model using examples of images with that type of manipulation. During Alpha testing, we found that fact-checkers and journalists are often tasked with debunking images that are underrepresented in our training sets and therefore the detectors aren’t always able to accurately identify manipulations in these types of images. Some of the tricky cases we’ve observed include images that are screenshots of other screenshots and images that have been severely downsampled (taking a high-definition, large image and making it small) or reformatted (for example, changing the image format from JPEG to PNG).
The Approach:
Identifying these gaps in the training set has allowed us to focus on sourcing example images that can be used to train existing models to be able to accurately detect these cases, as well as source additional detectors to cover these gaps. We hope that in doing so we’ll be able to help improve these detectors, as well as contribute back to the broader industry.
The Challenge:
Many fact-checkers deal with low-resolution and small images, coming from social media and instant messengers. This poses unique challenges to the detection technology, which is generally more accurate when images are in their original format and quality.
The Approach:
We are integrating an image auto-upgrading process, powered by TinEye, a popular reverse image search provider, which takes original images and finds larger and/or better quality versions of them in an effort to ensure the best image possible is analyzed by the detectors.
We heard from our Alpha testers that they need to better understand the relative strengths and weaknesses of each detector. They want to know: which type of manipulation is each detector good for and which manipulation types does this detector not help with? As a result, we are reworking our user interface to provide clearer explanations on individual detector performance and different detector results.