AI summaries can warp the memories of people who are shown them, a study has found.
The new research shows not only that artificial intelligence systems commonly wrongly summarise things they are shown – but that those mistakes can then trick people into having the wrong information.
The study comes amid increasing use of AI summaries for everything from transcribed meetings to crucial settings such as analysing police footage. Researchers from Georgetown University Medical Center looked to understand how those summaries could change people’s understanding of the truth.
First, the researchers examined how good AI systems are actually summarising information, and found that they repeatedly produced misinformation. The most common error found was that the summaries would leave out critical details from the thing they were summarising – with researchers finding that across different prompts and models, the system left out 51.6 per cent of important information.
They then examined how those summaries might affect the memory and understanding of the people who relied on them. The researchers discovered that if participants were shown an AI summary that misleadingly characterised the events of an incident they had actually previously seen, they would not be able to accurately recall those events.
That happened even when people believed the summary was AI or human, the researchers found. And it did not seem to depend on whether people trusted or were familiar with AI.
In the new research, 331 participants watched an animated video that showed a car approaching a junction, at which there would be a sign telling the driver to either stop or go. The car then turns at the junction and hits a pedestrian who is walking across the road.
Researchers then waited 24 to 48 hours and gave those participants an AI-generated summary that was either accurate or had one wrong detail. Participants were then asked to recall the event, and suggest who was at fault – but those who had read the misleading AI summary were significantly less likely to recall it correctly, the study found.
The researchers behind the study noted that such summaries are being used in a wide range of applications, including reviewing police footage. And they warned that it suggested that such uses could be misleading – even if there was no malicious intent from anyone involved in that process.
