Learning-based inference of longitudinal image changes: Applications in embryo development, wound healing, and aging brain
Abstract
Longitudinal imaging data are routinely acquired for health studies and patient monitoring. A central goal in longitudinal studies is tracking relevant change over time. Traditional methods remove nuisance variation with custom pipelines to focus on significant changes. In this work, we present a machine learning–based method that automatically ignores irrelevant changes and extracts the time-varying signal of interest. Our method, called Learning-based Inference of Longitudinal imAge Changes (LILAC), performs a pairwise comparison of longitudinal images in order to make a temporal difference prediction. LILAC employs a convolutional Siamese architecture to extract feature pairs, followed by subtraction and a bias-free fully connected layer to learn meaningful temporal image differences. We first showcase LILAC’s ability to capture key longitudinal changes by simply training it to predict the temporal ordering of images. In our experiments, temporal ordering accuracy exceeded 0.98, and predicted time differences were strongly correlated with actual changes in relevant variables (Pearson Correlation Coefficient r = 0.911 with embryo phase change, and r = 0.875 with time interval in wound healing). Next, we trained LILAC to explicitly predict specific targets, such as the change in clinical scores in patients with mild cognitive impairment. LILAC models achieved over a 40% reduction in root mean square error compared to baseline methods. Our empirical results demonstrate that LILAC effectively localizes and quantifies relevant individual-level changes in longitudinal imaging data, offering valuable insights for studying temporal mechanisms or guiding clinical decisions.
Article Details
Journal Info
Proceedings of the National Academy of Sciences
National Academy of Sciences
Authors (902)
Heejong Kim
Artificial Intelligence in Radiology
Batuhan K. Karaman
Artificial Intelligence in Radiology
Qingyu Zhao
Artificial Intelligence in Radiology
Alan Q. Wang
School of Electrical and Computer Engineering
Mert R. Sabuncu
Artificial Intelligence in Radiology
Michael Weiner
Leslie M. Shaw
Duygu Tosun
Pallavi Sachdev
Robert C. Green
Erin Drake
Tom Montine
Cat Conti
Rachel Nosheny
Diana Truran Sacrey
Juliet Fockler
Melanie J. Miller
Winnie Kwang
Chengshi Jin
Miriam Ashford
Derek Flenniken
Adrienne Kormos
Paul Aisen
Michael Rafii
Rema Raman
Gustavo Jimenez
Jennifer Salazar
Andrea Fidell
Virginia Boatwright
Justin Robison
Caileigh Zimmerman
Yuliana Cabrera
Sarah Walter
Taylor Clanton
Elizabeth Shaffer
Caitlin Webb
Lindsey Hergesheimer
Stephanie Smith
Sheila Ogwang
Olusegun Adegoke
Payam Mahboubi
Jeremy Pizzola
Cecily Jenkins
Laurel Beckett
Michael Donohue
Naomi Saito
Adam Diaz
Kedir Adem Hussen
Monica Rivera-Mindt
Hannatu Amaza
Mai Seng Thao
Shaniya Parkins
Omobolanle Ayo
Matt Glittenberg
Isabella Hoang
Kaori Kubo Germano
Joe Strong
Trinity Weisensel
Fabiola Magana
Lisa Thomas
Vanessa Guzman
Adeyinka Ajayi
Joseph Di Benedetto
Sandra Talavera
Clifford R. Jack
Joel Felmlee
Nick C. Fox
Paul Thompson
Charles DeCarli
Arvin Forghanian-Arani
Bret Borowski
Calvin Reyes
Caitie Hedberg
Chad Ward
Christopher Schwarz
Denise Reyes
Jeff Gunter
John Moore-Weiss
Kejal Kantarci
Leonard Matoush
Matthew Senjem
Prashanthi Vemuri
Robert Reid
Ian Malone
Sophia I. Thomopoulos
Talia M. Nir
Neda Jahanshad
Alexander Knaack
Evan Fletcher
Danielle Harvey
Duygu Tosun-Turgut
Stephanie Rossi Chen
Mark Choe
Paul A. Yushkevich
Sandhitsu Das
William Jagust
Susan Landau
Robert A. Koeppe
Gil Rabinovici
Victor Villemagne
Brian LoPresti
Richard J. Perrin
John Morris
Erin Franklin
Haley Bernhardt
Nigel J. Cairns
Lisa Taylor-Reinwald
Leslie Shaw
Edward B. Lee
Virginia M.Y. Lee
Magdalena Korecka
Magdalena Brylska
Yang Wan
National Pharmaceutical Engineering Center for Solid Preparation in Chinese Herbal Medicine
J.Q. Trojanowki
Arthur W. Toga
Karen Crawford
Scott Neu
Andrew J. Saykin
Kwangsik Nho
Tatiana M. Foroud
Taeho Jo
Shannon L. Risacher
Hannah Craft
Liana G. Apostolova
Kelly Nudelman
Kelley Faber
Zoë Potter
Kaci Lacy
Rima Kaddurah-Daouk
Li Shen
Jason Karlawish
Claire Erickson
Joshua Grill
Emily Largent
Kristin Harkins
Michael W. Weiner
Leon Thal
Zaven Kachaturian
Richard Frank
Peter J. Snyder
Neil Buckholtz
Susan Molchan
Zaven Khachaturian
Maria Carrillo
William Potter
Lisa Barnes
Marie Bernard
Hector González
Carole Ho
John K. Hsiao
Jonathan Jackson
Eliezer Masliah
Donna Masterman
Ozioma Okonkwo
Richard Perrin
Laurie Ryan
Nina Silverberg
Lisa Silbert
Jeffrey Kaye
Sylvia White
Aimee Pierce
Amy Thomas
Tera Clay
Daniel Schwartz
Gillian Devereux
Janet Taylor
Jennifer Ryan
Mike Nguyen
Madison DeCapo
Yanan Shang
Lon Schneider
Cynthia Munoz
Diana Ferman
Carlota Conant
Katherin Martin
Kristin Oleary
Sonia Pawluczyk
Elizabeth Trejo
Karen Dagerman
Liberty Teodoro
Mauricio Becerra
Madiha Fairooz
Sonia Garrison
Julia Boudreau
Yair Avila
James Brewer
Aaron Jacobson
Antonio Gama
Chi Kim
Emily Little
Jennifer Frascino
Nichol Ferng
Socorro Trujillo
Judith Heidebrink
Robert Koeppe
Steven MacDonald
Dariya Malyarenko
Jaimie Ziolkowski
James O’Connor
Nicole Robert
Suzan Lowe
Virginia Rogers
Ronald Petersen
Barbara Hackenmiller
Bradley Boeve
Colleen Albers
Connie Kreuger
David Jones
David Knopman
Hugo Botha
Jessica Magnuson
Jonathan Graff-Radford
Kerry Crawley
Michael Schumacher
Steven Smith
Tascha Helland
Val Lowe
Vijay Ramanan
Valory Pavlik
Jacob Faircloth
Jeffrey Bishop
Jessica Nath
Maria Chaudhary
Melissa Yu
Nathiel Pacini
Randall Barker
Regan Brooks
Ruchi Aggarwal
Lawrence Honig
Yaakov Stern
Akiva Mintz
Jonathan Cordona
Michelle Hernandez
Justin Long
Abbey Arnold
Alex Groves
Anna Middleton
Blake Vogler
Cierra McCurry
Connie Mayo
Cyrus Raji
Fatima S. Amtashar
Heather Klemp
Jasmina Kusuran
Jasmine Stewart
Jennifer Horenkamp
Julia Greeson
Kara Wever
Katie Vo
Kelly Larkin
Lesley Rao
Lisa Schoolcraft
Lora Gallagher
Madeline Paczynski
Maureen McMillan
Michael Holt
Nicole Gagliano
Rachel Henson
Renee LaBarge
Robert Swarm
Sarah Munie
Serena Cepeda
Stacey Winterton
TaNisha Wilson
Tanya Harte
Zach Bonacorsi
David Geldmacher
Amber Watkins
Brandi Barger
Bryan Smelser
Charna Bates
Cynthia Stover
Emily McKinley
Gregory Ikner
Haley Hendrix
Harold Matthew Cooper
Jennifer Mahaffey
Lindsey Booth Robbins
Loren Brown Ashley
Marissa Natelson-Love
Princess Carter
Veronika Solomon
Hillel Grossman
Alexandra Groome
Allison Ardolino
Anthony Kaplan
Faye Sheppard
Genesis Burgos-Rivera
Gina Garcia-Camilo
Joanne Lim
Judith Neugroschl
Kimberly Jackson
Kirsten Evans
Laili Soleimani
Mary Sano
Nasrin Ghesani
Sarah Binder
Xiomara Mendoza Apuango
Ajay Sood
Amelia Troutman
Kimberly Blanchard
Arlene Richards
Grace Nelson
Kirsten Hendrickson
Erin Yurko
Jamie Plenge
Victoria Rufo
Raj Shah
Ranjan Duara
Brendan Lynch
Cesar Chirinos
Christine Dittrich
Debbie Campbell
Diego Mejia
Gilberto Perez
Helena Colvee
Joanna Gonzalez
Josalen Gondrez
Joshua Knaack
Mara Acevedo
Maria Cereijo
Maria Greig-Custo
Michelle Villar
Morris Wishnia
Sheryl Detling
Warren Barker
Marilyn Albert
Abhay Moghekar
Barbara Rodzon
Corey Demsky
Gregory Pontone
Jim Pekar
Leonie Farrington
Martin Pomper
Nicole Johnson
Tolulope Alo
Martin Sadowski
Anaztasia Ulysse
Arjun Masurkar
Brittany Marti
David Mossa
Emilie Geesey
Emily Petrocca
Evan Schulze
Jennifer Wong
Joseph Boonsiri
Sunnie Kenowsky
Tatianne Martinez
Veronica Briglall
P. Murali Doraiswamy
Adaora Nwosu
Alisa Adhikari
Cammie Hellegers
Jeffrey Petrella
Olga James
Terence Wong
Thomas Hawk
Sanjeev Vaishnavi
Hannah McCoubrey
Ilya Nasrallah
Rachel Rovere
Jeffrey Maneval
Elizabeth Robinson
Francisco Rivera
Jade Uffelman
Martha Combs
Patricia O’Donnell
Sara Manning
Richard King
Alayne Nieto
Amanda Glueck
Anjana Mandal
Audrie Swain
Bethanie Gamble
Beverly Meacham
Denece Forenback
Dorothy Ross
Elizabeth Cheatham
Ellen Hartman
Gary Cornell
Jordan Harp
Laura Ashe
Laura Goins
Linda Watts
Morgan Yazell
Prabin Mandal
Regan Buckler
Sylvia Vincent
Triana Rudd
Oscar Lopez
Ann Arlene Malia
Caitlin Chiado
Cary Zik
Kathleen Savage
Linda Fenice
MaryAnn Oakley
Paige C Tacey
Sarah Berman
Sarah Bowser
Stephen Hegedus
Xanthia Saganis
Anton Porsteinsson
Abigail Mathewson
Asa Widman
Bridget Holvey
Emily Clark
Esmeralda Morales
Iris Young
James Ruszkiewicz
Kevin Hopkins
Kimberly Martin
Nancy Kowalski
Rebecca Hunt
Roberta Calzavara
Russell Kurvach
Stephen D’Ambrosio
Gaby Thai
Beatriz Vides
Brigit Lieb
Catherine McAdams-Ortiz
Cyndy Toso
Ivan Mares
Kathryn Moorlach
Luter Liu
Maria Corona
Mary Nguyen
Melanie Tallakson
Michelle McDonnell
Milagros Rangel
Neetha Basheer
Patricia Place
Romina Romero
Steven Tam
Trung Nguyen
Abey Thomas
Alexander Frolov
Alka Khera
Amy Browning
Courtney Dawson
Dana Mathews
Elaine Most
Elizeva Phillips
Lynn Nguyen
Maribel Nunez
Matalin Miller
Matthew R. Jones
Natalie Martinez
Rebecca Logan
Roderick McColl
Sari Pham
Tiffani Fox
Tracey Moore
Allan Levey
Abby Brown
Andrea Kippels
Ashton Ellison
Casie Lyons
Chadwick Hales
Cindy Parry
Courtney Williams
Elizabeth McCorkle
Guy Harris
Heather Rose
Inara Jooma
Jahmila Al-Amin
James Lah
James Webster
Jessica Swiniarski
Latasha Chapman
Laura Donnelly
Lauren Mariotti
Mary Locke
Phyllis Vaughn
Rachael Penn
Sallie Carpentier
Samira Yeboah
Sarah Basadre
Sarah Malakauskas
Stefka Lyron
Tara Villinger
Terra Burney
Jeffrey Burns
Ala Abusalim
Alexandra Dahlgren
Alexandria Montero
Anne Arthur
Heather Dooly
Katelynn Kreszyn
Katherine Berner
Lindsey Gillen
Maria Scanlan
Mercedes Madison
Nicole Mathis
Phyllis Switzer
Ryan Townley
Samantha Fikru
Samantha Sullivan
Ella Wright
Maryam Beigi
Anthony Daley
Ashley Ko
Brittney Luong
Glen Nyborg
Jessica Morales
Kelly Durbin
Lauren Garcia
Leila Parand
Lorena Macias
Lorena Monserratt
Maya Farchi
Pauline Wu
Robert Hernandez
Thao Rodriguez
Mayo Clinic
Neill Graff-Radford
A’llana Marolt
Anton Thomas
Deborah Aloszka
Ercilia Moncayo
Erin Westerhold
Gregory Day
Kandise Chrestensen
Mary Imhansiemhonehi
Sanna McKinzie
Sochenda Stephens
Sylvia Grant
Jared Brosch
Amy Perkins
Aubree Saunders
Debra Silberberg Kovac
Heather Polson
Isabell Mwaura
Kassandra Mejia
Katherine Britt
Kathy King
Kayla Nichols
Kayley Lawrence
Lisa Rankin
Martin Farlow
Patricia Wiesenauer
Robert Bryant
Scott Herring
Sheryl Lynch
Skylar Wilson
Traci Day
William Korst
Christopher van Dyck
Adam Mecca
Alyssa Miller
Amanda Brennan
Amber Khan
Audrey Ruan
Carol Gunnoud
Chelsea Mendonca
Danielle Raynes-Goldfinger
Elaheh Salardini
Elisa Hidalgo
Emma Cooper
Erawadi Singh
Erin Murphy
Jeanine May
Jesse Stanhope
Jessica Lam
Julia Waszak
Kimberly Nelsen
Kimberly Sacaza
Mayer Joshua Hasbani
Meghan Donahue
Ming-Kai Chen
Nicole Barcelos
Paul Eigenberger
Robin Bonomi
Ryan O’Dell
Sarah Jefferson
Siddharth Khasnavis
Stephen Smilowitz
Susan DeStefano
Susan Good
Terry Camarro
Vanessa Clayton
Yanis Cavrel
YuQuan Lu
Howard Chertkow
Howard Bergman
Chris Hosein
Sandra Black
Anish Kapadia
Aparna Bhan
Benjamin Lam
Christopher Scott
Gillian Gabriel
Jennifer Bray
Ljubica Zotovic
Maria Samira Gutierrez
Mario Masellis
Marjan Farshadi
Maurylette Gui
Meghan Mitchell
Rebecca Taylor
Ruby Endre
Zhala Taghi-Zada
Robin Hsiung
Carolyn English
Ellen Kim
Eugene Yau
Haley Tong
Laura Barlow
Lauren Jennings
Michele Assaly
Paula Nunes
Tahlee Marian
Andrew Kertesz
John Rogers
Dick Trost
Dylan Wint
Charles Bernick
Donna Munic
Ian Grant
Aaliyah Korkoyah
Ali Raja
Allison Lapins
Caila Ryan
Jelena Pejic
Kailey Basham
Leena Lukose
Loreece Haddad
Lucas Quinlan
Nathaniel Houghtaling
Carl Sadowsky
Walter Martinez
Teresa Villena
Brigid Reynolds
Angelica Forero
Carolyn Ward
Emma Brennan
Esteban Figueroa
Giuseppe Esposito
Jessica Mallory
Kathleen Johnson
Kathryn Turner
Katie Seidenberg
Kelly McCann
Margaret Bassett
Melanie Chadwick
Raymond Scott Turner
Robin Bean
Saurabh Sharma
Gad Marshall
Aferdita Haviari
Alison Pietras
Bradley Wallace
Catherine Munro
Gladiliz Rivera-Delpin
Hadley Hustead
Isabella Levesque
Jennifer Ramirez
Karen Nolan
Kirsten Glennon
Mariana Palou
Michael Erkkinen
Nicole DaSilva
Pamela Friedman
Regina M. Silver
Ricardo Salazar
Roxxanne Polleys
Scott McGinnis
Seth Gale
Tia Hall
Tuan Luu
Steven Chao
Emmeline Lin
Jaila Coleman
Kevin Epperson
Minal Vasanawala
Alireza Atri
Amy Rangel
Brittani Evans
Candy Monarrez
Carol Cline
Carolyn Liebsack
Daniel Bandy
Danielle Goldfarb
Debbie Intorcia
Jennifer Olgin
Kelly Clark
Kelsey King
Kylee York
Marina Reade
Michael Callan
Michael Glass
Michaela Johnson
Michele Gutierrez
Molly Goddard
Nadira Trncic
Parichita Choudhury
Priscilla Reyes
Serena Lowery
Shaundra Hall
Sonia Olgin
Stephanie de Santiago
Alyssa Ton
Amanda Jimenez
Andrew Ellison
Anh Tran
Brandon Anderson
Della Carter
Donna Veronelli
Jason Weller
Jennifer Johns
Jesse Mez
Jessica Harkins
Ina Hoti
Jane Mwicigi
Alexa Puleio
Michael Alosco
Olivia Schultz
Mona Lauture
Eric Steinberg
Ridiane Denis
Ronald Killiany
Sarab Singh
Steven Lenio
Wendy Qiu
Ycar Devis
Thomas Obisesan
Andrew Stone
Debra Ordor
Ifreke Udodong
Immaculata Okonkwo
Javed Khan
Jillian Turner
Kyliah Hughes
Oshoze Kadiri
Charles Duffy
Ariana Moss
Katherine Stapleton
Maria Toth
Marianne Sanders
Martin Ayres
Melissa Hamski
Parianne Fatica
Paula Ogrocki
Sarah Ash
Stacy Pot
Doris Chen
Andres Soto
Costin Tanase
David Bissig
Hafsanoor Vanya
Heather Russell
Hitesh Patel
Hongzheng Zhang
Kelly Wallace
Kristi Ayers
Maria Gallegos
Martha Forloines
Meghan Sinn
Queennie Majorie S Kahulugan
Richard Isip
Sandra Calderon
Talia Hamm
Michael Borrie
T-Y Lee
Rob Bartha
Sterling Johnson
Sanjay Asthana
Cynthia M. Carlsson
Allison Perrin
Pierre Tariot
Adam Fleisher
Stephanie Reeder
Horacio Capote
Allison Emborsky
Anna Mattle
Bela Ajtai
Benjamin Wagner
Bennett Myers
Daryn Slazyk
Delaney Fragale
Erin Fransen
Heather Macnamara
Jonathan Falletta
Joseph Hirtreiter
Laszlo Mechtler
Megan King
Michael Asbach
Michelle Rainka
Richard Zawislak
Scott Wisniewski
Stephanie O’Malley
Tatiana Jimenez-Knight
Todd Peehler
Traci Aladeen
Vernice Bates
Violet Wenner
Wisam Elmalik
Douglas W. Scharre
Arun Ramamurthy
Soumya Bouchachi
Maria Kataki
Rawan Tarawneh
Brendan Kelley
Dzintra Celmins
Alicia Leader
Chris Figueroa
Heather Bauerle
Katlynn Patterson
Michael Reposa
Steven Presto
Tuba Ahmed
Wendy Stewart
Hartford Hosp
Godfrey D. Pearlson
Karen Blank
Karen Anderson
Robert B. Santulli
Eben S. Schwartz
Jeff Williamson
Alicia Jessup
Andrea Williams
Crystal Duncan
Abigail O’Connell
Karen Gagnon
Ezequiel Zamora
James Bateman
Freda Crawford
Deb Thompson
Eboni Walker
Jennifer Rowell
Mikell White
Phillip Ledford
Sarah Bohlman
Susan Henkle
Joseph Bottoms
Lena Moretz
Bevan Hoover
Michael Shannon
Samantha Rogers
Wendy Baker
William Harrison
Chuang-Kuo Wu
Alexis DeMarco
Ava Stipanovich
Daniel Arcuri
Jan Clark
Jennifer Davis
Kerstin Doyon
Marie Amoyaw
Mauro Veras Acosta
Ronald Bailey
Scott Warren
Terry Fogerty
Victoria Sanborn
Meghan Riddle
Stephen Salloway
Paul Malloy
Stephen Correia
Charles Windon
Morgan Blackburn
Howard J. Rosen
Bruce L. Miller
Amanda Smith
Ijeoma Mba
Jenny Echevarria
Juris Janavs
Emily Roglaski
Meagan Yong
Rebecca Devine
Hamid Okhravi
Edgardo Rivera
Teresa Kalowsky
Caroline Smith
Christina Rosario
Joseph Masdeu
Richard Le
Maushami Gurung
Marwan Sabbagh
Angelica Garcia
Micah Ellis Slaughter
Nadeen Elayan
Skieff Acothley
Nunzio Pomara
Raymundo Hernando
Vita Pomara
Chelsea Reichert
Olga Brawman-Mintzer
Allison Acree
Arthur Williams
Campbell Long
Rebecca Long
Paul Newhouse
Sydni Jenee Hill
Amy Boegel
Sudha Seshadri
Amy Saklad
Floyd Jones
William Hu
V. Sotelo
Yaneicy Gonazalez Rojas
Jacobo Mintzer
Crystal Flynn Longmire
Kenneth Spicer