Artificial intelligence and video as a resource to timely discover anomalies in premature babies

Due to an increased risk of various problems concerning growth, movement and development, premature babies are in need of special care. According neonatologist and professor of Pediatrics at the UMC Groningen, Arie Bos, it is important for early detection to discover possible anomalies on time in these premature babies, to minimize the consequences at a later age. In a movement analysis based on video images, such anomalies could be better assessed with the assistance of artificial intelligence.

At the University Medical Center Groningen (UMCG) 80 to 120 premature babies are admitted anually to the hospital from the provinces Groningen, Friesland, Overijssel and Drenthe. The babies end up on the Intensive Care Neonatology, which is the only intensive care unit for neonates out of eight hospitals in the Northern Netherlands. Due to a premature birth of ten weeks or more before the due date, these babies need special care. When the neonates are no longer in need of intensive care, they are transferred to one of the eight regional hospitals. There, the care is taken over by a pediatrician and the child is closer to the parents.

Periodic check

Periodic checks are of great importance in the case of an extreme preterm. Parent and child will visit the outpatient neonatology clinic of the UMCG during prebooked appointments to discover potential areas of concern in the development at an early stage. This consists of five moments in total, the first of which takes place when the baby is six months old and the last taking place when the child is eight years old.

The NeoLifeS cohort

To learn more about the development and most sufficient treatment of premature babies, the need arose for a central database of all the hospitals. In 2016, Bos together with his colleagues started NeoLifeS, a cohort with the purpose to identify problems and risk factors, and to improve the care for prematures. Premature babies are more at risk of various issues with growth, movement and development – including spasticity.

Within this cohort, data is collected on health and development issues of premature babies. The database contains information on the brain, lungs, eyes, respiration, the gastrointestinal system, infections, the placenta and on the start and course of the pregnancy of mother and child. Bos: “By systematically collecting and assessing clinical data of all the hospitals, of completed questionnaires by parents, and of movement patterns of the baby at three months past the calculated age, we can improve our intensive care for premature babies even further. After all, we want a bright and healthy future for these vulnerable kids.”

Since the start in 2016, after receiving permission from the parents, all clinical data of prematures has been collected from the moment of birth and stored in one databank. At present, the databank contains the data on 340 babies that were born before 30 weeks and/or weighed less than 1000 grams.

Movement analysis with own smartphone

Babies are often allowed to go home with the new parents if they are as old as they would originally be on the mother’s due date. This almost always occurs from one of the regional hospitals. Afterwards, it is essential that measurements are put in place to record the movements of the baby. Throug these measurement moments the baby is monitored for, amongst other things, spasticity. Spasticity occurs regularly and is often caused by a brain haemorrhage as a complication of preterm birth.

At the moment, spasticity is often only discovered after eighteen months. In the past this occured under the watchful eye of the specialist at the hospital, however, now, it can take place in a home environment, as the baby’s movements can be viewed with a smartphone. This new situation results in less stress for both parent and child, which ultimately provides a more reliable image.

Based on the video images, it is possible to determine whether there is a normal development or a potential anomaly as soon as three motngs. In this case, a rehabilitation specialist can be quickly called in for issues regarding arm and hand functions as a consequence of spasticity and these effects can be minimized when the child is older.

Timely recognition of certain patterns

The recorded video images are subsequently sent to the UMCG, where the NeoLifeS-team starts an analysis. The researchers watch approximately six to eight videos an hour, whereby it only takes 5 to 10 minutes of video to determine whether there is a case of normal or abnormal movement patterns and whether there is a need for closer examination. If it is suspected that there is a higher risk of a deviation in the motor development (especially spasticity), a consultation with the parents and the regional pediatrician is advised to refer to a rehabilitation center.

Bos explains: “The movement patterns at the age of three months is extremely important. With children that develop normally, you will see small, moderate speed, dancelike movements of the entire body, so in the shoulders, arms, hips, legs, torso and neck; then here, then there. Children who have a spastic movement disorder later on, do not display these movements at all. This knowledge has existed for a while, but only in the last few years we have discovered that by starting targeted therapy early, we can greatly improve the future results of children with spasticity.”

Technology as an essential factor

Neolook Solutions supports NeoLifeS with the development and expansion of the used movement analysis, which is internationally known as the General Movement Assessment (GMA). Marco D’Agata, Managing Director at Neolook Solutions: “UMCG is the national academic expertise center for the General Movement Assessment. If we want the GMA to be accessible for those thousands of children who are at risk every year in the Netherlands, just like in other countries, than we better work together with existing parties such as NeoLifeS.''

Neolook thinks ahead: where previously the specialist received the parents in the hospital or the nurse came by the house to record the video, it is now possible to virtually visit the parents. A livestream provides the nurse or the specialist with a direct view of the child. This takes less time and causes less stress for both the parent and the child. The video is then safely stored at the UMCG for the team of NeoLifeS to watch and analyse the video at a suitable moment.

Innovation with Artificial Intelligence

The next step in the process is to apply artificial intelligence. By visualizing the movements in the video with so-called ‘key points’ (key points which together form a wire figure of the child), potential deviations can be recognized by artificial intelligence software. Artificial intelligence makes it possible to automatically detect certain patterns in the movement of new-born babies. The application of the abovementioned form of artificial intelligence in the movement analysis of NeoLifeS can enable the specialists whom assess the movements to be more efficient and better supported in the assessment process.

The results from the recording are graphically displayed for the specialist, with any peculiarities being highlighted. Thus, the specialist can immediately investigate possible anomalies. The application of artificial intelligence in the movement analysis therefore supports the specialists in their tasks and speeds up the assessment process.

                          Keypoints in movement analysis graphically displayed

D’Agata: “You cannot use just any livestream. Parents need to be coached live, because the quality has to be good. Then, we can overlay the 23 key points on the small body, mapping simple and complex movements for the specialist.”

At the moment, NeoLifeS works on the direct application of AI on livestream videos in an international consortium. It is therefore possible to act more rapidly, leading to earlier detection of potential anomalies in premature babies.

More about cohorts and biobanks

Currently, there are 175 cohorts and biobanks at the UMCG. These cohorts and biobanks collect data over extended periods, as well as body materials for future medical scientific research. The Cohort and Biobank Coordination Hub (CBCH) unites all these cohorts and biobanks, supports researchers and stimulates new research and cooperations.