Together with Lufthansa Technik (LHT), ZAL has been researching how the heart rate of passengers and cabin crew can be measured without contact during a flight to further improve safety on board (HOPEKI project, LuFo 6.1).
Optical methods are one way to measure heart rate. Usually, these methods are applied, for example, in smartwatches. The principle behind this is as follows: the beating of the heart causes a pulsatile blood flow in the vessels, i.e. there is a change in the blood volume in the vessels and capillaries. These changes in blood volume, caused by each heartbeat, produce visually perceptible changes in the upper layers of the skin. Depending on the amount of blood present in the vessels or capillaries, more or less light is absorbed or reflected by the skin surface as a result. These subtle changes can be detected by optical sensors. In turn, a subsequent signal analysis can determine the heart rate from the temporal changes in this signal.
The aim was now to make the optical measurement principle used in a smartwatch (or also in medical devices) contactless. For this purpose, a camera was used to record the changes visible on the skin surface, which are caused by the pulsatile blood flow. The camera-based measurement was carried out on the face, as the skin is clearly visible here in most cases. For this, different processing steps were necessary to determine the heart rate. After a face detection, certain regions within the face, which are particularly well supplied with blood, were automatically extracted. A median color value was then determined within these regions. This color value was determined for each image from a continuous video stream (approximately 20 frames per second). By analyzing the temporal changes of this color value, the heart rate could be determined.
As a result, it can be stated that a non-contact measurement of the heart rate inside the aircraft in real time is possible with an inexpensive camera and comparatively little computing power. For the practical implementation, the camera including the computing unit was integrated in the area above the passenger seats. It was shown that the measurement of the heart rate was possible under different lighting conditions as well as during (slight) head movements. Challenges arose, for example, in the case of strong head movements, fluctuations in brightness or masking of the face, e.g. by a mouth/nose protection. The vibrations caused by the engines in a real flight, which are also transmitted to the camera and the passenger, have no negative influence on this measurement principle or the results.
What is your opinion on image and signal processing or machine learning? Talk to us about the Intelligent Digital Cabin and give us a call:
Dr.-Ing. Johannes Passand
Technical Manager Electronic Cabin Systems
+49 40 248 595 106
