ENACT-Heart -- ENsemble-based Assessment Using CNN and Transformer on Heart Sounds

arXiv:2502.16914 · cs.SD, cs.AI, eess.AS · Submitted 2025-02-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "ENACT-Heart -- ENsemble-based Assessment Using CNN and Transformer on Heart Sounds".

Jane: Here's the gist of the paper. This study introduces ENACT-Heart, a novel ensemble approach that leverages Convolutional Neural Networks (CNN) and Vision Transformers (ViT) through a Mixture of Experts (MoE) framework to achieve a classification accuracy of 97.52% for heart sounds, demonstrating enhanced diagnostic potential in cardiovascular health monitoring.

Tom: Model Architecture and Fusion The ENACT-Heart system is designed as an ensemble approach that combines the complementary strengths of CNN and ViT through a Mixture of Experts (MoE) framework.

Jane: This integration effectively leverages multiple image modalities derived from the same input data, enhancing diagnostic accuracy.

Tom: The system utilizes a ViT model to analyze medical time-series sound data by converting it into audiovisual representations.

Title and authors: Tom: So we’re looking at the summary of ENACT-Heart now, which basically says they put together a system using a mix of two different deep learning models to figure out what heart sounds are, and it got a classification accuracy of ninety-seven point fifty-two percent.

Jane: That's high. So what does that actually mean for someone listening right now who just hears their heart sound?

Tom: It means they aren't relying on just one type of analysis anymore; they’re combining the global understanding from a Vision Transformer with the local pattern recognition from a Convolutional Neural Network.

Lu: The key mechanism is this Mixture of Experts framework where a gating network decides which expert or combination of experts to use for each input, meaning it dynamically adjusts the weights based on what the sound clip actually looks like.

Meng: So instead of just running both models at once, the AI gets smart enough to pick the right tool for that specific sound. That sounds much more efficient for when you actually deploy this thing on a device.

Tom: Exactly. And they used specific data processing steps, like segmenting audio into five-second clips and adding some noise with Gaussian noise—nine extra noisy versions of each original file—to make sure the model wasn't just memorizing the clean recordings.

Jane: That makes sense for real-world use; if a recording is noisy or short, the system still has something to work with. And they even looked at common heart sound problems like murmurs and extrasystoles in their analysis.

Lu: The paper shows that this ensemble method beats both individual models: the CNN got ninety-five point forty-five percent accuracy, and the Vision Transformer got ninety-three point eighty-eight percent accuracy, but ENACT-Heart hit that ninety-seven point five two percent mark.

Meng: That gap is significant. It shows that just stacking models or using one model isn't enough; you need that smart combination to get those extra points of performance.

Tom: And the implication for me, as someone who ships models, is how they handled the different data representations—using spectrograms for the Transformer and centroid graphs for the CNN—to make sure both parts are feeding off each other correctly.

Jane: So if you only listen to this show, it means we’re moving toward tools that can give doctors or patients much more nuanced insights from simple recordings of their own heart sounds.

Lu: Right, and they're not just classifying a single sound; they're aiming for that continuous analysis capability where the system can monitor and analyze heart sounds over time from a smart wearable device.

Meng: That shifts the focus for me to how we optimize the model size so it actually runs on low-power edge devices instead of needing huge servers.

Tom: And Lalam is confirming this direction, saying that this combination improves performance in every general way and points toward those continuous monitoring systems.

Jane: It sounds like the next big thing isn't just getting a good number in a lab; it’s making it something you can wear on your wrist for constant health tracking.

The paper's summary: Tom: So ENACT-Heart isn't just about getting a high score; they’re actually suggesting improvements to how the system works, mainly focusing on making those experts work better together in real time.

Jane: What kind of improvements are we talking about? I mean, if it's already at ninety-seven point fifty-two percent accuracy, what's left to fix?

Lu: They’re pushing for a system where the gating network that decides which expert to use is dynamic. Instead of having fixed weights for the CNN and the Transformer, this network learns and adjusts those weights based on the specific characteristics of each input sound clip.

Meng: That sounds like it could save us a ton on computational resources because we only really pay attention to the relevant part of the model for that particular recording. That's what I need when shipping something to production.

Tom: Yeah, that dynamic weighting is crucial because it means the system can adapt instantly if the sound quality or type changes during continuous monitoring. It moves beyond a fixed ensemble approach.

Jane: So, this isn't just about having two models; it’s about giving them an intelligent way to collaborate on every single heart sound analysis.

Lu: Exactly. And they also talked about building this into a real-time cardiovascular health monitoring system that can continuously analyze data streaming in from smart wearable devices.

Meng: That’s the big goal, right? Continuous monitoring without needing some bulky equipment attached to the patient for every single check-in. That's where the practical value is.

Tom: They want to democratize access to diagnostics globally, so this technology could potentially be used by anyone with a wearable device for early detection of cardiovascular issues.

Jane: It sounds like they’re aiming for something that changes how we think about long-term heart health management, moving away from just episodic checkups.

Lu: And they acknowledge that while the setup is promising, validating those results across broader clinical datasets is still the next big step before it's truly ready for widespread medical use.

The paper's improvements: Tom: So to wrap things up, ENACT-Heart is an ensemble system using CNN and Transformer models that achieved ninety-seven point fifty-two percent accuracy for heart sound classification on medical data.

Jane: That’s a very high number when you think about diagnosing something as subtle as a heart murmur or an extra heartbeat. So what’s the real impact of this for us?

Lu: The main idea is that combining the strengths of both models through that dynamic gating network allows the system to be much more robust and accurate than either model could achieve on its own.

Meng: From my side, I see it as a strong proof-of-concept for how we can build smarter AI that adapts to messy, real-world input data instead of relying on perfect lab conditions.

Lalam: I'm confident in the direction here; the combination of ViT’s global context and CNN’s local features is a very solid way to handle complex audio patterns.

Jane: So what does this mean for someone just listening to the show? It means we're getting closer to tools that can give doctors or patients much more nuanced insights from simple recordings of their own heart sounds.

Tom: Right, and they’re pushing for that continuous monitoring aspect, meaning this technology could eventually be integrated into smart wearables for tracking heart health over time.

Lu: They are pointing toward the future where this kind of ensemble approach is used to create truly adaptive diagnostic systems that handle continuous data streams.

Meng: I just hope the engineering challenges of deploying a constantly adapting system on a low-power device are manageable because that’s where most projects fall apart.

Lalam: I'm sure they have the right direction in mind for tackling those real-time constraints, given how well the architecture is structured to handle different data modalities.

Jane: It sounds like the next big thing we should watch is how quickly researchers can move this from a high accuracy benchmark to something that actually runs reliably on consumer hardware.

Conclusion: Tom: So we’ve talked about ENACT-Heart, which is this ensemble system using CNN and Transformer models to figure out heart sounds, hitting ninety-seven point fifty-two percent accuracy on medical data.

Jane: That’s a very high number when you think about diagnosing something as subtle as a heart murmur or an extra heartbeat. So what does that actually mean for someone listening right now?

Lu: The main idea is combining the strengths of both models through that dynamic gating network allows the system to be much more robust and accurate than either model could achieve on its own.

Meng: From my side, I see it as a strong proof-of-concept for how we can build smarter AI that adapts to messy, real-world input data instead of relying on perfect lab conditions.

Lalam: Based on the literature and my analysis of the architecture described in ENACT-Heart -- Ensemble-based Assessment Using CNN and Transformer on Heart Sounds, I am quite sure that the proposed MoE ensemble method is highly effective at combining global context with local feature extraction.

Jane: So what does that mean for someone just listening to the show? It means we're getting closer to tools that can give doctors or patients much more nuanced insights from simple recordings of their own heart sounds.

Tom: Right, and they’re pushing for that continuous monitoring aspect, meaning this technology could eventually be integrated into smart wearables for tracking heart health over time.

Lu: They are pointing toward the future where this kind of ensemble approach is used to create truly adaptive diagnostic systems that handle continuous data streams.

Meng: I just hope the engineering challenges of deploying a constantly adapting system on a low-power device are manageable because that’s where most projects fall apart.

Lalam: I'm sure they have the right direction in mind for tackling those real-time constraints, given how well the architecture is structured to handle different data modalities.

Jane: It sounds like the next big thing we should watch is how quickly researchers can move this from a high accuracy benchmark to something that actually runs reliably on consumer hardware.

Jiho Han, Adnan Shaout

Industrial AI Lab · Department of Electrical and Computer Engineering, The University of Michigan - Dearborn

cs.SD, cs.AI, eess.AS

Submitted: 2025-02-24

Updated: 2026-11-04

Comments: 7 pages. Revised figures and tables, clarified fixed-weight late-fusion terminology, corrected a reference, moderated claims, and reorganized supporting material into appendices. Added an arXiv change log. No new experiments

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 90/100

The gist: This study introduces ENACT-Heart, a novel ensemble approach that leverages Convolutional Neural Networks (CNN) and Vision Transformers (ViT) through a Mixture of Experts (MoE) framework to achieve a…

Key concepts

Mixture of Experts (MoE)
A framework where a gating network dynamically selects the most relevant expert or combination of experts for each input. This allows the system to adaptively choose which model component is best suited to analyze a specific heart sound instance, improving overall prediction.
Vision Transformer (ViT)
A deep learning model used here to analyze medical time-series sound data by converting it into audiovisual representations. It excels at understanding global contextual information within the sound patterns, providing broad understanding of the input data.
Convolutional Neural Network (CNN)
A model trained using centroid graphs to extract robust, localized features from audio data. CNNs are effective at identifying specific, detailed patterns within the heart sounds, such as subtle murmurs or extra heart sounds.
Audiovisual Representations
The process of generating different formats from the original audio input for analysis. This includes creating spectrograms (visual frequency representations) and centroid graphs (localized feature summaries), which are used by both the ViT and CNN models.

Terminology

Summary

This study introduces ENACT-Heart, a novel ensemble approach that leverages Convolutional Neural Networks (CNN) and Vision Transformers (ViT) through a Mixture of Experts (MoE) framework to achieve a classification accuracy of 97. p. 0

Where the paper says it

Abstract— This study explores the application of Vision Transformer (ViT) principles in audio analysis, specifically focusing on heart sounds. This paper introduces ENACT-Heart – a novel ensemble approach that leverages the complementary strengths of Convolutional Neural Networks (CNN) and ViT through a Mixture of Experts (MoE) framework, achieving a remarkable classification accuracy of 97.

52% for heart sounds, demonstrating enhanced diagnostic potential in cardiovascular health monitoring. p. 0
Where the paper says it

88%) and CNN (95. p. 0

Where the paper says it

ENACT-Heart – ENsemble-based Assessment Using CNN and Transformer on Heart Sounds Jiho Han1 Industrial AI Lab SimPlatform Co. Affiliate Research Institute Geumcheon-gu, Seoul, Republic of Korea jihohan@simplatform.

45%), demonstrating the potential for enhanced diagnostic accuracy in cardiovascular health monitoring.

Model Architecture and Fusion

The ENACT-Heart system is designed as an ensemble approach that combines the complementary strengths of CNN and ViT through a Mixture of Experts (MoE) framework. This integration effectively leverages multiple image modalities derived from the same input data, enhancing diagnostic accuracy. p. 1

Where the paper says it

• We explored the feasibility of employing the Mixture of Experts (MoE) approach across different AI architectures, combining CNN and ViT. This integration effectively leverages multiple image modalities derived from the same input data, enhancing diagnostic accuracy.

The system utilizes a ViT model to analyze medical time-series sound data by converting it into audiovisual representations. p. 1
Where the paper says it

This integration effectively leverages multiple image modalities derived from the same input data, enhancing diagnostic accuracy. • We applied a ViT model to medical time-series sound data by converting it into audiovisual representations.

Furthermore, the CNN model is trained using centroid graphs to extract robust, localized features. p. 4
Where the paper says it

Data augmentation techniques, such as random noise addition, were applied to improve model robustness. The CNN model was trained using the centroid graphs.

The MoE approach dynamically selects the most relevant expert or combination of experts for each input instance via a gating network. p. 3
Where the paper says it

These experts make predictions independently based on their specialized knowledge. The key innovation of MoE lies in the gating network, which dynamically selects the most relevant expert or combination of experts for each input instance.

Data Preparation and Augmentation

The experimental setup involves several crucial data preprocessing steps to prepare the audio data for model training. The process begins by segmenting audio files into 5-second clips and zero-padding shorter recordings to standardize the input length. p. 3

Where the paper says it

Each audio file was segmented into 5-second clips. If an audio file was shorter than 5 seconds, it was zero-padded to meet the required input length.

Data augmentation is employed to increase the dataset size and introduce variability through Gaussian noise addition, generating 9 additional noisy versions for each original file. p. 3
Where the paper says it

Data Augmentation. To increase the dataset size and introduce variability, Gaussian noise was added to the audio files, generating 9 additional noisy versions for each original file.

These augmented audio data are then used to generate audiovisual representations consisting of spectrograms and centroid graphs. p. 3
Where the paper says it

The augmented audio data was included in the dataframe, and corresponding labels were updated to reflect the augmentation process. Audiovisual Image Data Generation Two types of visual representations were generated for each audio file: spectrograms and centroid graphs.

Performance Evaluation and Comparison

The performance evaluation compares the individual models against the ensemble method to determine superior predictive capabilities. Individual model results show that the CNN model achieved higher overall precision and demonstrated more balanced performance across all classes compared to ViT. p. 4

Where the paper says it

This, however, doesn’t mean the usage of ViT model is futile, as ViT might have identified characteristics that is not evident through the CNN model. The CNN model, on the other hand, achieved higher overall precision and demonstrated more balanced performance across all classes.

The ENACT-Heart ensemble method achieved the highest accuracy of 97. p. 4
Where the paper says it

The ensemble was created using an additive weighted approach, where different weights were tested to find the optimal combination. ENACT-Heart achieved the highest accuracy of 97.

52%, significantly outperforming both individual models, ViT (93. p. 4
Where the paper says it

ENACT-Heart achieved the highest accuracy of 97. 52%, significantly outperforming both individual models.

88%) and CNN (95.45%). This superior performance indicates the effectiveness of the ensemble approach in enhancing model robustness and generalization capabilities. p. 4
Where the paper says it

52%, significantly outperforming both individual models. The improvement in accuracy demonstrates the effectiveness of the ensemble approach, particularly in enhancing the model’s robustness and generalization capabilities.

Clinical Implications and Future Directions

The findings suggest that ENACT-Heart has significant promise for improving diagnostic accuracy in cardiovascular health monitoring. The technology can revolutionize the practice of medicine by enabling non-invasive, continuous monitoring of heart health through smart wearable devices. p. 6

Where the paper says it

Wearable devices equipped with advanced models like ENACT-Heart can bridge this gap by enabling non-invasive, continuous monitoring of heart health, thus providing timely and accurate diagnostics without the need for expensive and bulky equipment. This technology can revolutionize the practice of medicine in poorer regions, making high-quality healthcare more approachable and affordable.

This approach holds great potential for democratizing access to advanced medical diagnostics globally by providing timely and accurate diagnostics in underserved regions. p. 6
Where the paper says it

As wearable devices become more affordable and their usage more prevalent, the integration of sophisticated models like ENACT-Heart can play a crucial role in democratizing access to advanced medical diagnostics globally. In summary, the synergy between the ENACT-Heart model and smart wearable technology holds great potential for enhancing healthcare delivery, particularly in underserved regions.

The synergy between the ENACT-Heart model and smart wearable technology can lead to early detection of cardiovascular issues and prompt intervention. p. 6
Where the paper says it

The ability to monitor and analyze heart sounds continuously can lead to early detection of cardiovascular issues, prompt intervention, and ultimately, better health outcomes. As wearable devices become more affordable and their usage more prevalent, the integration of sophisticated models like ENACT-Heart can play a crucial role in democratizing access to advanced medical diagnostics globally.

Historical Context of Diagnostics

Cardiac diagnostics have evolved from manual palpation to modern AI integration. Early breakthroughs include Jean Baptiste de Senac establishing the connection between atrial fibrillation and mitral valve disease in the 1700s. p. 1

Where the paper says it

HISTORICAL BACKGROUND The diagnosis of heart conditions dates back to the early days of medicine, where physicians relied on palpation and pulse assessment to detect abnormalities. A significant breakthrough occurred in the 1700s when Jean Baptiste de Senac, physician to King Louis XV of France, established the connection between atrial fibrillation and mitral valve disease.

The invention of the stethoscope by Rene Laennec in 1816 marked a pivotal moment in cardiac diagnostics through his technique of ”mediate auscultation”. p. 1
Where the paper says it

The invention of the stethoscope by Rene Laennec in 1816 ´ marked a pivotal moment in cardiac diagnostics. Laennec introduced the technique of ”mediate auscultation” using his newly created paper acoustic device, allowing for more accurate detection of heart sounds and abnormalities [3].

These traditional methods have been augmented by electrocardiograms (ECGs) and other imaging modalities. p. 1
Where the paper says it

These early diagnostic methods, based on manual interpretation of heart sounds, evolved significantly over the centuries. With technological advancements, traditional auscultation has been augmented by electrocardiograms (ECGs) and other imaging modalities.

The integration of artificial intelligence has opened new avenues for the analysis of heart sounds, leading to more precise and efficient diagnostic tools. p. 1
Where the paper says it

With technological advancements, traditional auscultation has been augmented by electrocardiograms (ECGs) and other imaging modalities. In recent years, the integration of artificial intelligence (AI) and machine learning has opened new avenues for the analysis of heart sounds, leading to more precise and efficient diagnostic tools [4], [5].

Anomalies in Heart Sounds

Various anomalies are commonly detected during cardiac auscultation, including heart murmurs, extra heart sounds, and extrasystoles. Heart murmurs are produced by turbulent blood flow and can indicate underlying conditions such as valve defects or congenital heart defects. p. 1

Where the paper says it

Produced by turbulent blood flow strong enough to generate audible noise, these ”whooshing” sounds can be heard in various scenarios, including some healthy individuals. While innocent murmurs, also known as functional or benign murmurs, are typically harmless and not associated with structural heart abnormalities, pathologic murmurs indicate underlying conditions such as valve defects, congenital heart defects, or abnormal blood flow patterns.

Extra Heart Sounds refer to additional heart sounds beyond the normal ”lub-dub” pattern and may not be easily identified through other diagnostic tools like ultrasound. p. 1
Where the paper says it

While innocent murmurs, also known as functional or benign murmurs, are typically harmless and not associated with structural heart abnormalities, pathologic murmurs indicate underlying conditions such as valve defects, congenital heart defects, or abnormal blood flow patterns. Extra Heart Sounds refer to additional heart sounds beyond the normal ”lub-dub” pattern.

Extrasystole involves irregular heart rhythms that can signify underlying heart disease if detected early. p. 1
Where the paper says it

Detection of these sounds is crucial as they may not be easily identified through other diagnostic tools like ultrasound. Extrasystole involves irregular heart rhythms, typically presenting as extra or skipped heartbeats.

The paper details these anomalies in Table I. p. 1
Where the paper says it

A detailed explanation of these anomalies can be found in Table I. Organization of the Paper This paper is organized as follows: Section II covers the background of the study, providing the necessary theoretical foundation.

Related Methodologies in Cardiac Sound Analysis

The proposed method lies within the field of computer-assisted diagnosis (CAD) systems for heart disease. Existing CAD systems often focus on computer vision over audio AI due to advanced deep learning models. p. 2

Where the paper says it

These systems analyze cardiac data to identify abnormalities and patterns - especially indicators of certain heart diseases. However, the direction of the majority of these researches are pointed mainly toward computer vision over audio AI, mainly due to the advanced deep learning models available.

Related works have utilized CNNs for feature extraction and Data-efficient Image Transformer (DieT) variants for classification tasks through stacking. p. 2
Where the paper says it

For instance, Jumphoo et al. utilized a CNN for feature extraction and Data-efficient Image Transformer (DieT), a variant of the ViT model, for classification tasks through stacking [6].

However, the integration of multiple distinct AI models and modalities from the same sound inputs has not been attempted yet. p. 2
Where the paper says it

While these studies highlight the effectiveness of ViT individually, they do not explore the potential benefits of using an ensemble approach. Overall, the integration of multiple distinct AI models and modalities from the same sound inputs, as proposed in the ENACT-Heart using an MoE approach, has not been attempted yet.

Spectral Data Visualization Techniques

Researchers employ various techniques to gain insights into signal frequency content using spectral visualization. Spectrograms provide detailed representations of frequency spectra over time, revealing how the frequency composition changes temporally. p. 2

Where the paper says it

They offer detailed representations of frequency spectra over time, revealing how the frequency composition of a signal changes temporally. By plotting frequency on the vertical axis, time on the horizontal axis, and intensity or magnitude using color or brightness, spectrograms provide a comprehensive view of signal dynamics.

In contrast, spectral centroids offer a simplified summary of a signal’s frequency content by indicating the ”center of mass” or average frequency. p. 2
Where the paper says it

In contrast to the detailed temporalfrequency mapping provided by spectrograms, spectral centroids offer a simplified summary of a signal’s frequency content. The spectral centroid indicates the ”center of mass” or average frequency of a signal within each time frame.

The synergy in using spectrograms and spectral centroids with different models allows the MoE approach to combine detailed, global timefrequency information with robust, localized features. p. 3
Where the paper says it

By employing a MoE approach, the proposed model effectively combines these diverse representations. The spectrograms allow the model to capture detailed, global timefrequency information, while the spectral centroids and waveforms facilitate the extraction of robust, localized features.

Comparative Analysis of State-of-the-Art Models

The proposed ENACT-Heart demonstrates competitive performance when compared to other state-of-the-art models in heart sound classification. While some studies report higher accuracies, the balanced performance of ENACTHeart across multiple metrics underscores its reliability and robustness. p. 5

Where the paper says it

44% with ConvDeiT, and their precision, recall, and F1-score metrics are comparable to those of the ENACT-Heart. As summarized in Table III, while some studies report higher accuracies, the balanced performance of ENACTHeart across multiple metrics underscores its reliability and robustness in heart sound classification tasks.

The high accuracy and comprehensive performance metrics indicate the potential of the MoE ensemble method in enhancing diagnostic accuracy and reliability. p. 5
Where the paper says it

As summarized in Table III, while some studies report higher accuracies, the balanced performance of ENACTHeart across multiple metrics underscores its reliability and robustness in heart sound classification tasks. The high accuracy and comprehensive performance metrics indicate the potential of the MoE ensemble method in enhancing diagnostic accuracy and reliability in cardiovascular health monitoring.

The data augmentation techniques employed also played a key role in enhancing model robustness and performance. p. 5
Where the paper says it

9870 (PhysioNet Challenge) strengths and the potential benefits of using ensemble methods to achieve superior results. The data augmentation techniques employed also played a key role in enhancing model robustness and performance.

Conclusion on Ensemble Effectiveness

In conclusion, the combination of ViT and CNN models using an ensemble method improved the classification performance in every aspect in general. This study highlights the importance of evaluating individual models to identify their strengths and weaknesses. p. 5

Where the paper says it

CONCLUSION AND DISCUSSION In conclusion, the combination of ViT and CNN models using an ensemble method improved the classification performance in every aspect in general. This study highlights the importance of evaluating individual models to identify their TABLE III: Comparison of other State-of-the-Art MoE heart sound classification studies.

The findings can inform future research and development of advanced classification systems in the medical field. p. 5
Where the paper says it

The data augmentation techniques employed also played a key role in enhancing model robustness and performance. These findings can inform future research and development of advanced classification systems in the medical field.

The proposed ENACT-Heart model demonstrates significant promise in the field of heart sound classification. p. 5
Where the paper says it

These findings can inform future research and development of advanced classification systems in the medical field. The proposed ENACT-Heart model demonstrates significant promise in the field of heart sound classification, particularly when considered alongside the advancements in smart wearable devices.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Create an ensemble system that dynamically adjusts expert weights based on input data characteristics, as described by The gating network, often implemented as a neural network, learns to assign weights to the experts based on the input data. This allows for superior predictive performance compared to traditional fixed-weight ensembles by adaptively selecting the most relevant model contribution.

  2. Implement a system that processes audiovisual representations using both ViT and CNN models simultaneously, leveraging the strengths of both ViT and CNN models through the Mixture of Experts approach. This integration ensures that the analysis benefits from both global contextual understanding (ViT) and local pattern recognition (CNN).

  3. Develop a real-time cardiovascular health monitoring system capable of continuously analyzing heart sound data from smart wearable devices, as suggested by the conclusion regarding the ability to monitor and analyze heart sounds continuously. This enables early detection of cardiovascular issues without requiring expensive, bulky equipment.

Abstract

ENACT-Heart is a two-branch ensemble for heart-sound classification. A Vision Transformer (ViT) processes spectrogram images, while a convolutional neural network (CNN) processes visualizations of waveforms and spectral centroids derived from the same recordings. The branch predictions are combined by a weighted average with fixed, input-independent weights. On the PASCAL heart-sound dataset, the ensemble achieved an accuracy of 97.52%, compared with 93.88% for the ViT and 95.45% for the CNN. The ensemble achieved a macro-F1 of 0.94 and recall of 0.74 for extra heart sounds. On this dataset, late fusion improved accuracy and macro-F1 relative to both individual branches.

Sources

Related papers