Overview
In the Cardiac tutorial, we will use KaleCardiac, a Python library for multimodal cardiovascular machine learning built on PyKale. KaleCardiac supports multiple data modalities, including cardiac imaging, physiological signals, and clinical data. Its API follows a modular pipeline architecture, with each module named according to its functionality:
loaddata → prepdata → model → pipeline → evaluate → interpret
(access) (transform) (learn) (train) (metrics) (explain)
The library is organised around two features of cardiac data. Data are easy to collect but labels are expensive, so a model first learns from unlabelled data and is then fine-tuned on a labelled cohort. Modalities also go missing in practice, so models keep working when an input is absent.
The tutorial notebooks
Each notebook builds a published model from raw DICOM files using the KaleCardiac API. You will find both under Cardiac Overview in the sidebar.
Chest X-ray and ECG |
12-lead ECG |
|
|---|---|---|
Input data (Modality) |
Chest X-ray and 12-lead ECG |
12-lead ECG, with each lead as separate modality |
Model |
An encoder and a decoder for each modality; the two encodings are fused by a product of experts (PoE) into one shared latent representation |
An encoder for each of the 12 leads and one decoder shared by all of them; the 12 encodings are fused by hierarchical modality experts (HiME) into one shared latent representation |
Pretrained on |
Paired 12-lead ECGs and chest X-rays |
12-lead ECGs |
Fine-tuned on |
Both modalities, or 12-lead ECG or chest X-rays |
Six limb lead ECG, or 12-lead ECG |
Missing modalities |
12-lead ECG or chest X-rays |
Any ECG leads |
Notebook |
What you will do
Both tutorials follow the same five steps:
Set up: install KaleCardiac and download the tutorial data.
Pretrain: the model learns from the data without any labels.
Fine-tune: a classifier is trained on top of the model using labelled data.
Evaluate: the model is tested on subjects it has not seen.
Interpret: you see which inputs, and which parts of them, the predictions relied on.
Each tutorial runs in Google Colab, which provides the GPU it uses for training.