Novel Representation Learning Technique Using Graphs For Performance Analytics
Most publicly available datasets are in a tabular format. It is one of the common data formats used for machine learning (ML) applications, especially in HPC, where models solve regression problems, such as predicting the execution time. Existing ML techniques leverage the correlations among features given tabular datasets, disregarding any relationship between the samples. Moreover, the success of the downstream analysis techniques depends on how well information is extracted from the raw features. For high-quality embeddings, existing methods rely on extensive feature engineering and preprocessing steps, which come at a high cost and require a human in the loop. To fill these two gaps, we propose a novel idea of transforming performance data into graphs to leverage the advancement of graph neural network-based (GNN) techniques in capturing complex relationships between features and samples. In contrast to other ML application domains such as social networks, the graph is not given; instead, we need to build it. To address this gap, we propose graph building methods where nodes represent samples, and the edges are automatically inferred iteratively based on the similarity between the features in the samples. We evaluate the effectiveness of the generated embeddings from GNNs based on how well they make even a simple feed-forward neural network perform for regression tasks compared to other state-of-the-art representation learning techniques. Our evaluation demonstrates that even with up to 25% random missing values for each dataset, our method outperforms commonly used graph and deep neural network (DNN)-based approaches and achieves up to 51.77% improvement in MSE loss over the DNN baseline.
Machine learning, Tabular data, Graph neural network, Deep learning, Deep neural networks
Ramadan, T. (2022). Novel representation learning technique using graphs for performance analytics (Unpublished thesis). Texas State University, San Marcos, Texas.