Item type:Thesis, Open Access

Attention-based multitask learning for image analysis

Thumbnail Image

Files

Primary dds.pdf (25.41 MB)

Date

2026-05-21

Authors

Publisher

DOI

Supervisors

Profile Picture
Item type:Person,
Profile Picture
Item type:Person,
Ewerth, Ralph

Abstract

In recent years, research in the field of artificial intelligence has shifted away from highly specialized models designed for specific tasks toward generalist architectures that learn more complex representations from large amounts of data. Within this paradigm, multitask learning (MTL) models aim to share knowledge across multiple tasks in order to utilize data more efficiently as well as achieve transfer and synergy effects between related tasks. This dissertation systematically investigates MTL models in the field of image analysis. It focuses on the use of attention mechanisms in MTL models to improve feature exchange between shared and task-specific layers, as well as between task-specific layers of different tasks. The dissertation examines architectural and algorithmic design principles through three challenging practical applications: organ tablature music notation recognition, classification of wildlife taxonomies in camera trap images, and analysis of sample parameters in electron microscopy diffraction patterns. In particular, the dissertation presents new, tailored attention-based MTL approaches for these different use cases and evaluates them empirically in terms of accuracy, data efficiency, and practical applicability. Thus, this dissertation contributes to the development of high-performance image analysis systems designed for demanding application contexts.

Review

Metadata

Contributors
Supervisor:
Dates
Issued: 2026-05-21
Relations
Is based on: https://open.uni-marburg.de/handle/openumr/10220Is based on: https://github.com/umr-ds/mammal-bird-camera-trap-recognitionIs based on: https://github.com/umr-ds/deep-pacbedIs based on: https://github.com/umr-ds/tf-model-trainingIs based on: https://open.uni-marburg.de/handle/openumr/10131Is based on: https://data.uni-marburg.de/handle/dataumr/556Is based on: https://open.uni-marburg.de/handle/openumr/10220Is based on: https://doi.org/10.5334/tismir.77Is based on: https://doi.org/10.25366/2023.149Is based on: https://inf-cv.uni-jena.de/wordpress/wp-content/uploads/2023/09/Talk-8-Daniel-Schneider.pdfIs based on: https://doi.org/10.1111/gcb.17056Is based on: https://doi.org/10.1049/cvi2.12294Is based on: https://doi.org/10.1016/j.ultramic.2026.114412
DOI
https://doi.org/10.17192/openumr/752
Faculty
FB12:Mathematik und Informatik
Language
en
Keywords
Deep LearningImage AnalysisMultitask Learning
DFG-subjects
4.43-04 - Künstliche Intelligenz und Maschinelle Lernverfahren
Funding
Funding Organisations:
License
Attribution 4.0 International
URI
https://open.uni-marburg.de/handle/openumr/10565
show more
Schneider, Daniel: Attention-based multitask learning for image analysis. : 2026-05-21. DOI: https://doi.org/10.17192/openumr/752.

License

Attribution 4.0 International
Except where otherwise noted, this item's license is described as Attribution 4.0 International

Related Items

isAuthorOfThesis

Profile Picture
Item type:Person,

Funding Organisation

OrgUnit Logo
Item type:Organizational Unit,
OrgUnit Logo
Item type:Organizational Unit,

isSupervisorOfThesis

Profile Picture
Item type:Person,
Profile Picture
Item type:Person,
Ewerth, Ralph

Dataset

Thumbnail Image
Item type:Research Data, Open Access
Deep PACBED: Multitask Analysis of PACBED Images Using Deep Neural Networks
(Philipps-Universität Marburg)
Schneider, Daniel
;
Jonas Scheunert
;
Heimes, Damien
;
Firoozabadi, Seyyed Salehedin
;
Volz, Kerstin
This dataset contains the trained models and the test data set presented in the paper "Deep PACBED: Multitask Analysis of PACBED Images Using Deep Neural Networks". In this paper, we present a deep learning approach to automatically analyzing sample parameters from position-averaged convergent beam electron diffraction (PACBED) images. While previous machine learning approaches analyze each sample parameter individually, our work investigates multitask deep neural networks that simultaneously predict multiple sample parameters, outperforming models trained on a single parameter. We provide downloads for two trained models in the Tensorflow2 SavedModel format (https://www.tensorflow.org/guide/saved_model) in the archive files "model_EfficientNetV2S_branching.tar" (a branching neural network based on the EfficientNetV2 architectures) and "model_EfficientNetV2S_MTAN.tar" (our proposed adaptation of the multitask attention network (MTAN) to the EfficientNetV2 architecture). A script to load and run the models can be found in our Git-Repository: https://github.com/umr-ds/Deep-PACBED/. The archive file "data_experimental.tar" contains our experimental test dataset consisting of 1050 PACBED images. The images are organized into separate folders based on the different materials and crystallographic orientations (010 and 110). Each folder contains a "full.csv"-file that contains the metadata and labels for the images in the "img" subfolder, that are stored as .png-files. The "md_split" folder contains the annotations split into .csv files for the training and test subsets used in our work.