TAMAN - Multi-Source Video Domain Adaptation

Download this dataset

Temporal Attentive Moment Alignment Network (TAMAN)

A Novel Method and A New Benchmark Dataset for Multi-Source Video Domain Adaptation (MSVDA)


alt text


Multi-Source Domain Adaptation (MSDA) is a more practical domain adaptation scenario in real-world scenarios. It relaxes the assumption in conventional Unsupervised Domain Adaptation (UDA) that source data are sampled from a single domain and match a uniform data distribution. MSDA is more difficult due to the existence of different domain shifts between distinct domain pairs. When considering videos, the negative transfer would be provoked by spatial-temporal features and can be formulated into a more challenging Multi-Source Video Domain Adaptation (MSVDA) problem. In this paper, we address the MSVDA problem by proposing a novel Temporal Attentive Moment Alignment Network (TAMAN) which aims for effective feature transfer by dynamically aligning both spatial and temporal feature moments. TAMAN further constructs robust global temporal features by attending to dominant domain-invariant local temporal features with high local classification confidence and low disparity between global and local feature discrepancies. To facilitate future research on the MSVDA problem, we introduce comprehensive benchmarks, covering extensive MSVDA scenarios. Empirical results demonstrate a superior performance of the proposed TAMAN across multiple MSVDA benchmarks.

Structure of TAMAN

The structure of TAMAN is as follows:

alt text

The Daily-DA and Sports-DA datasets

There are very limited cross-domain benchmark datasets for VUDA and its variant tasks. For the few cross-domain datasets available such as UCF-HMDBfull for standard VUDA and HMDB-ARIDpartial for Partial Video Domain Adaptation (PVDA), the source domains are always constraint to be a single domain. To facilitate MSVDA research, we propose two sets of comprehensive benchmarks, namely the Daily-DA and the Sports-DA datasets. Both datasets cover extensive MSVDA scenarios and provide adequate baselines with distinct domain shifts to facilitate future MSVDA research. These datasets could also be utilized as comprehensive sets of cross-domain datasets for VUDA.

Sampled frames from Daily-DA and Sports-DA:

alt text

alt text

Benchmark Results

We tested our proposed TAMAN on both Daily-DA and Sports-DA, while comparing with previous domain adaptation methods. The results are as follows:

alt text

Meanwhile, both Daily-DA and Sports-DA can also be used for closed-set Video-based Unsupervised Domain Adaptation (VUDA). The results utilizing TRN as base network are as follows:

alt text

Data Download

CC BY 4.0

How to use the train/test list

Take the Daily->A settting in Daily-DA as an example, the train list should be “hmdb51_msda_train.txt”, “kinetics_daily_msda_train.txt”, “mit_msda_train.txt” and “arid_msda_train.txt” in the “Daily-DA” folder (decompressed), where the “arid_msda_train.txt” is utilized in an unsupervised fashion. That is, the label should not be loaded during training. For testing, the test list is simply “arid_msda_test.txt”.


Back to Project Page