Benchmark multi-model/multi-view models.

Note

This page is a reference documentation. It only explains the function signature, and not how to use it. Please refer to the gallery for the big picture.

mmbench.workflow.barrier.benchmark_barrier_exp(dataset, datasetdir, configfile, outdir, downstream_name, dtype='full', n_coeffs=10)[source]¶

Compare the performance barrier interpolating the weights of any two pairs of intances of the same network and monitoring a common downstream task.

Parameters

dataset : str

the dataset name: euaims or hbn.

datasetdir : str

the path to the dataset associated data.

configfile : str

the path to the config file descibing the different models to compare. This configuration file is a Python (*.py) file with a dictionary named ‘_models’ containing the different model settings. Keys of this dictionary are the model names, each beeing described with a model getter function ‘get’ and associated kwargs ‘get_kwargs’, as weel as an evaluation function ‘eval’ and associated kwargs ‘eval_kwargs’. The getter and evaluation functions are defined in the ‘mmbench.model’ module.

outdir : str

the destination folder.

downstream_name : str

the name of the column that contains the downstream classification task.

dtype : str, default ‘full’

the data type: ‘complete’ or ‘full’.

n_coeffs : int, default 10

number of interpolation points.