plenoptic.validate.validate_metric#

plenoptic.validate.validate_metric(metric, image_shape=None, image_dtype=torch.float32, device='cpu')[source]#

Determine whether a metric can be used for MADCompetition synthesis.

In particular, this functions checks the following (with associated exceptions):

  • Whether metric is callable and accepts two tensors of shape image_shape as input (TypeError).

  • Whether metric returns a scalar when called with two tensors of shape image_shape as input (ValueError).

  • Whether metric returns a value less than 5e-7 when with two identical tensors of shape image_shape as input (ValueError). (This threshold was chosen because 1-SSIM of two identical images is 5e-8 on GPU).

Parameters:

  • metric (Module | Callable[[Tensor, Tensor], Tensor]) – The metric to validate.

  • image_shape (tuple[int, int, int, int] | None (default: None)) – Some models (e.g., the steerable pyramid) can only accept inputs of a certain shape. If that’s the case for model, use this to specify the expected shape. If None, we use an image of shape (1,1,16,16).

  • image_dtype (dtype (default: torch.float32)) – What dtype to validate against.

  • device (str | device (default: 'cpu')) – What device to place the test images on.

Raises:

  • TypeError – If metric cannot be called with two tensors of specified shape.

  • ValueError – If metric does not return a scalar or doesn’t return a value less than 5e-7 when given two identical tensors.

Examples

Check that 1-SSIM works:

>>> import plenoptic as po
>>> po.validate.validate_metric(lambda x, y: 1 - po.metric.ssim(x, y))

Check that SSIM doesn’t work (because SSIM=0 means that images are different, whereas we need metric=0 to mean identical):

>>> import plenoptic as po
>>> po.validate.validate_metric(po.metric.ssim)
Traceback (most recent call last):
ValueError: metric should return ...