plenoptic.models.OnOff

Note

This object is a torch.nn.Module. It therefore has all the methods and attributes from that class, even though they are not documented here (to avoid cluttering this page).

class plenoptic.models.OnOff(kernel_size, amplitude_ratio=1.25, pad_mode='reflect', pretrained=False, activation=<built-in function softplus>, apply_mask=False, cache_filt=False)

Bases: Module

On-off and off-on center-surround with contrast and luminance gain control.

Model is described in Berardino et al., 2017 [13] and online [14], where it is called OnOff.

Parameters:

• kernel_size (int | tuple[int, int]) – Shape of convolutional kernel.

• amplitude_ratio (float (default: 1.25)) – Ratio of center/surround amplitude. Applied before filter normalization.

• pad_mode (str (default: 'reflect')) – Padding for convolution.

• pretrained (bool (default: False)) – Whether or not to load model params from Berardino, 2018 [15]. See Notes for details.

• activation (Callable[[Tensor], Tensor] (default: <built-in function softplus>)) – Activation function following linear and gain control operations.

• apply_mask (bool (default: False)) – Whether or not to apply circular disk mask centered on the input image. This is useful for synthesis methods like Eigendistortions to ensure that the synthesized distortion will not appear in the periphery. See plenoptic.data.disk for details on how mask is created.

• cache_filt (bool (default: False)) – Whether or not to cache the filter. Avoids regenerating filt with each forward pass.

center_surround

2-channel (on-off and off-on) difference of Gaussians linear filter.

Type:

CenterSurround

luminance

2-channel Gaussian convolutional kernel used to normalize signal by local luminance.

Type:

Gaussian

contrast

2-channel Gaussian convolutional kernel used to normalize signal by local contrast.

Type:

Gaussian

luminance_scalar

Scale factor for luminance normalization.

Type:

torch.nn.parameter.Parameter

contrast_scalar

Scale factor for contrast normalization.

Type:

torch.nn.parameter.Parameter

Notes

These 12 parameters (standard deviations & scalar constants) were taken from Table 2, page 149 from Berardino, 2018 [15] and are the values used Berardino et al., 2017 [13]. Please use these pretrained weights at your own discretion.

References

[13] (1,2)

A Berardino, J Ballé, V Laparra, EP Simoncelli, Eigen-distortions of hierarchical representations, NeurIPS 2017; https://arxiv.org/abs/1710.02266

[14]

https://www.cns.nyu.edu/~lcv/eigendistortions/ModelsIQA.html

[15] (1,2)

A Berardino, Hierarchically normalized models of visual distortion sensitivity: Physiology, perception, and application; Ph.D. Thesis, 2018; https://www.cns.nyu.edu/pub/lcv/berardino-phd.pdf

Examples

>>> import plenoptic as po
>>> onoff_model = po.models.OnOff(31, pretrained=True, cache_filt=True)

Methods

display_filters([vrange, zoom, title, col_wrap])	Display convolutional filters of model.
forward(x)	Compute model response on input tensor.

display_filters(vrange='indep0', zoom=5.0, title=['linear filt on', 'linear filt off', 'luminance filt on', 'luminance filt off', 'contrast filt on', 'contrast filt off'], col_wrap=2, **kwargs)

Display convolutional filters of model.

Parameters:

• vrange (tuple[float, float] | str (default: 'indep0')) – Arguments for imshow, see its docstrings for details.

• zoom (float | None (default: 5.0)) – Arguments for imshow, see its docstrings for details.

• title (str | list[str] | None (default: ['linear filt on', 'linear filt off', 'luminance filt on', 'luminance filt off', 'contrast filt on', 'contrast filt off'])) – Arguments for imshow, see its docstrings for details.

• col_wrap (int | None (default: 2)) – Arguments for imshow, see its docstrings for details.

• **kwargs (Any) – Keyword args for imshow.

Return type:

PyrFigure

Returns:

fig – The figure containing the displayed filters.

Examples

>>> import plenoptic as po
>>> onoff_model = po.models.OnOff(31, pretrained=True, cache_filt=True)
>>> onoff_model.display_filters()
<PyrFigure ...>

(png, hires.png, pdf)

forward(x)

Compute model response on input tensor.

We use same-padding to ensure that the output and input shapes are matched.

Parameters:

x (Tensor) – The input tensor, should be 4d (batch, channel, height, width).

Return type:

Tensor

Returns:

y – Model response to input.

Examples

>>> import plenoptic as po
>>> onoff_model = po.models.OnOff(31, pretrained=True, cache_filt=True)
>>> img = po.data.einstein()
>>> y = onoff_model.forward(img)
>>> titles = ["Input image", "Output channel 0", "Output channel 1"]
>>> po.plot.imshow([img, y], title=titles)
<PyrFigure size...>

(png, hires.png, pdf)