plenoptic.models.LinearNonlinear

Note

This object is a torch.nn.Module. It therefore has all the methods and attributes from that class, even though they are not documented here (to avoid cluttering this page).

class plenoptic.models.LinearNonlinear(kernel_size, on_center=True, amplitude_ratio=1.25, pad_mode='reflect', pretrained=False, activation=<built-in function softplus>, cache_filt=False)

Bases: Module

Linear-Nonlinear model.

This model applies a difference of Gaussians filter followed by an activation function.

Model is described in Berardino et al., 2017 [4] and online [5], where it is called LN.

Parameters:

• kernel_size (int | tuple[int, int]) – Shape of convolutional kernel.

• on_center (bool (default: True)) – Dictates whether center is on or off; surround will be the opposite of center (i.e. on-off or off-on).

• amplitude_ratio (float (default: 1.25)) – Ratio of center/surround amplitude. Applied before filter normalization.

• pad_mode (str (default: 'reflect')) – Padding for convolution.

• pretrained (bool (default: False)) – Whether or not to load model params from Berardion, 2018 [6]. See Notes for details.

• activation (Callable[[Tensor], Tensor] (default: <built-in function softplus>)) – Activation function following linear convolution.

• cache_filt (bool (default: False)) – Whether or not to cache the filter. Avoids regenerating filt with each forward pass.

center_surround

Difference of Gaussians filter.

Type:

CenterSurround

Notes

These 2 parameters (standard deviations) were taken from Table 2, page 149 from Berardino, 2018 [6] and are the values used in Berardino et al., 2017 [4]. Please use these pretrained weights at your own discretion.

References

[4] (1,2)

A Berardino, J Ballé, V Laparra, EP Simoncelli, Eigen-distortions of hierarchical representations, NeurIPS 2017; https://arxiv.org/abs/1710.02266

[5]

https://www.cns.nyu.edu/~lcv/eigendistortions/ModelsIQA.html

[6] (1,2)

A Berardino, Hierarchically normalized models of visual distortion sensitivity: Physiology, perception, and application; Ph.D. Thesis, 2018; https://www.cns.nyu.edu/pub/lcv/berardino-phd.pdf

Examples

>>> import plenoptic as po
>>> ln_model = po.models.LinearNonlinear(31, pretrained=True, cache_filt=True)

Methods

display_filters([vrange, zoom, title])	Display convolutional filter of model.
forward(x)	Compute model response on input tensor.

display_filters(vrange='indep0', zoom=5.0, title='linear filter', **kwargs)

Display convolutional filter of model.

Parameters:

• vrange (tuple[float, float] | str (default: 'indep0')) – Arguments for imshow, see its docstrings for details.

• zoom (float | None (default: 5.0)) – Arguments for imshow, see its docstrings for details.

• title (str | list[str] | None (default: 'linear filter')) – Arguments for imshow, see its docstrings for details.

• **kwargs (Any) – Keyword args for imshow.

Return type:

PyrFigure

Returns:

fig – The figure containing the image.

Examples

>>> import plenoptic as po
>>> ln_model = po.models.LinearNonlinear(31, pretrained=True, cache_filt=True)
>>> ln_model.display_filters()
<PyrFigure ...>

(png, hires.png, pdf)

forward(x)

Compute model response on input tensor.

We use same-padding to ensure that the output and input shapes are matched.

Parameters:

x (Tensor) – The input tensor, should be 4d (batch, channel, height, width).

Return type:

Tensor

Returns:

y – Model response to input.

Examples

>>> import plenoptic as po
>>> ln_model = po.models.LinearNonlinear(31, pretrained=True, cache_filt=True)
>>> img = po.data.einstein()
>>> y = ln_model.forward(img)
>>> titles = ["Input image", "Output"]
>>> po.plot.imshow([img, y], title=titles)
<PyrFigure size...>

(png, hires.png, pdf)