Penalty Functionals¶

Module: pycsou.func.penalty

Repository of common penalty functionals.

Norms

`L1Norm`(dim)	\(\ell_1\)-norm, \(\Vert\mathbf{x}\Vert_1:=\sum_{i=1}^N \|x_i\|\).
`SquaredL1Norm`(dim[, prox_computation])	\(\ell^2_1\)-norm, \(\Vert\mathbf{x}\Vert^2_1:=\left(\sum_{i=1}^N \|x_i\|\right)^2\).
`L2Norm`(dim)	\(\ell_2\)-norm, \(\Vert\mathbf{x}\Vert_2:=\sqrt{\sum_{i=1}^N \|x_i\|^2}\).
`SquaredL2Norm`(dim)	\(\ell^2_2\)-norm, \(\Vert\mathbf{x}\Vert^2_2:=\sum_{i=1}^N \|x_i\|^2\).
`LInftyNorm`(dim)	\(\ell_\infty\)-norm, \(\Vert\mathbf{x}\Vert_\infty:=\max_{i=1,\ldots,N} \|x_i\|\).
`L21Norm`(dim, groups)	\(\ell_{2,1}\)-norm, \(\Vert\mathbf{x}\Vert_{2,1}:=\sum_{g=1}^G \sqrt{ \sum_{i\in\mathcal{G}_g} \|x_i\|^2}\,.\)
`QuadraticForm`(dim[, linop])	Quadratic form \(\mathbf{x}^\ast \mathbf{L} \mathbf{x}\).

Balls

`L1Ball`(dim, radius)	Indicator function of the \(\ell_1\)-ball \(\{\mathbf{x}\in\mathbb{R}^N: \\|\mathbf{x}\\|_1\leq \text{radius}\}\)
`L2Ball`(dim, radius)	Indicator function of the \(\ell_2\)-ball \(\{\mathbf{x}\in\mathbb{R}^N: \\|\mathbf{x}\\|_2\leq \text{radius}\}\)
`LInftyBall`(dim, radius)	Indicator function of the \(\ell_\infty\)-ball \(\{\mathbf{x}\in\mathbb{R}^N: \\|\mathbf{x}\\|_\infty\leq \text{radius}\}\)

Convex Sets

`ImagLine`(dim)	Indicator function of the imaginary line \(j\mathbb{R}\).
`NonNegativeOrthant`(dim)	Indicator function of the non negative orthant (positivity constraint).
`RealLine`(dim)	Indicator function of the real line \(\mathbb{R}\).
`Segment`(dim[, a, b])	Indicator function of the segment \([a,b]\subset\mathbb{R}\).

Information Theoretic

`LogBarrier`(dim)	Log barrier, \(f(\mathbf{x}):= -\sum_{i=1}^N \log(x_i).\)
`ShannonEntropy`(dim)	Negative Shannon entropy, \(f(\mathbf{x}):= \sum_{i=1}^N x_i\log(x_i).\)

class L2Norm(dim: int)[source]¶

Bases: pycsou.func.base.LpNorm

\(\ell_2\)-norm, \(\Vert\mathbf{x}\Vert_2:=\sqrt{\sum_{i=1}^N |x_i|^2}\).

Examples

>>> x = np.arange(10)
>>> norm = L2Norm(dim=x.size)
>>> norm(x)
16.881943016134134
>>> tau = 1.2; np.allclose(norm.prox(x, tau=tau),np.clip(1 - tau / norm(x), a_min=0, a_max=None) * x)
True
>>> lambda_ = 3; scaled_norm = lambda_ * norm; scaled_norm(x)
50.645829048402405
>>> np.allclose(scaled_norm.prox(x, tau=tau),np.clip(1 - tau * lambda_ / norm(x), a_min=0, a_max=None) * x)
True

Notes

The \(\ell_2\)-norm is a strictly-convex but non differentiable penalty functional. Solutions to \(\ell_2\)-penalised convex optimisation problems are usually non unique and very smooth. The proximal operator of the \(\ell_2\)-norm can be found in [ProxAlg] section 6.5.1.

__init__(dim: int)[source]¶

Parameters: dim (int) – Dimension of the domain.

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

class SquaredL2Norm(dim: int)[source]¶

Bases: pycsou.core.functional.DifferentiableFunctional

\(\ell^2_2\)-norm, \(\Vert\mathbf{x}\Vert^2_2:=\sum_{i=1}^N |x_i|^2\).

Examples

>>> x = np.arange(10)
>>> norm = SquaredL2Norm(dim=x.size)
>>> norm(x)
285.00000000000006
>>> np.allclose(norm.gradient(x), 2 * x)
True
>>> lambda_=3; scaled_norm = lambda_ * norm
>>> scaled_norm(x)
855.0000000000002
>>> np.allclose(scaled_norm.gradient(x), 2 * lambda_ *  x)
True
>>> Gmat = np.arange(100).reshape(10, 10)
>>> G = DenseLinearOperator(Gmat, is_symmetric=False)
>>> weighted_norm = norm * G
>>> np.allclose(weighted_norm.gradient(x), 2 * Gmat.transpose() @ (Gmat @ x))
True

Notes

The \(\ell^2_2\) penalty or Tikhonov penalty is strictly-convex and differentiable. It is used in ridge regression. Solutions to \(\ell^2_2\)-penalised convex optimisation problems are unique and usually very smooth.

__init__(dim: int)[source]¶

Parameters: dim (int) – Dimension of the domain.

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

jacobianT(x: Union[numbers.Number, numpy.ndarray]) → Union[numbers.Number, numpy.ndarray][source]¶: Gradient of the squared L2 norm at x.

L2Ball(dim: int, radius: numbers.Number) → pycsou.func.base.IndicatorFunctional[source]¶

Indicator function of the \(\ell_2\)-ball \(\{\mathbf{x}\in\mathbb{R}^N: \|\mathbf{x}\|_2\leq \text{radius}\}\)

It is defined as:

\[\begin{split}\iota(\mathbf{x}):=\begin{cases} 0 \,\text{if} \,\|\mathbf{x}\|_2\leq \text{radius},\\ \, +\infty\,\text{ortherwise}. \end{cases}\end{split}\]

Parameters

dim (int) – Dimension of the domain.
radius (Number) – Radius of the \(\ell_2\)-ball.

Returns

Indicator function of the \(\ell_2\)-ball.

Return type

py:class:pycsou.core.functional.IndicatorFunctional

Examples

>>> x1 = np.arange(10); x2 = x1 / np.linalg.norm(x1)
>>> radius=10; ball = L2Ball(dim=x1.size, radius=radius)
>>> ball(x1), ball(x2)
(inf, 0)
>>> np.allclose(ball.prox(x1,tau=1), proj_l2_ball(x1, radius=radius)), np.linalg.norm(ball.prox(x1,tau=1))
(True, 10.0)
>>> np.allclose(ball.prox(x2,tau=1), x2)
True

Notes

The \(\ell_2\)-ball penalty is convex and proximable. It is a constrained variant of the \(\ell_2\)-norm penalty. The proximal operator of the \(\ell_2\)-ball indicator is the projection onto the \(\ell_2\)-ball (see [ProxAlg] Section 1.2).

See also

L2BallLoss(), L2Norm, py:class:pycsou.class.penalty.SquaredL2Norm.

class L1Norm(dim: int)[source]¶

Bases: pycsou.func.base.LpNorm

\(\ell_1\)-norm, \(\Vert\mathbf{x}\Vert_1:=\sum_{i=1}^N |x_i|\).

Examples

>>> x = np.arange(10)
>>> norm = L1Norm(dim=x.size)
>>> norm(x)
45
>>> tau=1.2; np.allclose(norm.prox(x, tau=tau),soft(x,tau=tau))
True
>>> lambda_=3; scaled_norm = lambda_ * norm; scaled_norm(x)
135
>>> np.allclose(scaled_norm.prox(x, tau=tau),soft(x,tau=tau * lambda_))
True

Notes

The \(\ell_1\)-norm penalty is convex and proximable. This penalty tends to produce non unique and sparse solutions. The proximal operator of the \(\ell_1\)-norm is provided in [ProxAlg] Section 6.5.2.

__init__(dim: int)[source]¶

Parameters: dim (int) – Dimension of the domain.

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

soft(x: Union[numbers.Number, numpy.ndarray], tau: numbers.Number) → Union[numbers.Number, numpy.ndarray][source]¶: Soft thresholding operator (see soft() for a definition).

class SquaredL1Norm(dim: int, prox_computation: str = 'sort')[source]¶

Bases: pycsou.core.functional.ProximableFunctional

\(\ell^2_1\)-norm, \(\Vert\mathbf{x}\Vert^2_1:=\left(\sum_{i=1}^N |x_i|\right)^2\).

Examples

>>> x = np.arange(10)
>>> norm = SquaredL1Norm(dim=x.size, prox_computation='sort')
>>> norm(x)
2025
>>> norm2 = SquaredL1Norm(dim=x.size, prox_computation='root')
>>> np.allclose(norm.prox(x, tau=1),norm2.prox(x, tau=1))
True

Notes

The \(\ell^2_1\)-norm penalty is strictly-convex and proximable. This penalty tends to produce a unique and sparse solution. Two alternative ways of computing the proximal operator of the \(\ell^2_1\)-norm are provided in [FirstOrd] Lemma 6.70 and [OnKerLearn] Algorithm 2 respectively.

See also

L1Ball(), L1Norm, SquaredL1Loss().

__init__(dim: int, prox_computation: str = 'sort')[source]¶

Parameters

dim (int) – Dimension of the domain.
prox_computation (str, optional) – Algorithm for computing the proximal operator: ‘root’ uses [FirstOrd] Lemma 6.70, while ‘sort’ uses [OnKerLearn] Algorithm 2 (faster).

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

prox(x: Union[numbers.Number, numpy.ndarray], tau: numbers.Number) → Union[numbers.Number, numpy.ndarray][source]¶: Proximal operator, see pycsou.core.functional.ProximableFunctional for a detailed description.

L1Ball(dim: int, radius: numbers.Number) → pycsou.func.base.IndicatorFunctional[source]¶

Indicator function of the \(\ell_1\)-ball \(\{\mathbf{x}\in\mathbb{R}^N: \|\mathbf{x}\|_1\leq \text{radius}\}\)

It is defined as:

\[\begin{split}\iota(\mathbf{x}):=\begin{cases} 0 \,\text{if} \,\|\mathbf{x}\|_1\leq \text{radius},\\ \, +\infty\,\text{ortherwise}. \end{cases}\end{split}\]

Parameters

dim (int) – Dimension of the domain.
radius (Number) – Radius of the \(\ell_1\)-ball.

Returns

Indicator function of the \(\ell_1\)-ball.

Return type

py:class:pycsou.core.functional.IndicatorFunctional

Examples

>>> x1 = np.arange(10); x2 = x1 / np.linalg.norm(x1, ord=1)
>>> radius=10; ball = L1Ball(dim=x1.size, radius=radius)
>>> ball(x1), ball(x2)
(inf, 0)
>>> np.allclose(ball.prox(x1,tau=1), proj_l1_ball(x1, radius=radius)), np.linalg.norm(ball.prox(x1,tau=1), ord=1)
(True, 10.0)
>>> np.allclose(ball.prox(x2,tau=1), x2)
True

Notes

The \(\ell_1\)-ball penalty is convex and proximable. It is a constrained variant of the \(\ell_1\)-norm penalty. The proximal operator of the \(\ell_1\)-ball indicator is the projection onto the \(\ell_1\)-ball (see [ProxAlg] Section 6.5.2).

See also

L1BallLoss(), L1Norm, py:class:pycsou.func.penalty.SquaredL1Norm.

class LInftyNorm(dim: int)[source]¶

Bases: pycsou.func.base.LpNorm

\(\ell_\infty\)-norm, \(\Vert\mathbf{x}\Vert_\infty:=\max_{i=1,\ldots,N} |x_i|\).

Examples

>>> x = np.arange(10)
>>> norm = LInftyNorm(dim=x.size)
>>> norm(x)
9
>>> lambda_ = 3; scaled_norm = lambda_ * norm; scaled_norm(x)
27

Notes

The \(\ell_\infty\)-norm is a convex but non differentiable penalty functional. Solutions to \(\ell_\infty\)-penalised convex optimisation problems are non unique and binary (i.e. \(x_i\in\{1,-1\}\)). The proximal operator of the \(\ell_\infty\)-norm does not admit a closed-form but can be computed effficiently as discussed in [ProxAlg] section 6.5.2.

See also

LInftyLoss(), LInftyBall().

__init__(dim: int)[source]¶

Parameters

dim (int) – Dimension of the functional’s domain.
proj_lq_ball (Callable[x: np.ndarray, radius: float]) – Projection onto the \(\ell_q\)-ball where \(1/p+1/q=1.\)

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

LInftyBall(dim: int, radius: numbers.Number) → pycsou.func.base.IndicatorFunctional[source]¶

Indicator function of the \(\ell_\infty\)-ball \(\{\mathbf{x}\in\mathbb{R}^N: \|\mathbf{x}\|_\infty\leq \text{radius}\}\)

It is defined as:

\[\begin{split}\iota(\mathbf{x}):=\begin{cases} 0 \,\text{if} \,\|\mathbf{x}\|_\infty\leq \text{radius},\\ \, +\infty\,\text{ortherwise}. \end{cases}\end{split}\]

Parameters

dim (int) – Dimension of the domain.
radius (Number) – Radius of the \(\ell_\infty\)-ball.

Returns

Indicator function of the \(\ell_\infty\)-ball.

Return type

py:class:pycsou.core.functional.IndicatorFunctional

Examples

>>> x1 = np.arange(10); x2 = x1 / np.linalg.norm(x1, ord=np.inf)
>>> radius=8; ball = LInftyBall(dim=x1.size, radius=radius)
>>> ball(x1), ball(x2)
(inf, 0)
>>> np.allclose(ball.prox(x1,tau=1), proj_linfty_ball(x1, radius=radius)), np.linalg.norm(ball.prox(x1,tau=1), ord=np.inf)
(True, 8.0)
>>> np.allclose(ball.prox(x2,tau=1), x2)
True

Notes

The \(\ell_\infty\)-ball penalty is convex and proximable. It is a constrained variant of the \(\ell_\infty\)-norm penalty. The proximal operator of the \(\ell_\infty\)-ball indicator is the projection onto the \(\ell_\infty\)-ball (see [ProxAlg] Section 6.5.2).

See also

LInftyBallLoss(), LInftyNorm.

class L21Norm(dim: int, groups: numpy.ndarray)[source]¶

Bases: pycsou.core.functional.ProximableFunctional

\(\ell_{2,1}\)-norm, \(\Vert\mathbf{x}\Vert_{2,1}:=\sum_{g=1}^G \sqrt{ \sum_{i\in\mathcal{G}_g} |x_i|^2}\,.\)

Examples

>>> x = np.arange(10,dtype=np.float64)
>>> groups = np.concatenate((np.ones(5),2*np.ones(5)))
>>> group_norm = L21Norm(dim=x.size,groups=groups)
>>> type(group_norm)
<class 'pycsou.func.penalty.L21Norm'>
>>> group_norm(x)
21.44594499772297
>>> l2_norm = L21Norm(dim=x.size,groups=np.ones(x.size))
>>> type(l2_norm)
<class 'pycsou.func.penalty.L2Norm'>
>>> l1_norm = L21Norm(dim=x.size,groups=np.arange(x.size)) # Also if groups = None
>>> type(l1_norm)
<class 'pycsou.func.penalty.L1Norm'>
>>> single_group_l2 = L2Norm(dim=x.size/2)
>>> tau = 0.5; np.allclose(group_norm.prox(x,tau=tau),np.concatenate((single_group_l2.prox(x[0:5], tau=tau),single_group_l2.prox(x[5:10],tau=tau))))
True

Notes

The \(\ell_{2,1}\)-norm penalty is convex and proximable. This penalty tends to produce group sparse solutions, where all elements \(x_i\) for \(i\in\mathcal{G}_g\) for some \(g\in\lbrace 1,2,\dots,G\) tend to be zero or non-zero jointly. A critical assumtion is that the groups are not overlapping, i.e., \(\mathcal{G}_j \cap \mathcal{G}_i = \emptyset\) for \(j,i \in \lbrace 1,2,\dots,G\rbrace\) such that \(j\neq i.\) The proximal operator of the \(\ell_{2,1}\)-norm is obtained easily from that of the \(\ell_2\)-norm and the separable sum property.

See also

L2Norm(), SquaredL2Norm, L1Norm().

__init__(dim: int, groups: numpy.ndarray)[source]¶

Parameters

dim (int) – Dimension of the domain.
groups (np.ndarray, optional, defaults to None) – Numerical variable of the same size as \(x\), where different groups are distinguished by different values. If each element of x belongs to a different group, an L1Norm is returned. If all elements of x belong to the same group, an L2Norm is returned.

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

prox(x: Union[numbers.Number, numpy.ndarray], tau: numbers.Number) → Union[numbers.Number, numpy.ndarray][source]¶

Evaluate the proximity operator of the tau-scaled functional at the point x.

Parameters

x (Union[Number, np.ndarray]) – Point at which to perform the evaluation.
tau (Number) – Scale.

Returns

Evaluation of the proximity operator of the tau-scaled functional at the point x.

Return type

Union[Number, np.ndarray]

NonNegativeOrthant(dim: int) → pycsou.func.base.IndicatorFunctional[source]¶

Indicator function of the non negative orthant (positivity constraint).

It is used to enforce positive real solutions. It is defined as:

\[\begin{split}\iota(\mathbf{x}):=\begin{cases} 0 \,\text{if} \,\mathbf{x}\in \mathbb{R}^N_+,\\ \, +\infty\,\text{ortherwise}. \end{cases}\end{split}\]

Parameters: dim (int) – Dimension of the domain.
Returns: Indicator function of the non negative orthant.
Return type: py:class:pycsou.core.functional.IndicatorFunctional

Examples

>>> x1 = np.arange(10) - 5
>>> func = NonNegativeOrthant(dim=x1.size)
>>> func(x1), func(np.abs(x1))
(inf, 0)
>>> np.allclose(func.prox(x1,tau=1), proj_nonnegative_orthant(x1))
True
>>> np.alltrue(func.prox(x1,tau=1) >= 0)
True

See also

LogBarrier().

Segment(dim: int, a: numbers.Number = 0, b: numbers.Number = 1)[source]¶

Indicator function of the segment \([a,b]\subset\mathbb{R}\).

It is defined as:

\[\begin{split}\iota(\mathbf{x}):=\begin{cases} 0 \,\text{if} \,\mathbf{x}\in [a,b]^N,\\ \, +\infty\,\text{ortherwise}. \end{cases}\end{split}\]

Parameters

dim (int) – Dimension of the domain.
a (Number) – Left endpoint of the segement.
b (Number) – Right endpoint of the segment.

Returns

Indicator function of the segment \([a,b]\subset\mathbb{R}\).

Return type

py:class:pycsou.core.functional.IndicatorFunctional

Examples

>>> x1 = np.arange(10) - 3
>>> func = Segment(dim=x1.size, a=1, b=4)
>>> func(x1), func(np.clip(x1, a_min=1, a_max=4))
(inf, 0)
>>> np.allclose(func.prox(x1,tau=1), proj_segment(x1, a=1, b=4))
True
>>> func.prox(x1,tau=1)
array([1, 1, 1, 1, 1, 2, 3, 4, 4, 4])

See also

RealLine(), ImagLine().

RealLine(dim: int)[source]¶

Indicator function of the real line \(\mathbb{R}\).

It is defined as:

\[\begin{split}\iota(\mathbf{x}):=\begin{cases} 0 \,\text{if} \,\mathbf{x}\in \mathbb{R}^N,\\ \, +\infty\,\text{ortherwise}. \end{cases}\end{split}\]

Parameters: dim (int) – Dimension of the domain.
Returns: Indicator function of the real line \(\mathbb{R}\).
Return type: py:class:pycsou.core.functional.IndicatorFunctional

Examples

>>> x1 = np.arange(10) + 1j
>>> func = RealLine(dim=x1.size)
>>> func(x1), func(np.real(x1))
(inf, 0)
>>> np.allclose(func.prox(x1,tau=1), np.real(x1))
True

See also

NonNegativeOrthant(), ImagLine().

ImagLine(dim: int)[source]¶

Indicator function of the imaginary line \(j\mathbb{R}\).

It is defined as:

\[\begin{split}\iota(\mathbf{x}):=\begin{cases} 0 \,\text{if} \,\mathbf{x}\in (j\mathbb{R})^N,\\ \, +\infty\,\text{ortherwise}. \end{cases}\end{split}\]

Parameters: dim (int) – Dimension of the domain.
Returns: Indicator function of the imaginary line \(j\mathbb{R}\).
Return type: py:class:pycsou.core.functional.IndicatorFunctional

Examples

>>> x1 = np.arange(10) + 1j * np.arange(10)
>>> func = ImagLine(dim=x1.size)
>>> func(x1), func(1j * np.imag(x1))
(inf, 0)
>>> np.allclose(func.prox(x1,tau=1), np.imag(x1))
True

See also

NonNegativeOrthant(), RealLine().

class LogBarrier(dim: int)[source]¶

Bases: pycsou.core.functional.ProximableFunctional

Log barrier, \(f(\mathbf{x}):= -\sum_{i=1}^N \log(x_i).\)

The log barrier is defined as:

\[\begin{split}f(x):=\begin{cases} -\log(x) & \text{if} \, x>0,\\ +\infty & \text{otherwise.} \end{cases}\end{split}\]

Examples

>>> x1 = np.arange(10)
>>> func = LogBarrier(dim=x1.size)
>>> func(x1), func(x1+2)
(inf, -17.502307845873887)
>>> np.round(func.prox(x1,tau=1))
array([1., 2., 2., 3., 4., 5., 6., 7., 8., 9.])

Notes

The log barrier can be used to enforce positivity in the solution. Its proximal operator is given in [ProxAlg] Section 6.7.5.

__init__(dim: int)[source]¶

Parameters: dim (int) – Dimension of the domain.

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

prox(x: Union[numbers.Number, numpy.ndarray], tau: numbers.Number) → Union[numbers.Number, numpy.ndarray][source]¶

Proximal operator of the log barrier.

Parameters

x (Union[Number, np.ndarray]) – Input.
tau (Number) – Scaling constant.

Returns

Proximal point of x.

Return type

Union[Number, np.ndarray]

class ShannonEntropy(dim: int)[source]¶

Bases: pycsou.core.functional.ProximableFunctional

Negative Shannon entropy, \(f(\mathbf{x}):= \sum_{i=1}^N x_i\log(x_i).\)

The (negative) Shannon entropy is defined as:

\[\begin{split}f(x):=\begin{cases} x\log(x) & \text{if} \, x>0,\\ 0& \text{if} \, x=0,\\ +\infty & \text{otherwise.} \end{cases}\end{split}\]

Examples

>>> x1 = np.arange(10); x2=np.zeros(10); x2[0]=10
>>> func = ShannonEntropy(dim=x1.size)
>>> func(x1), func(x2)
(79.05697962199447, 23.02585092994046)
>>> np.round(func.prox(x1,tau=2))
array([0., 0., 1., 1., 1., 2., 2., 3., 3., 4.])

Notes

This regularization functional is based the information-theoretic notion of entropy, a mathematical generalisation of entropy as introduced by Boltzmann in thermodynamics. It favours solutions with maximal entropy. The latter are typically positive and featureless: smooth functions indeed carry much less spatial information than functions with sharp, localised features, and hence have higher entropy. This penalty is restricted to positive solutions and often results in overly-smooth estimates. Its proximal operator is given in [ProxSplit] Table 2 or [ProxEnt] Section 3.1.

ProxEnt: Afef, Cherni, Chouzenoux Émilie, and Delsuc Marc-André. “Proximity operators for a class of hybrid sparsity+ entropy priors application to dosy NMR signal reconstruction.” 2016 International Symposium on Signal, Image, Video and Communications (ISIVC). IEEE, 2016.

See also

LogBarrier(), KLDivergence.

__init__(dim: int)[source]¶

Parameters: dim (int) – Dimension of the domain.

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

prox(x: Union[numbers.Number, numpy.ndarray], tau: numbers.Number) → Union[numbers.Number, numpy.ndarray][source]¶

Proximal operator of the Shannon entropy functional.

Parameters

x (Union[Number, np.ndarray]) – Input.
tau (Number) – Scaling constant.

Returns

Proximal point of x.

Return type

Union[Number, np.ndarray]

class QuadraticForm(dim: int, linop: Optional[pycsou.core.linop.LinearOperator] = None)[source]¶

Bases: pycsou.core.functional.DifferentiableFunctional

Quadratic form \(\mathbf{x}^\ast \mathbf{L} \mathbf{x}\).

Examples

>>> rng = np.random.default_rng(0)
>>> L =  rng.standard_normal(100).reshape(10,10)
>>> L = L.transpose() @ L #make definite positive
>>> Lop = DenseLinearOperator(L)
>>> F = QuadraticForm(dim=10,linop=Lop)
>>> x = np.arange(10)
>>> np.allclose(F(x), np.dot(x, Lop @ x))
True
>>> np.allclose(F.gradient(x), 2 * Lop @ x)
True

Notes

The quadratic form is defined as the functional \(F:\mathbb{R}^n\to \mathbb{R}, \; \mathbf{x}\mapsto \mathbf{x}^\ast \mathbf{L}\mathbf{x}\) for some positive semi-definite linear operator \(\mathbf{L}:\mathbb{R}^n\to \mathbb{R}^n\). Its gradient is given by \(\nabla F(\mathbf{x})=2 \mathbf{L}\mathbf{x}.\) The latter is \(\beta\)-Lipschitz continuous with \(\beta=2\|\mathbf{L}\|_2.\)

See also

SquaredL2Norm()

__init__(dim: int, linop: Optional[pycsou.core.linop.LinearOperator] = None)[source]¶

Parameters

dim (int) – Dimension of the domain.
linop (LinearOperator) – Positive semi-definite operator defining the quadratic form. If None the identity operator is assumed.

__call__(x: Union[numbers.Number, numpy.ndarray]) → numbers.Number[source]¶

Call self as a function.

Parameters: arg (Union[Number, np.ndarray]) – Argument of the map.
Returns: Value of arg through the map.
Return type: Union[Number, np.ndarray]

jacobianT(x: Union[numbers.Number, numpy.ndarray]) → numpy.ndarray[source]¶

Transpose of the Jacobian matrix of the differentiable map evaluated at arg.

Parameters: arg (Union[Number, np.ndarray]) – Point at which the transposed Jacobian matrix is evaluated.
Returns: Linear operator associated to the transposed Jacobian matrix.
Return type: LinearOperator