What are the regularization functions in lasagne?
This recipe explains what are the regularization functions in lasagne.
Recipe Objective - What are the regularization functions in lasagne?
Lasagne provides "lasagne.regularization" class to apply regularization to the weights in a network.
Learn to Build a Multi Class Image Classification Model in Python from Scratch
Regularization Functions:-
1. lasagne.regularization.apply_penalty() - Computes the total cost for applying a specified penalty to a tensor or group of tensors.
2. lasagne.regularization.regularize_layer_params() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers.
3. lasagne.regularization.regularize_layer_params_weighted() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers.
4. lasagne.regularization.regularize_layer_params_weighted() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers, weighted by a coefficient for each layer.
5. lasagne.regularization.l1(x) - Computes the L1 norm of a tensor.
6. lasagne.regularization.l2(x) - Computes the L2 norm of a tensor.
For more related projects:-
/projects/data-science-projects/deep-learning-projects
/projects/data-science-projects/tensorflow-projects
Example:-
import lasagne
import theano.tensor as T
import theano
from lasagne.nonlinearities import softmax
from lasagne.layers import InputLayer, DenseLayer, get_output
from lasagne.regularization import regularize_layer_params_weighted, l2, l1
from lasagne.regularization import regularize_layer_params
layer_in = InputLayer((100, 20))
layer_1 = DenseLayer(layer_in, num_units=3)
layer_2 = DenseLayer(layer1, num_units=5, nonlinearity=softmax)
x = T.matrix('x') # shp: num_batch x num_features
y = T.ivector('y') # shp: num_batch
ly_out = get_output(layer2, x)
loss = T.mean(T.nnet.categorical_crossentropy(ly_out, y))
layers = {layer_1: 0.1, layer_2: 0.5}
l2_penalty = regularize_layer_params_weighted(layers, l2)
l1_penalty = regularize_layer_params(layer2, l1) * 1e-4
loss = loss + l2_penalty + l1_penalty
loss
Elemwise{add,no_inplace}.0
What Users are saying..
Abhinav Agarwal
I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More