What are the updates modification functions in lasagne?

This recipe explains what are the updates modification functions in lasagne.

Recipe Objective - What are the updates modification functions in lasagne?

Updates Modification Functions:-

1. apply_momentum - Returns a modified update dictionary including momentum.

2. apply_nesterov_momentum - Returns a modified update dictionary including Nesterov momentum.

Access YOLO OCR Character Recognition Project with Source Code

For more related projects:-

/projects/data-science-projects/deep-learning-projects
/projects/data-science-projects/tensorflow-projects

Update Modification Functions Example:-

import lasagne
import theano.tensor as T
import theano
from lasagne.nonlinearities import softmax
from lasagne.layers import InputLayer, DenseLayer, get_output
from lasagne.updates import nesterov_momentum
ly_in = InputLayer((100, 20))
ly1 = DenseLayer(ly_in, num_units=3, nonlinearity=softmax)
x = T.matrix('x') # shp: num_batch x num_features
y = T.ivector('y') # shp: num_batch
ly_out = get_output(ly1, x)
params = lasagne.layers.get_all_params(ly1)
loss = T.mean(T.nnet.categorical_crossentropy(ly_out, y))
updates = nesterov_momentum(loss, params, learning_rate=1e-4, momentum=.9)
train_fn = theano.function([x, y], updates=updates)

updates = lasagne.updates.rmsprop(loss, params, learning_rate=0.0001)
updates = lasagne.updates.apply_momentum(updates, params, momentum=0.9)
updates

OrderedDict([(<TensorType(float64, matrix)>, Elemwise{add,no_inplace}.0),
             (W, Elemwise{add,no_inplace}.0),
             (<TensorType(float64, vector)>, Elemwise{add,no_inplace}.0),
             (b, Elemwise{add,no_inplace}.0),
             (<TensorType(float64, matrix)>, Elemwise{sub,no_inplace}.0),
             (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])

What Users are saying..

profile image

Abhinav Agarwal

Graduate Student at Northwestern University
linkedin profile url

I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More

Relevant Projects

Abstractive Text Summarization using Transformers-BART Model
Deep Learning Project to implement an Abstractive Text Summarizer using Google's Transformers-BART Model to generate news article headlines.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Build Multi Class Text Classification Models with RNN and LSTM
In this Deep Learning Project, you will use the customer complaints data about consumer financial products to build multi-class text classification models using RNN and LSTM.

Census Income Data Set Project-Predict Adult Census Income
Use the Adult Income dataset to predict whether income exceeds 50K yr based oncensus data.

Learn How to Build PyTorch Neural Networks from Scratch
In this deep learning project, you will learn how to build PyTorch neural networks from scratch.

Deep Learning Project for Beginners with Source Code Part 1
Learn to implement deep neural networks in Python .

ML Model Deployment on AWS for Customer Churn Prediction
MLOps Project-Deploy Machine Learning Model to Production Python on AWS for Customer Churn Prediction

Learn to Build an End-to-End Machine Learning Pipeline - Part 1
In this Machine Learning Project, you will learn how to build an end-to-end machine learning pipeline for predicting truck delays, addressing a major challenge in the logistics industry.

LLM Project to Build and Fine Tune a Large Language Model
In this LLM project for beginners, you will learn to build a knowledge-grounded chatbot using LLM's and learn how to fine tune it.

Time Series Forecasting Project-Building ARIMA Model in Python
Build a time series ARIMA model in Python to forecast the use of arrival rate density to support staffing decisions at call centres.

OSZAR »