ELEC-E5510 - Speech Recognition D, Lecture, 3.11.2021-17.12.2021
This course space end date is set to 17.12.2021 Search Courses: ELEC-E5510
Lecture exercise 3: NNLM
The goal is to verify that you understood how the neural network-based language models work.
Assume that we have a simple feed-forward neural network, which knows 100K words, and contains
2 hidden layers with 100 neurons. This network uses the one-hot
embedding to process words.
• Question 1. What percentage of the parameters are in the input and output layers?
• Question 2. If we want to cover 10 past words with this network how would the percentage change?
• Question 3. How would you change this model to reduce its size and increase its speed?
Hint: You can either upload your answer as a pdf file, a photo of your notes or a text file with numbers. Whatever is easiest for you. To get the activity point the answer does not have to be correct.