Syllabus

LEARNING OUTCOMES

Credits: 5

Schedule: 04.09.2023 - 27.11.2023

Teacher in charge (valid for whole curriculum period):

Teacher in charge (applies in this implementation): Pekka Marttinen

Contact information for the course (applies in this implementation):

CEFR level (valid for whole curriculum period):

Language of instruction and studies (applies in this implementation):

Teaching language: English. Languages of study attainment: English

CONTENT, ASSESSMENT AND WORKLOAD

Content

• valid for whole curriculum period:

  Introductory lecture, recent research articles on NLP.

• applies in this implementation

  Preliminary list of papers: https://docs.google.com/spreadsheets/d/14KMAYPSm9A_Z_R6EvWpSy58_GG4mPrVksvLN25NsyNE/edit#gid=0
  
  Introductory lecture on transformers, encoder and decoder architectures, Large Language Models (LLMs), fine-tuning and OpenSource LLMs.

  22 seminar presentations (2 per lecture) on the following topics (order to be defined depending on the students' preferences):

  • Transformer alternatives
  • LLMs for coding
  • In-Context Learning
  • Mixture of Experts
  • Multimodality
  • Parameter-Efficient Fine-Tuning
  • Performance improvements
  • Positional Encodings
  • Quantization
  • Reasoning and acting with LLMs
  • Reinforcement Learning from Human Feedback
  • Scaling LLMs
  • Security and alignment
  • Open problems and limitations of the state-of-the-art approaches
  
  

Assessment Methods and Criteria

• applies in this implementation

  Students will be graded based on their presentation, opponent's task and writing assignments.
  

Workload

• applies in this implementation

  Attending meetings 12*2h=24h. Preparing the presentation 14h. Preparing to serve as opponent: 7h. Familiarizing oneself with the week’s papers: 11*2*3h=66h. Writing assignments: 11*2h=22h. Total: 133h, 133/26.7 = 5 cr.
  

DETAILS

Study Material

• valid for whole curriculum period:

  Articles and slides available online.

• applies in this implementation

  Articles and slides available online.
  

Substitutes for Courses

• valid for whole curriculum period:

  Substitutes for Courses

Prerequisites

• valid for whole curriculum period:

  Prerequisites

FURTHER INFORMATION

Further Information

• valid for whole curriculum period:

  Seminar, one 2-hour meeting per week. No exam. In-person attendance required (minimum 10/12 weeks to pass).

  Number of participating students: max. 22. If more students want to attend, the selection will be made based on the amount and average grade of Master's level machine learning courses.

  Assesment criteria: Presentation, opponent's task, writing assignments.

  Estimated workload: Attending meetings 12*2h=24h. Preparing the presentation 14h. Preparing to serve as opponent: 7h. Familiarizing oneself with the week’s papers: 11*2*3h=66h. Writing assignments: 11*2h=22h. Total: 133h, 133/26.7 = 5 cr

Details on the schedule

• applies in this implementation

   Ma 04.09.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 11.09.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 18.09.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 25.09.2023 klo 10:15 - 12:00, R030/A140 T4
   Ma 02.10.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 09.10.2023 klo 10:15 - 12:00, R030/A133 T5
   (exam week)
   Ma 23.10.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 30.10.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 06.11.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 13.11.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 20.11.2023 klo 10:15 - 12:00, R030/A133 T5
   Ma 27.11.2023 klo 10:15 - 12:00, R030/A133 T5
  

Description