CS-E407517 - Special Course in Machine Learning, Data Science and Artificial Intelligence: Seminar on NLP Research, Lectures, 4.9.2023-27.11.2023

Guest access
Please note! Course description is confirmed for two academic years, which means that in general, e.g. Learning outcomes, assessment methods and key content stays unchanged. However, via course syllabus, it is possible to specify or change the course execution in each realization of the course, such as how the contact sessions are organized, assessment methods weighted or materials used.

LEARNING OUTCOMES

Credits: 5

Schedule: 04.09.2023 - 27.11.2023

Teacher in charge (valid for whole curriculum period):

Teacher in charge (applies in this implementation): Pekka Marttinen

Contact information for the course (applies in this implementation):

CEFR level (valid for whole curriculum period):

Language of instruction and studies (applies in this implementation):

Teaching language: English. Languages of study attainment: English

CONTENT, ASSESSMENT AND WORKLOAD

Content

  • valid for whole curriculum period:

    Introductory lecture, recent research articles on NLP.

  • applies in this implementation

    Preliminary list of papers: https://docs.google.com/spreadsheets/d/14KMAYPSm9A_Z_R6EvWpSy58_GG4mPrVksvLN25NsyNE/edit#gid=0

    Introductory lecture on transformers, encoder and decoder architectures, Large Language Models (LLMs), fine-tuning and OpenSource LLMs.

    22 seminar presentations (2 per lecture) on the following topics (order to be defined depending on the students' preferences):

    • Transformer alternatives
    • LLMs for coding
    • In-Context Learning
    • Mixture of Experts
    • Multimodality
    • Parameter-Efficient Fine-Tuning
    • Performance improvements
    • Positional Encodings
    • Quantization
    • Reasoning and acting with LLMs
    • Reinforcement Learning from Human Feedback
    • Scaling LLMs
    • Security and alignment
    • Open problems and limitations of the state-of-the-art approaches

Assessment Methods and Criteria

  • applies in this implementation

    Students will be graded based on their presentation, opponent's task and writing assignments.

Workload

  • applies in this implementation

    Attending meetings 12*2h=24h. Preparing the presentation 14h. Preparing to serve as opponent: 7h. Familiarizing oneself with the week’s papers: 11*2*3h=66h. Writing assignments: 11*2h=22h. Total: 133h, 133/26.7 = 5 cr.

DETAILS

Study Material

  • valid for whole curriculum period:

    Articles and slides available online.

  • applies in this implementation

    Articles and slides available online.

Substitutes for Courses
Prerequisites

FURTHER INFORMATION

Further Information

  • valid for whole curriculum period:

    Seminar, one 2-hour meeting per week. No exam. In-person attendance required (minimum 10/12 weeks to pass).

    Number of participating students: max. 22. If more students want to attend, the selection will be made based on the amount and average grade of Master's level machine learning courses.

    Assesment criteria: Presentation, opponent's task, writing assignments.

    Estimated workload: Attending meetings 12*2h=24h. Preparing the presentation 14h. Preparing to serve as opponent: 7h. Familiarizing oneself with the week’s papers: 11*2*3h=66h. Writing assignments: 11*2h=22h. Total: 133h, 133/26.7 = 5 cr

Details on the schedule

  • applies in this implementation

     Ma 04.09.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 11.09.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 18.09.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 25.09.2023 klo 10:15 - 12:00, R030/A140 T4
     Ma 02.10.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 09.10.2023 klo 10:15 - 12:00, R030/A133 T5
     (exam week)
     Ma 23.10.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 30.10.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 06.11.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 13.11.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 20.11.2023 klo 10:15 - 12:00, R030/A133 T5
     Ma 27.11.2023 klo 10:15 - 12:00, R030/A133 T5

Rules of Conduct - Students´ Rights and Responsibilities (in force from 1 August 2020)

Aalto University Code of Academic Integrity and Handling Violations Thereof