After ChatGPT: where do we stand with language models?

Language models (such as BERT, GPT3, ChatGPT… and soon GPT4) have deeply changed the landscape of research in natural language processing in recent years. These models have permitted previously unseen results on many tasks and in many languages. At the same time, their internal mechanisms remain rather opaque, and is the subject of intense research (Bertology). This situation raises many questions.

  • Can we say that these models ‘understand’ language? And if so, in what way? To what extent?
  • What is their interest and their benefits for research outside NLP? For creative work?
  • On a practical side, how can we deal with them and/or integrate them into our research, given the computing power required to train them? Have we become dependent on the major (often private) players in the field?
  • What are the limits of these models and their potential dangers?

We will probably not have all the answers to these questions on January 11, but this workshop will at least be an opportunity to think about these models, with various actors in the field, both private and public.

What? half-day workshop
Where? Ecole normale supérieure, salle des résistants, RDC, 45 rue d’Ulm 75005 Paris
When? Wednesday 11 January, from 2pm to 5:30pm
Registration: free entry, but please indicate your name here ici
(remote attending should be possible; the link will be sent to participants who have indicated their email address in the file above)

Programme

(presentations will be in English)

2-2:45pm : Thea Sommeesheild (U. Ca Foscari, Venise) : Fair AI for ancient languages: a proposal for dos and don’ts’’, presentation following her experience in the framework of the Ithaca project (Thea Sommershield will also give a presentation the day before, more directly dedicated to the Ithaca project, in the framework of the DHAI seminar)

2:45-3:30pm : Tm Van De Cruys (KU. Leuven, Belgique) : Using language models for poetry generation, language models and creation

3:30-4:15pm : Laurent Daudet et Olga Lopusanschi : (LightOn, Paris) “I need my own ! Developing private Large Language Models, on the development of language models within a start-up like LightOn.

4:30-5:30pm : Debate on the current situation, research and future of these models. Discussion with the speakers and with Anne Bouverot (Abeona Foundation)

17h30 : Thierry Poibeau : Wrap-up