Released in November 2022 by OpenAI, ChatGPT already made and still keeps headlines not only for its remarkable performances but also for its most ridiculous failures. The Institute has recently issued a guideline for the use of the ChatGPT. To support our community, the Tech Hub offers several workshops for students, researchers and teaching faculty, as well as a working paper to explore various scenarios related to its possible uses and misuses.
ChatGPT is not a sudden creation enabled by advancements in Artificial Intelligence. It has a genesis and comes with quite an impressive number of occurrences and contingencies, all of which allow a clearer understanding of its functioning and a better assessment of its added value.
As the name suggests, generative AI produces or generates text, images, music, speech, code or video. Behind this concept lies machine-learning techniques which have evolved over the past decade, allowing us to explore and produce a specific output out of a large corpora of data.
ChatGPT is a type of large language model (LLM) specialized for conversational interactions, (still) available as a free demo. It was released by OpenAI, which is a research lab running on a commercial model, with Microsoft being the main investor. Large models are trained on massive datasets represented by books, articles and websites; ChatGPT is one such LLM trained through an innovative technique in state-of-the art LLMs: reinforcement learning from human feedback (RLHF). The ‘non-scalable’ aspects and costs – the human component – are yet to be disclosed.
Indeed, OpenAI has not yet disclosed the full details, neither with regards to the reinforcement learning process, nor with regards to the sources for training; the ration human-machine is still a mystery. The model has accomplished all kinds of impressive tasks, including providing feedback on code, writing poetry, explaining technical concepts in different tones, generating prompts for generative AI models, and going on philosophical rants. However, the model is also prone to the kinds of errors that similar LLMs have made, and most quoted ones are references to non-existing papers and books, misinterpreting intuitive physics, and failing at compositionality. OpenAI acknowledges that their models can still generate toxic and biased outputs.
No matter how ground-breaking ChatGPT might be labelled, it still remains a large language model transformed into a successful commercial product combining computing power – it uses Microsoft’s cloud infrastructure – and talent – OpenAI was initially a start-up gathering top researchers in the field of machine learning/natural language processing.
To find our more about ChatGPT and its its possible uses and misuses for research and teaching, please read the working paper of Dr. Oana Ichim below.
The Tech Hub offer a workshop on the same topic for students and researchers on March, 9th. It will provide the conceptual tools for understanding how and why, in light of these main structural features ChatGPT breaks away from previous state-of-art in the field of large language and explores what are the stakes of such enhanced capacities. Moreover, the workshop proposes to interactively develop a series of practical advices on how to cultivate practical specific skills for editing prompts on an interface such as ChatGPT rather than engaging into a (human) conversation. The main scope of this event is to engage participants in discussions about how to make the best of conceptual tools and practical skills in this new era of digital products. You can register her.