#ise2023 — Public Fediverse posts

Last leg in our brief #timeline of (Large) #languagemodels (so far) is 2023, which saw the advent of many new and updated #LLMs:
- BARD #chatbot is introduced by Google
- LLaMA is introduced by Meta
- GPT-4 is introduced by OpenAI.
- LLaMA2.0 is introduced by Meta
- and many others...
#ISE2023 #lecture slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
GPT-4 tech report: https://arxiv.org/pdf/2303.08774
@fizise @KIT_Karlsruhe #ai #artificialintelligence #llm #llms #gpt #openai #llama #lamda #bard

Next stop on our brief #timeline of (Large) #LanguageModels is 2022:
InstructGPT is introduced by OpenAI, a GPT-3 model complemented and fine-tuned with reinforcement learning from human feedback.
ChatGPT is introduced by OpenAI as a combination of GPT-3, Codex, and InstructGPT including lots of additional engineering.
#ise2023 lecture slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
#RLHF explained: https://huggingface.co/blog/rlhf
#ai #creativeai #rlhf #gpt3 #gpt #openai #chatgpt #lecture #artificialintelligence #llm

Next stop in our brief #timeline of (large) #languagemodels is 2021:
DALL-E is released by OpenAI and raises text2img to a new level.
Codex is released by OpenAI able to translate natural language into programming code.
WebGPT is released by OpenAI for answering open-ended questions.
LaMDA is introduced by Google.
Slides from #ise2023 #lecture: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
Codex paper: https://arxiv.org/abs/2107.03374
DALL-E paper: https://arxiv.org/abs/2102.12092
@fizise #ai #generativeAI #GPT #dalle #openai
#lamda

Next leg in our brief history of (Large) #LanguageModel is 2020, when #GPT-3 was released by OpenAI, based on 45TB data crawled from the web. A “data quality” predictor was trained to boil down the training data to 550GB “high quality” data. Learning from the prompt was introduced (few-shot learning)
Lecture slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
paper: https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
@fizise #ai #artificialintelligence #creativeai #llm #ise2023 #lecture

Next stop in our Brief History of (Large) #languagemodels is 2019: GPT-2 was released by OpenAI as a direct scale-up of GPT, comprising 1.5B parameters and trained on 8M web pages.
Slides (from #ise2023 lecture): https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
Paper: https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf
#llm #llms #ai #artificialintelligence #generativeai #gpt #lecture #historyofAI

@bsletten The #ise2023 summer lecture was not recorded. The intention was to bring students back to university lecture halls. There is a 80% overlap with the #ise2021 lecture which is already on #youtube. However, if you are interested in our latest lecture on #knowledgegraphs, don't miss the (free) #KG2023 online course "Knowledge Graphs - Foundations and Applications" on #OpenHPI which starts in Oct 2023.
https://open.hpi.de/courses/knowledgegraphs2023
@fizise @tabea @sashabruns @Hasso_Plattner_Institute

Next leg in our brief history of (Large) #language models is then advent of the first (real) pretrained #LLMs: ElMO (Allan Institute for AI, 2017), GPT (OpenAI, 2018) and BERT (Google, 2018).
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
ELMO: https://aclanthology.org/N18-1202.pdf
GPT: https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
BERT: https://aclanthology.org/N19-1423
#AI @fizise #languagemodel #nlp #artificialintelligence #generativeai #lecture #ise2023

Next stop in our brief Timeline of (large) #languagemodels from the #ise2023 lecture is the advent of the Graphical Processing Units #gpu. In 1999 Nvidias GeForce 256 was one of the very first, which enabled highly parallel computations for #neuralnetworks
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
@fizise #artificialintelligence #lecture #ai #machinelearning #llm

1997 with the advent of Long Short-Term Memory recurrent #neuralnetworks marks the subsequent step in our brief history of )large) #languagemodels from last week's #ise2023 lecture. Introduced by Sepp Hochreiter and Jürgen Schmidhuber #LSTM #RNNs enabled efficient processing of sequences of data.
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
#nlp #llm #llms #ai #artificialintelligence #lecture @fizise

Next step in our brief timeline of (large) #languagemodels from our #ise2023 lecture was statistical language modeling with n-grams based on large text corpora as introduced and popularized by Frederick Jelinek and Stanley F. Chen using statistical tricks like Bayes Theorem, Markov Assumption, and Maximum Likelihood Estimation, etc.
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
@fizise #nlp #llm #llms #artificialintelligence #ai #lecture #creativeAI

Went to the university office to collect the #ise2023 final exams for later reviewing together with the @fizise ta team. But for now, a #perfectEspresso at home… because it was too rainy today for walking to Espresso Stazione ☕️ #lecture #coffeechallenge #karlsruhe @KIT_Karlsruhe

Slide 2 of our Brief Timeline for (Large) #LanguageModels from the last #ise2023 lecture introduced us to #ELIZA, Joseph Weizenbaum's simple #Chatbot from 1966 that simulates a conversation with a psychoanalyst. Weizenbaum was shocked that some persons including his secretary attributed human-like feelings to the computer program...
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
#nlp #ai #llm #artificialintelligence @fizise

One of the final sections of the #ise2023 lecture was an excursion with a #timeline of (Large) #LanguageModels. We started our tour in 1948 with Claude Shannon's seminal work "A Mathematical Theory of Communication""
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
@fizise #llm #ai #nlp #artificialintelligence #informationtheory #lecture

As a 2nd topic of this last #ise2023 lecture, we were discussing #KnowledgeGraph Completion. Most simple approach for unsupervised #linkprediction based on (here translation-based) knowledge graph embeddings was explained on the example of Isaac Asimov.
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
@fizise @enorouzi #scifi #knowledgegraphs #ai #deeplearning #embeddings

Ok, I tried out Runway Gen-2. I did some tests with prompt only but also with uploaded images (mostly generated by another generative AI). Lessons learned: 1) don't expect too much...
2) you have to try very often...
3) don't expect too much ;-)
Below you can see the video generated based on my stablediffusion "Singularity" picture from the #ise2023 lecture. #generativeAI #runway #stablediffusion #stablediffusionart #aiart #singularity

How can we find out the importance of a node in a #knowledgeGraph? In the last #ise2023 lecture, we were discussing graph centrality measures and how they can be applied in the context of knowledge graphs.
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
SPARQL query (cf image below, the 100 most "important" #SciFi authors according to #wikidata) https://w.wiki/78Un
@fizise @enorouzi #semanticweb #lecture #ai #datascience #analytics

Topics of the last #ise2023 lecture; The Graph in #KnowledgeGraphs, Knowledge Graph Completion, A Brief History of Large Language Models, and Knowledge Graphs and Large Language Models. I will highlight some topics with the upcoming toots...
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
#llms #languagemodels #deeplearning #linkprediction #kgc #lecture #machinelearning #transformers #gpt @fizise @enorouzi

Last #ise2023 lecture of this semester is about to start. 8:00AM is always tough for the students as well as for the professor 🥳 @fizise @KIT_Karlsruhe @enorouzi #ai #machinelearning #KnowledgeGraphs

Last thing we've discussed in the "Limits of #AI" chapter of the #ISE2023 lecture was the threat of the so-called #Singularity. What is the singularity? Under which circumstances can it possibly happen? How real is this threat and should we better already aim for potential regulations?
Slides: https://drive.google.com/file/d/1LUOA-NiE4nJ4sn5elbXUN4c7YDtK9uXB/view?usp=drive_link
@fizise @enorouzi @KIT_Karlsruhe #machinelearning #deeplearning #philosophy #aiart #stablediffusionart #creativeai

When discussing the limits of #AI in last week's #ise2023 lecture, we also talked about the Chinese Room Problem introduced by John Searle in 1980.
Slides: https://drive.google.com/file/d/1LUOA-NiE4nJ4sn5elbXUN4c7YDtK9uXB/view?usp=drive_link
#machinelearning #artificialintelligence #deeplearning #lecture #philosophy @fizise @enorouzi

In last week's #ise2023 lecture we were discussing the Turing Test aka the Imitation Game as means to test a machine's ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human... However, in times of #ChatGPT we have to rethink our concepts of "intelligence".
Slides: https://drive.google.com/file/d/1LUOA-NiE4nJ4sn5elbXUN4c7YDtK9uXB/view?usp=drive_link
#machinelearning #ai #turing #turingtest #intelligence #lecture @fizise @enorouzi #aiart #creativeai #stablediffusionart

"A perceptron cannot even solve the XOR problem, so why bother?"...Welcome to the first #AI Winter. We delved a bit into the history of #neuralnetworks in this week's #ise2023 lecture. Want to learn more about the XOR problem, just have a look into the slides (pp. 12-13):
https://drive.google.com/file/d/1LUOA-NiE4nJ4sn5elbXUN4c7YDtK9uXB/view?usp=drive_link
@fizise @enorouzi #machinelearning #lecture @KIT_Karlsruhe #aiart #stablediffusionart #creativeAI

In today's #ise2023 lecture, we discussed neural networks, from the very simply McCulloch-Pitts Neuron up to Convolutional Neural Networks and Generative Adversarial Networks. A lot of content for only 90 minutes of lecture ;-)
Slides: https://drive.google.com/file/d/1KsAGBViEzu7UDkz-F2QJJJ23ce-AQ9pl/view?usp=drive_link
#machinelearning #deeplearning #representationlearning #knowledgegraphs #graphembeddings #embeddings #lecture @fizise @enorouzi #aiart #stablediffusionart

To demonstrate the simple #machinelearning model of linear regression we have chosen an example with climate data to predict future temperatures. Of course this prediction is lacking any scientific grounding. Nevertheless, we want to raise attention for #climatechange also in our #computerscience #ise2023 lectures.
Slides: https://drive.google.com/file/d/1TXYYCDOia9VD20i0o1xW0Os01nb2WvGO/view?usp=drive_link
Colab Notebook: https://drive.google.com/file/d/1lA25T1ZTHz9ck7hRMRby4bneAMszrhCW/view?usp=drive_link
#semanticweb #knowledgegraphs #ai #climate #scientists4future #colab #notebook #python

We discussed basic (classical) machine learning algorithms, as e.g. k-Means clustering, linear regression, and decision trees in yesterday's #ise2023 lecture. Despite the dominance of #neuralnetworks those classical ml algorithms are still very useful, in particular if you have not many training data.
Slides: https://drive.google.com/file/d/1TXYYCDOia9VD20i0o1xW0Os01nb2WvGO/view?usp=sharing
#lecture #machinelearning #ai #artificialintelligence @fizise @enorouzi #aiart #creativeai

"There is no bias free learning" - Introducing the many variants of potential bias that we have to face in #machinelearning was one of the topics of last week's #ise2023 lecture
Slides: https://drive.google.com/file/d/18_xPnJDM04I7pDihMMhoFIuDshfUKMk9/view?usp=drive_link

#semanticWeb #knowledgegraphs #ai #bias #lecture @fizise @enorouzi

“The only way to rectify our reasonings is to make them as tangible as those of the Mathematicians, so that we can find our error at a glance, and when there are disputes among persons, we can simply say: Let us calculate [calculemus], without further ado, to see who is right.”
Quoting Leibniz for last week's #ise2023 lecture with a brief #HistoryofAI
Slides: https://drive.google.com/file/d/18_xPnJDM04I7pDihMMhoFIuDshfUKMk9/view?usp=sharing

#ai #artificialintelligence #lecture @fizise @KIT_Karlsruhe @ebrahim #calculemus #logics #leibniz #philosophy

An allusion can work on various levels of abstraction. But it is futile, if the students don't know the reference..
This week's #ise2023 lecture, we were shading light on the history of #AI. I expected that the students had never heard of Marvin Minsky, but nobody realized the what the picture in the background was all about. Do you...? ;-)
Slides: https://drive.google.com/file/d/18_xPnJDM04I7pDihMMhoFIuDshfUKMk9/view?usp=sharing
#artificialintelligence #stanleykubrick #kubrick #hal #lecture @fizise @enorouzi @KIT_Karlsruhe #isws2023 #slidekaraoke

In the #ise2023 lecture today, we are opening a new chapter on "Basic Machine Learning". This week you will learn about the history of AI, the types of AI and Machine Learning, the main challenges of ML and the general ML workflow.
Slides: https://drive.google.com/file/d/18_xPnJDM04I7pDihMMhoFIuDshfUKMk9/view?usp=sharing
#machinelearning #AI #artificialintelligence #lecture #deeplearning #historyofAI @fizise @enorouzi @KIT_Karlsruhe @nfdi4culture #stablediffusionart #creativeai #aiart

@fizise @enorouzi ...and here is the notorious flying sheep sketch I alluded to in the #ise2023 lecture ;-)
#flyingsheep #montypython #lecture
https://youtu.be/9TeiSsJ3G_0