Notes

Goals dari course Gen-AI:

• Explain how generative artificial intelligence (Gen-AI) works
• Describe what foundation models are and how they drive progress in machine learning systems
• Explain the functionality of transformer models and how they work to solve language-related tasks
• Explain the principles of prompt engineering and modeling
• Creating a sandbox project within watsonx

Gen-AI = A subset from deeplearning to create a new content, images, text. ML = Brach of AI and CS focus use of data and algorithms to imitate the way human learn. DL = Subset of ML, essenstially a neural network with 3 more layer.

Several large language model (llm) model (i think this model release at-2020):

• GPT-3 (Generative Pre-treined Transformer 3) by OpenAI
• T5 (Text-to-Text Transformer) by Google Ai
• BERT by Google AI
• RoBERTa (A Robustly Optimized BERT Pretraining Approach) by Meta
• ALBERT (A Little BERT) by Google Research
• LLaMA by Meta

Evolution Gen-AI

1943: Birth Artifial Neuron Neuroscientist Warren McCulloch and Logician Wallter Pitts create the mathematical model of a neural network on their paper “A Logical Calculus of the Ideas Immanent in Nervous Activity.”

1949: Hebbian Learning Canadian pyschologist Donnald Hebb published “The Organization of Behaviour,”, explore a realitionship beetwen brain’s neural networks and behaviour.

1950: The Turing Test British mathematician Alan Turing introduce Turing Test on his paper. “Computing Machinery and Intelligence,” and the damous question “Can machine think?” and proposed “Imitation Game” as a practical answer.

1957: Perceptron Frank Rosenclatt created perceptron. Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, was published in 1962.

1959: ML spoken to existence Arthur Samuel make program for playing checkers. Samuel’s program employed the concept of reinforcement learning. And he make “Machine Learning” Popular.

1965: Multilayer Perceptron Alexey Ivakhnenko and Valentin Lapa introduced the first step deep learning version of multilayer perceptron.

1973: Winter Comes Compute power slow and memory not enough.

1985: Baby Talk Dr. Sejnowski and Charles Rosenberg created NETtalk, a program that learns to pronounce words like a baby does.

1997: Chess anyone? IBM Deepblue VS Garry Kasparov.

1997: MNIST database Under the leadership of Yann LeCun, the Modified National Institute of Standards and Technology (MNIST) database, consisting of handwritten digits sourced from American Census Bureau employees and high school students, was unveiled.

2009: ImageNet Created by Dr. Fei-Fei Li’s team at Stanford University, primarily to support machine learning research in computer vision.

2016: Let’s Go Google’s DeepMind’s AlphaGo VS Lee Sedol

2021: AlphaFold 2 AlphaFold 2, created by DeepMind (a Google-related company), can predict the intricate 3D shapes of proteins.

2022: ChatGPT ChatGPT, which stands for Chat Generative Pre-trained Transformer.

2023: IBM watsonx IBM watsonx.ai AI studio is part of the IBM watsonx AI and data platform.

Foundation Models

Foundation Models is interchangeably with “large language model”. That’s mistaken. LLM is one of the branches of the “foundation model”.

Here are two notable examples of foundation models in different domains:

• NLP (Natural Language Processing)
• Computer Vision

Large Language Model

LLM = a type Gen-AI to proccess and generate text

two type LLMs:

• Proprietary LLMs = owned by some company or organizarion.
• Open-source LLMs = made public under open-source license.