Glossary of terms

Some of the most important terms you should know

AI: Short for artificial intelligence, this broadly refers to the idea of computers that can learn and make decisions in a human-like way.

Machine learning: A subfield of artificial intelligence, this is the practice of teaching computers to recognize patterns through data and algorithms. It differs from traditional programming in that the computer doesn’t need to be explicitly coded to address every potential scenario. From the traditional deductive approach (input + rules = outcomes), it is used a inductive approach (examples -> rules (model), input + model -> outcomes)

Neural network: A network of artificial neurons or perceptrons. Each artificial neurons takes a collection of inputs that are multiplied by a weight, then a simple function is applied to the sum of inputs * weights to give a final value that it is transmitted as input to other neurons. The training process consists of get a optimal collection of weights for all the neurons using a large data set of real examples.

Deep learning: Describes a neural network whose data passes through several layers of processing—some of which are hidden from the programmer—before arriving at a response. AI tools such as ChatGPT and Stable Diffusion are examples of applications that use deep learning techniques.

Language model: In artificial intelligence (AI), a language model is a statistical model that is used to predict the likelihood of a sequence of words. It is often used in natural language processing (NLP) tasks such as machine translation, speech recognition, and text generation.

A language model estimates the probability of a sequence of words by considering the context in which they appear. For example, in the sentence "The cat sat on the mat," the word "sat" is more likely to appear after "The cat" than after "the mat." A language model would assign a higher probability to the sequence "The cat sat" than to the sequence "the mat sat."

Language models are typically trained on large datasets of text and use techniques such as statistical modeling, machine learning, and neural networks to learn the patterns and relationships between words. They are an important component of many NLP systems and are used to improve the accuracy and fluency of machine-generated text.

GPT: Short for “Generative Pre-Trained Transformer,” this is an AI language model that uses deep learning to generate human-like text, created by OpenAI. The name itself requires some unpacking:

  • “Generative” refers to its ability to generate text.

  • “Pre-Training” GPT involves pre-training on a large corpus of text in an unsupervised way.

  • A “Transformer” is a kind of neural network that holistically learns about the relationship between all parts of a data sequence (in this case, the words in a sentence). It’s seen as a breakthrough for AI because it understands context and nuance better than previous approaches.

GPT-3: The third-generation language model created by OpenAI. It forms the basis for a slew of AI writing tools that have launched over the past two years, using OpenAI’s API. (ChatGPT uses an improved version, called GPT-3.5, while GPT-4 is in development.)

ChatGPT: ChatGPT is built on top of GPT-3 and improved with a machine learning technique called Reinforcement Learning with Human Feedback. This is a process where you label generated results as good or bad, and feed those ratings back to the machine learning model, and the model improves over time.

OpenAI: The AI research company behind GPT-3, ChatGPT and DALL-E. It began as a non-profit group, but now operates a “capped-profit” company that employs most of its staff.

Diffusion model: A method for creating images from text prompts. It works by adding random noise to a set of training images, then learning how to remove noise to construct the desired image.

Last updated