Theory of Intelligence
DEEP LEARNING•Reinforcement And Frontier•Frontier Architectures•2.1 Graph Neural Networks

Introduction

Deep learning on networks of connected things: representing data as graphs, the message-passing idea at the heart of every GNN, and the GCN and GAT layers.

Introduction

A lot of important data isn't a grid (like an image) or a sequence (like text). It's a network of connected things:

  • a social network: people connected by friendships;
  • a molecule: atoms connected by chemical bonds;
  • a road map: junctions connected by roads;
  • a knowledge graph: facts like (Paris, capital of, France);
  • recommendations: users connected to the products they bought.

These are graphs. Their size and shape vary, nodes have different numbers of neighbours, and there's no natural "first" or "left" node, so CNNs and RNNs don't fit. Graph neural networks (GNNs) are built for exactly this kind of data.

Think of how opinions form in a group of friends. Each person starts with their own view, then talks to their friends and updates it based on what they hear. After a few rounds of conversation, each person's opinion reflects not just their friends, but their friends' friends too. GNNs work the same way: each node repeatedly gathers information from its neighbours.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!