What Are Word Vectors?
The problem: Computers don't understand words
When you type "cat" on your keyboard, the computer sees something like 01100011 01100001 01110100.
It's just a sequence of bits representing characters. The computer has no idea that a cat is a furry animal
that meows and chases mice.
This is a fundamental problem. How can AI understand language if it can only process numbers?
The solution: Convert words to vectors
The breakthrough idea is simple: represent each word as a list of numbers (a vector) that captures its meaning.
These numbers aren't random. They're learned from analyzing billions of words in books, websites, and documents. Words that appear in similar contexts end up with similar numbers.
Key insight: "Cat" and "dog" will have similar vectors because they appear in similar contexts (pets, animals, furry, etc.). "Cat" and "democracy" will have very different vectors.
Why this matters for LLMs
Large Language Models like Claude are built on this foundation. Before any text reaches the AI's "brain," it first gets converted into these numerical representations. This is how the model "understands" what you're asking.
Understanding word vectors helps you understand:
- Why similar prompts give similar results
- How AI captures relationships between concepts
- The foundation of modern AI language understanding
Key Takeaways
- Word vectors convert words into lists of numbers
- Similar meanings result in similar numbers
- This is the foundation of how LLMs understand language