CS Theory

Key Figures

  • Alan Turing
  • Alonzo Church
  • Kurt Godel

Alan Turing

His Life

  • 1912-1957
  • Cracked Enigma machine
  • Turing Test
  • Turing Machine

Enigma

Substitution cipher, mapping changed with each key press

Bombe

Stops

  • Bombe contained multiple simulated enigma machines.
  • input: intercepted ciphertext and plaintext guess
  • output: Engima rotor settings that could produce the mapping

General Computing

The Bombe was a good computer for a specific use, but what about general purpose computers?

Turing Machine

Lego Turing Machine!

TM Definition

  • infinite tape, divided into cells
  • head, read/write cells
  • state, limited memory
  • instruction table, state + current cell =
    • write or erase current cell
    • move head left or right one cell
    • change state

Church-Turing Thesis

If a function of natural numbers can be calculated by a person following a systematic process with an paper, pencil, and no limitations on time or other resources, it is computable.

All computable functions can be computed by a Turing machine.

Universal Turing Machine

A Turing machine can simulate other Turing machine! This is analogous to a modern computer running a program. The idea of a UTM encouraged the invention of general purpose computers.

Alonzo Church

His Life

  • 1903 - 1995
  • Lambda Calculus
  • Church-Turing Thesis

Church-Turing Thesis Part 2

Turing machines and Lambda Calculus are equivalent.

$$x$$ Variable
$$\lambda x . M$$ Function with argument x and body M
$$(M N)$$ Applying function M to N
$$\lambda x . x$$ Identity
$$\lambda x . x y$$ Constant
$$(\lambda x . x) \; s \rightarrow s$$
$$\text{TRUE} := \lambda x . \lambda y . x $$ $$\text{FALSE} := \lambda x . \lambda y . y $$ $$\text{AND} := \lambda p . \lambda q . p\; q\; p\; $$ $$\text{OR} := \lambda p . \lambda q . p\; p\; q\; $$ $$\text{NOT} := \lambda p . p\; \text{TRUE}\; \text{FALSE} $$
$$\text{AND} \; \text{TRUE} \; \text{FALSE}$$ $$\lambda p . \lambda q . p\; q\; p\;\; \text{TRUE} \; \text{FALSE}$$ $$\text{TRUE} \; \text{FALSE} \; \text{TRUE}$$ $$\lambda x . \lambda y . x \; \text{FALSE} \; \text{TRUE}$$ $$FALSE$$

Y-Combinator

Recursively applies a function in lambda calculus, it is essentially the equivalent of a 'while' or 'for' loop.

$$\text{Y} := \lambda f . (\lambda x . (f (x\; x) ) \lambda x . (f\; (x\; x) ) ) $$

Church-Turing Thesis Conclusion

Turing machines and Lambda calculus are equivalent theoretical basis for computer science because both systems can represent anything that is computable.

Kurt Godel

His Life

  • 1906-1978
  • Incompleteness Theorems
  • Godel's Ontological Argument

David Hilbert

  • A Founder of Mathematical Logic
  • A Founder of Proof Theory

Hilbert's Quest

"Restart" math (and human knowledge?) on the correct foundational principles and build a completely rational, logical view of both math and reality.

Logical Positivism

Hilbert's views are related to philosophical school of Logical Positivism, that believed all legitimate knowledge is scientific.

Curry-Howard Isomorphism

How is this connected to CS?
Proofs = Programs = Logic

Pre-modern Mathematics

Based on intuition, simple observations e.g. addition "just works"

Euclid's Elements

5 Axioms of Euclidean Geometry

  1. A straight line can be draw from any point to any point.
  2. Lines are infinite in length.
  3. Circles are described by a center and radius.
  4. All right angles are equal to each other.
  5. A straight line that crosses two other straight lines will produce 2 pairs of angles both less than right angles

Hilbert's Question

What are the axioms of numbers?

"Tower of Knowledge"

If Hilbert was building a tower of knowledge and arithmetic is the "ground floor" that is obviously visible to everyone, what is the foundation?

Why does this matter?

  1. Firm foundation: avoid the "garbage in, garbage out" problem
  2. Can computers speed up the process?

Non-Euclidean Geometry

Maps vs Globe

Wasted Time

Many mathematicians worked on trying to prove the 5th axiom of geometry from the first 4. Eventually they realized that it is just an assumption.

  1. A straight line can be draw from any point to any point.
  2. Lines are infinite in length.
  3. Circles are described by a center and radius.
  4. All right angles are equal to each other.
  5. A straight line that crosses two other straight lines will produce 2 pairs of angles both less than right angles

Math is Logic

Different assumptions lead to different outcomes.

Modern Mathematics

Math is makes some assumptions, called axioms. From these axioms theorems are deduced via proof.

Peano Axioms

  1. 0 is a natural number
  2. x = x
  3. if x = y then y = x
  4. if x = y, y = z then x = z
  5. If x is a number, and s is a successor function then s(x) is a number

Strange, but enough

The number 4 is represented as s(s(s(s(0)))).

Are these axioms correct?

  1. Will anything be left out? A true theorem/conclusion with no possible proof.
  2. Will any contradictions arise?

Hilbert's Quest

"We can know, we must know"

Godel's Doubts



Can every truth about numbers be found and proven?

Three Elements

  1. Math about math
  2. Cantor's Diagonalization
  3. Self reference paradox

Godel Numbers

Godel developed a system, he gave every theorem an "id" number. The number is like a string, it shows the steps/other theorems needed to prove the theorem.
For example, "12 45 932" is the theorem that uses the theorems 12, 45, and 932

Math about math

Godel numbers mean that math can be used to make proofs about math!

System A is normal math, System B is "meta". Proofs in System B can describe properties of System A.

Cantor's Diagonalization

Godel's proof is similar to Cantor's proof about lack of correspondence between natural and real (decimal) numbers.

Paradox

"This sentence is false"

Incompleteness Theorem Core

"This theorem is false"

Read More

Godel's Incompleteness theorems

  • Theorem 1: no set of consistent axioms can prove everything that is true about natural numbers
  • Theorem 2: no axiom system can prove its own consistency

Hilbert's doomed quest

There is no set of axioms to prove everything that is true about numbers.

Examples of Undecidable Theorems

It gets worse

Godel framed the work of mathematicians as a "search" problem like "satisfiability". The process of proving theorems i.e. the work of mathematicians is computationally intractable.

Problems with proofs

There is no limit to the length of a Godel number e.g. there could be proofs that are a billion pages long.

What does this mean?

Truth beyond proof is possible.

There are real, provable limitations to what humanity can understand and know logically.

Conclusion

  • There are real limits to what humanity can know mathematically, computationally, & logically
  • Logical systems of thought are always grounded in unprovable beliefs i.e. axioms