3.1 MNIST

Goal

숫자 이미지를 주었을때 해당 숫자를 리턴하는 함수를 만들어 보자.

def predict(img):
    //이미지를 가지고 숫자를 결정
    return num # (0-9)

Data set

MNIST

SubGoal

  • 함수를 훈련한다.
  • 훈련하는 그래프를 본다.
  • 예측한다.
  • 평가한다.

Term

  • epoch: entire dataset is passed through NN only once.

Model 1

  • x -> softmax(linear) -> y
  • $x_{784} \to h_{10} \to y_{10}$
  • loss = sum(softmax_cross_entropy_with_logits)
  • optimizer = GradientDescentOptimizer

$$ \begin{align} \text{input } = & \ x_{m,784} \\ h_{m,32} = & \ \text{softmax}(x_{m,32} W_{32,10} + b_{10}) \\ \text{output } = & \ y_{m, 10} \end{align} $$

Model 2

  • x -> relu(linear) -> softmax(linear) -> y
  • $x_{784} \to h_{32}^{(1)} \to h_{10}^{(2)} \to y_{10}$
  • loss = sum(softmax_cross_entropy_with_logits)
  • optimizer = GradientDescentOptimizer

$$ \begin{align} \text{input } = & \ x_{m,784} \\ h_{m,32}^{(1)} = & \ \text{relu}(x_{m,784} W_{784,32}^{(1)} + b_{32}^{(1)}) \\ h_{m,10}^{(2)} = & \ \text{softmax}(h_{m,32}^{(1)} W_{32,10}^{(2)} + b_{10}^{(2)}) \\ \text{output } = & \ y_{m, 10} \end{align} $$ code

Model 3

  • x -> relu(linear) -> relu(linear) -> softmax(linear) -> y
  • $ x_{784} \to h_{32}^{(1)} \to h_{32}^{(2)} \to h_{10}^{(3)} \to y_{10}$

$$ \begin{align} \text{input } = & \ x_{m,784} \\ h_{m,32}^{(1)} = & \ \text{relu}(x_{m,784} W_{784,32}^{(1)} + b_{32}^{(1)}) \\ h_{m,32}^{(2)} = & \ \text{relu}(x_{m,32} W_{32,32}^{(2)} + b_{32}^{(1)}) \\ h_{m,10}^{(3)} = & \ \text{softmax}(h_{m,32}^{(2)} W_{32,10}^{(3)} + b_{10}^{(3)}) \\ \text{output } = & \ y_{m, 10} \end{align} $$