Tensorflow 다변수 선형 회귀


다변수(Multivariable)

기존의 단순선형회귀에서 하나의 feature(x)를 다뤘던 것에서 현실의 문제를 더 잘 다룰 수 있도록 다수의 x를 사용하는 선형회귀를 다변수 선형회귀라고 한다.

가설의 변화

\(H(x) = Wx+b\) \(H(x_1,x_2) = w_1x_1 + w_2x_2 + b\) \(H(x_1,...,x_n) = w_1x_1 + ... + w_nx_n + b\)

\[cost(W,b) = \frac{1}{m} \sum^m_{i=1}(H(x^{(i)}-y^{(i)})^2\] \[[w1 w2 w3] \times \begin{bmatrix} x_{1}\\ x_{2}\\ x_{3} \end{bmatrix} = [w_1\times x_1 + w_2 \times x_2 + w_3 \times x_3 + b]\] \[H(X) = WX + b\] \[[b w1 w2 w3] \times \begin{bmatrix} 1\\ x_{1}\\ x_{2}\\ x_{3} \end{bmatrix} = [b \times 1 + w_1\times x_1 + w_2 \times x_2 + w_3 \times x_3]\] \[H(X) = WX\] \[\begin{bmatrix} b\\ w_{1}\\ w_{2}\\ w_{3} \end{bmatrix} \times \begin{bmatrix} 1\\ x_{1}\\ x_{2}\\ x_{3} \end{bmatrix} \neq [b w1 w2 w3] \times \begin{bmatrix} 1\\ x_{1}\\ x_{2}\\ x_{3} \end{bmatrix}\]

Transpose

\[\begin{bmatrix} w_{1} && w_{4}\\ w_{2} && w_{5}\\ w_{3} && w_{6} \end{bmatrix}^T = \begin{bmatrix} w_{1} && w_{2} && w_{3}\\ w_{4} && w_{5} && w_{6} \end{bmatrix}\]

어떨 때 우리는 \(H(X) = W^T X\)를 사용하기도 한다.

import tensorflow as tf

# W contains two elements and bias
W = tf.Variable(tf.random_uniform([1,3], -1.0, 1.0))

x_data = [[1, 1, 1, 1, 1],
          [0.,2.,0.,4.,0.],
          [1.,0.,3.,0.,5.]]

y_data = [1,2,3,4,5]

# Matrix multiply. we don't need to add bias
hypothesis = tf.matmul(W, x_data)

# Simplified cost function
cost = tf.reduce_mean(tf.square(hypothesis - y_data))

# Minimize
a = tf.Variable(0.1)
optimizer = tf.train.GradientDescentOptimizer(a)
train = optimizer.minimize(cost)

init = tf.global_variables_initializer()

sess = tf.Session()
sess.run(init)

# Fit the line
for step in range(2001):
    sess.run(train)
    if step % 50 == 0:
        print(step,sess.run(cost),sess.run(W))

>>

0 2.28883 [[-0.07839197  0.96626461  1.59019899]]
50 0.000293455 [[-0.04061023  1.01266611  1.01067531]]
100 1.3238e-05 [[-0.00862522  1.0026902   1.00226748]]
150 5.9704e-07 [[-0.00183185  1.00057137  1.00048161]]
200 2.69668e-08 [[ -3.89174995e-04   1.00012136e+00   1.00010240e+00]]
250 1.21759e-09 [[ -8.27761978e-05   1.00002587e+00   1.00002170e+00]]
300 5.51985e-11 [[ -1.75806272e-05   1.00000548e+00   1.00000465e+00]]
350 2.45919e-12 [[ -3.73804278e-06   1.00000119e+00   1.00000095e+00]]
400 3.24718e-13 [[ -1.00576585e-06   1.00000036e+00   1.00000036e+00]]
450 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
500 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
550 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
600 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
650 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
700 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
750 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
800 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
850 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
900 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
950 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1000 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1050 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1100 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1150 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1200 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1250 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1300 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1350 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1400 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1450 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1500 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1550 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1600 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1650 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1700 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1750 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1800 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1850 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1900 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
1950 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]
2000 2.06057e-14 [[ -3.00046963e-07   1.00000012e+00   1.00000012e+00]]