1. 概率：
1.1 定义 概率(P)robability: 对一件事情发生的可能性的衡量
1.2 范围 0 <= P <= 1
1.3 计算方法：
1.3.1 根据个人置信
1.3.2 根据历史数据
1.3.3 根据模拟数据
1.4 条件概率：
2. Logistic Regression (逻辑回归)
2.1 例子
h(x) > 0.5
h(x) > 0.2
2.2 基本模型
测试数据为X（x0，x1，x2···xn）
要学习的参数为： Θ（θ0，θ1，θ2，···θn）
向量表示：
处理二值数据，引入Sigmoid函数时曲线平滑化
预测函数：
用概率表示：
正例(y=1)：
反例(y=0):
2.3 Cost函数
线性回归:
找到合适的 θ0，θ1使上式最小
Logistic regression:
Cost函数：
目标：找到合适的 θ0，θ1使上式最小
2.4 解法：梯度下降（gradient decent)
更新法则：
image.png
学习率
同时对所有的θ进行更新
重复更新直到收敛Python 实现：
import numpy as np
import random
# m denotes the number of examples here, not the number of features
def gradientDescent(x, y, theta, alpha, m, numIterations):
xTrans = x.transpose()
for i in range(0, numIterations):
hypothesis = np.dot(x, theta)
loss = hypothesis - y
# avg cost per example (the 2 in 2*m doesn't really matter here.
# But to be consistent with the gradient, I include it)
cost = np.sum(loss ** 2) / (2 * m)
print("Iteration %d | Cost: %f" % (i, cost))
# avg gradient per example
gradient = np.dot(xTrans, loss) / m
# update
theta = theta - alpha * gradient
return theta
def genData(numPoints, bias, variance):
x = np.zeros(shape=(numPoints, 2))
y = np.zeros(shape=numPoints)
# basically a straight line
for i in range(0, numPoints):
# bias feature
x[i][0] = 1
x[i][1] = i
# our target variable
y[i] = (i + bias) + random.uniform(0, 1) * variance
return x, y
# gen 100 points with a bias of 25 and 10 variance as a bit of noise
x, y = genData(100, 25, 10)
print("x:")
print(x)
print("y:")
print(y)
m, n = np.shape(x)
print("m:")
print(m)
print("n:")
print(n)
numIterations = 100000 #训练次数
alpha = 0.0005 # 学习率
theta = np.ones(n)
theta = gradientDescent(x, y, theta, alpha, m, numIterations)
print("theta:")
print(theta)
结果：
x:
[[ 1. 0.]
[ 1. 1.]
[ 1. 2.]
[ 1. 3.]
[ 1. 4.]
[ 1. 5.]
[ 1. 6.]
[ 1. 7.]
[ 1. 8.]
[ 1. 9.]
[ 1. 10.]
[ 1. 11.]
[ 1. 12.]
[ 1. 13.]
[ 1. 14.]
[ 1. 15.]
[ 1. 16.]
[ 1. 17.]
[ 1. 18.]
[ 1. 19.]
[ 1. 20.]
[ 1. 21.]
[ 1. 22.]
[ 1. 23.]
[ 1. 24.]
[ 1. 25.]
[ 1. 26.]
[ 1. 27.]
[ 1. 28.]
[ 1. 29.]
[ 1. 30.]
[ 1. 31.]
[ 1. 32.]
[ 1. 33.]
[ 1. 34.]
[ 1. 35.]
[ 1. 36.]
[ 1. 37.]
[ 1. 38.]
[ 1. 39.]
[ 1. 40.]
[ 1. 41.]
[ 1. 42.]
[ 1. 43.]
[ 1. 44.]
[ 1. 45.]
[ 1. 46.]
[ 1. 47.]
[ 1. 48.]
[ 1. 49.]
[ 1. 50.]
[ 1. 51.]
[ 1. 52.]
[ 1. 53.]
[ 1. 54.]
[ 1. 55.]
[ 1. 56.]
[ 1. 57.]
[ 1. 58.]
[ 1. 59.]
[ 1. 60.]
[ 1. 61.]
[ 1. 62.]
[ 1. 63.]
[ 1. 64.]
[ 1. 65.]
[ 1. 66.]
[ 1. 67.]
[ 1. 68.]
[ 1. 69.]
[ 1. 70.]
[ 1. 71.]
[ 1. 72.]
[ 1. 73.]
[ 1. 74.]
[ 1. 75.]
[ 1. 76.]
[ 1. 77.]
[ 1. 78.]
[ 1. 79.]
[ 1. 80.]
[ 1. 81.]
[ 1. 82.]
[ 1. 83.]
[ 1. 84.]
[ 1. 85.]
[ 1. 86.]
[ 1. 87.]
[ 1. 88.]
[ 1. 89.]
[ 1. 90.]
[ 1. 91.]
[ 1. 92.]
[ 1. 93.]
[ 1. 94.]
[ 1. 95.]
[ 1. 96.]
[ 1. 97.]
[ 1. 98.]
[ 1. 99.]]
y:
[ 26.27815269 32.66768058 28.22594145 30.77125223 29.22859695
38.02617578 38.77723704 38.75941693 37.51914005 34.70311263
39.38349805 36.82172645 37.53424558 43.89335788 39.86619043
42.77143872 46.97544428 50.24971924 45.30721118 44.55195142
51.70691022 46.56863106 52.32805153 52.84954093 54.55242641
52.14422122 56.27667761 56.98691298 53.56176317 63.44462043
60.08578544 65.41098273 65.92701345 64.24412903 67.53920778
63.35080039 64.43398594 63.34590094 63.11265328 67.07000322
69.69430602 67.07964006 71.26126237 72.33061819 78.99023496
78.62644886 75.33387876 74.23899871 80.06708854 81.03063236
82.09372834 76.65280126 86.38648144 86.39932245 79.56509259
86.62380336 88.00737772 91.95667651 86.30124993 91.39647352
87.791776 88.80877001 96.1679461 94.8139934 98.44559598
98.55320134 99.85464471 96.56094905 97.31222944 94.19160055
98.14492827 99.80317251 101.65405055 102.29465893 100.20862392
106.37400148 108.33447212 110.41768632 105.49789886 104.64961868
109.37812661 107.69358766 109.10927721 109.93432977 109.13875359
116.8197377 111.26240862 120.88567915 117.93786525 122.85693307
120.26210017 116.99993199 124.74461618 124.99292528 122.0402555
123.9124033 122.28379028 127.65976993 132.21417455 125.08727085]
m:
100
n:
2
theta:
[29.86975704 1.00423275]
