神经网络BP算法求导-手动推导

2018年02月27日

文章目录

1. 手动推导
2. 概念抽象
3. 代码实现

神经网络BP算法我这里不多说, 大家自行百度. 我这篇文章要手动计算一下神经网络算法求偏导数的过程, 其中用到了反向传播和链式法则. 为了简单, 我们用一个只有两层, 每层只有两个神经元的神经网络, 其网络结构如下:

其中, x代表神经元输入, o代表神经元输出, 假设激活函数用sigmoid, 则有:

\[ o = {\sigma(x)} = {1 \over {1 + e^{-x}\ } } \]

再看途中w代表权重, w13 代表神经元1和神经元3之间的连接权重, 那么神经元4的输入x4可以计算:

\[ x_4 = o_1 w_{14} + o_2 w_{24} \]

根据激活函数, 可以知道神经元4的输出为:

\[ o_4 = {\sigma(x_4)} = {1 \over 1 + e^{-x_4}\ } \]

其他神经元的输入输出可以一次类推计算出来.

神经元5的输入和输出分别是:

\[ x_5 = o_3 w_{35} + o_4 w_{45} \]

\[ o_5 = {\sigma(x_5)} ={ 1 \over 1 + e^{-x_5}\ } \]

假如我们的目标值为t, 也就是神经元5输出值的目标值. 那么我们的目标函数应该是最小化均方误差:

\[ E = 0.5(o_5-t)^2 \]

手动推导

至此, 我们要用到的信息基本都有了, 那么神经网络算法如何求梯度呢? 也就是参数的偏导数. 我们知道神经网络的参数就是权重w, 求梯度就是求E对每个w的偏微分, 那么下面先来求E对w45的偏微分.

根据链式法则有:

\[ {\ \partial E \over \partial w_{45}\ } = {\partial E \over \partial o_5}{\partial o_5 \over \partial x_5}{\partial x_5 \over \partial w_{45}\ } \]

分别求三个偏微分:

\[ \begin{align} {\partial E \over \partial o_5} &= {o_5-t} \\ {\partial o_5 \over \partial x_5} &= {\sigma(x_4)(1-\sigma(x_4))} \\ {\partial x_5 \over \partial w_{45}\ } &= {w_{35}\ } \end{align} \]

最后求得:

\[ {\ \partial E \over \partial w_{45}\ } = {(o_5-t)}{\sigma(x_4)(1-\sigma(x_4))}x_4 \]

同样的道理, w35的偏导也能求得:

\[ {\ \partial E \over \partial w_{35}\ } = {(o_5-t)}{\sigma(x_3)(1-\sigma(x_3))}x_3 \]

下面我们求一下E对w23的偏导数, 先列出已知的信息:
\[ \begin{align} E &= 0.5(o_5-t)^2 \\ o_5 &= {\sigma(x_5)} ={1 \over 1 + e^{-x_5}\ } \\ x_5 &= o_3 w_{35} + o_4 w_{45} \\ o_3 &= {\sigma(x_3)} = {1 \over 1 + e^{-x_3}\ } \\ o_4 &= {\sigma(x_4)} = {1 \over 1 + e^{-x_4}\ } \\ x_3 &= o_1 w_{13} + o_2 w_{23} \\ x_4 &= o_1 w_{14} + o_2 w_{24} \\ \end{align} \]

根据链式法则:

\[ {\ \partial E \over \partial w_{23}\ } = {\partial E \over \partial o_5}{\partial o_5 \over \partial x_5}{\partial x_5 \over \partial o_3 }{\partial o_3 \over \partial x_3 }{\partial x_3 \over \partial w_{23} } \]

同样分别求偏导:

\[ \begin{align} {\partial E \over \partial o_5} &= {o_5-t} \\ {\partial o_5 \over \partial x_5} &= {o_4(1-o_4)} \\ {\partial x_5 \over \partial o_3 } &= w_{35} \\ {\partial o_3 \over \partial x_3 } &= o_3(1-o_3) \\ {\partial x_3 \over \partial w_{23} } &= x_2 \end{align} \]

最后整合到一起就是:

\[ {\ \partial E \over \partial w_{23}\ } = (o_5-t)o_4(1-o_4)w_{35}o_3(1-o_3)x_2 \]

同样道理可以求得E对w24的偏导:

\[ {\ \partial E \over \partial w_{24}\ } = (o_5-t)o_4(1-o_4)w_{45}o_4(1-o_4)x_2 \]

概念抽象

代码实现

同样的神经网络结构, 我们准备一批数据:

from math import exp
# 初始城市距离
x1 = [29, 43, 50, 36, 2, 29, 94, 45, 83, 93, 41, 76, 39, 91, 45, 28, 28, 39, 57, 
      72, 12, 38, 52, 83, 32, 15, 100, 49, 83, 93, 50, 45, 72, 56, 36, 43, 11, 
      23, 67, 67, 2, 86, 14, 92, 51, 52, 94, 55, 87, 40, 14, 30, 24, 22, 17, 22, 
      7, 43, 71, 23, 32, 33, 53, 32, 27, 94, 19, 54, 76, 50, 67, 53, 2, 45, 73, 98, 
      19, 15, 9, 24, 64, 51, 62, 86, 58, 32, 53, 45, 52, 85, 84, 29, 83, 67, 91, 45, 71, 8, 76, 99]
x2 = [54, 47, 65, 79, 56, 80, 55, 65, 85, 67, 67, 42, 45, 56, 21, 67, 72, 50, 33, 2, 
      81, 55, 66, 13, 30, 63, 55, 7, 39, 19, 61, 43, 35, 69, 45, 69, 39, 83, 50, 60, 
      44, 10, 56, 99, 59, 17, 66, 68, 41, 83, 26, 42, 79, 0, 14, 48, 44, 96, 65, 37, 
      61, 85, 72, 40, 24, 82, 0, 44, 58, 35, 43, 28, 67, 58, 37, 28, 34, 40, 61, 45, 
      22, 28, 60, 91, 81, 51, 83, 75, 97, 91, 58, 83, 68, 57, 55, 38, 87, 59, 100, 80]

y = [973, 202, 77, 541, 59, 796, 429, 187, 131, 63, 853, 938, 40, 256, 738, 335, 77, 357, 
      114, 308, 100, 210, 906, 10, 12, 677, 181, 710, 250, 30, 594, 936, 160, 133, 83, 909, 
      222, 71, 42, 43, 591, 763, 33, 240, 479, 839, 144, 102, 27, 137, 9, 22, 65, 9, 220, 80, 
      25, 535, 216, 721, 794, 161, 63, 186, 22, 466, 35, 49, 463, 18, 354, 871, 638, 367, 216, 
      223, 818, 68, 38, 208, 604, 435, 47, 434, 135, 29, 710, 62, 457, 353, 899, 79, 64, 44, 
      39, 512, 119, 105, 114, 148]

# 对输出进行尺度调整至（0，1）范围
# 这种方法得到的值都在1附近
s = [1 / (1 + exp(-i)) for i in y]
# 这种方法比较好
p1 = [i / max(x1) for i in x1]
p2 = [i / max(x2) for i in x2]
s = [i / max(y) for i in y]

data = (p1, p2, s)

训练代码:

import numpy as np 

def bp_back(training_example, eta):
    # m:行, n:列, 一列代表一组输入, 一行代表一个变量的所有样本
    # 所以n代表样本量
    m, n = training_example.shape
    # 初始化权值矩阵
    # 第一层 2x2矩阵
    # [[w13, w14],[w23, w24]]
    w = np.random.rand(2,2)-0.5
    # 第二层 2x1矩阵
    # [[w35],[w45]]
    v = np.random.rand(2).reshape((1, 2))-0.5
    for t in range(10):
        Es =[]
        for num in range(1, n):
            one_sample = training_example[:, num] # 取一组
            x = one_sample[:2]
            y = one_sample[2:]
            # 隐层
            net2 = np.dot(w, x)
            # print(net2)
            hidden = np.exp(net2)
            net3 = np.dot(v, hidden)
            # 输出值
            output = np.exp(net3)
            # print(net3)
            # print(output)
            E = 0.5*(output-y)**2
            Es.append(E[0])
            # print('第{}次循环, 第{}个样本, 误差:{}'.format(t, num, E))
            # -----反向传播---------
            # 计算各层delta值
            # 输出层
            delta3=(y-output)*output*(1-output)
            # 隐层
            # print(np.dot(delta3, v))
            # print(hidden*(1 - hidden))
            delta2=np.dot(hidden*(1-hidden)*np.dot(delta3, v), w)
            # print(delta2)
            # 更新权重
            v += eta*delta3*hidden
            w += eta*delta2*x
        print('第{}次循环, 总体误差:{}'.format(t, sum(Es)))
        print('每个样本误差:', Es)


if __name__ == '__main__':
    from bp_data import data
    training_example = np.array(data)
    # print(training_example)
    bp_back(training_example, 0.5)

结果输出:

第0次循环, 总体误差:4.744542968142258
每个样本误差: [0.09307431216223538, 0.13482397893216636, 0.00018704728019664, 0.1046919893614765, 0.058956017922238216, 0.0036282732958129666, 0.050450890167652536, 0.06517168771792486, 0.07738121132961508, 0.11230615882898966, 0.13206927130629312, 0.10011690699810656, 0.02032154767301941, 0.05920000032102159, 0.006656422232387443, 0.0679442967849832, 0.0008677025088858484, 0.04162054516688042, 0.0014801508865888624, 0.034694549663665024, 0.009749293552389859, 0.16636888478898437, 0.0710884393181918, 0.05743462849231556, 0.0687225801146502, 0.021118042994188277, 0.07731371723037268, 0.009011379065124923, 0.058955897118622855, 0.03327133691287055, 0.1789504977069153, 0.031715810217841235, 0.03567017546619412, 0.03931500832823137, 0.16877492843490846, 0.010990866980906138, 0.04632053691772168, 0.05232127410675336, 0.04615033491133325, 0.04959695312638082, 0.10116160124325828, 0.0499072973336097, 0.007932525679858117, 0.012083239432233776, 0.13969503544745038, 0.03315587129262023, 0.0348070597043839, 0.054299956863141464, 0.016181924029705454, 0.039197720240548545, 0.03432475570460683, 0.020210146967107337, 0.02825411290125502, 8.35556257127824e-05, 0.013446237428159653, 0.02099870430501065, 0.04804258527146816, 0.0012262884444965325, 0.1250634929991989, 0.14934943964627748, 0.00885792852228099, 0.02841111217574339, 0.0035404672017931033, 0.029479948923050767, 0.016381135690093925, 0.023232901236498803, 0.02407178917746283, 0.0204323636088328, 0.030349738117992922, 0.004964066077222589, 0.20299524746261138, 0.077406799514915, 0.002251242858983825, 0.005013448792635658, 0.004363235268089541, 0.15367845461322194, 0.030916451147753345, 0.03419591474355951, 0.002979797552180166, 0.051792057225486236, 0.009045319818791777, 0.04082751571907755, 0.006361209444404315, 0.016983912287479187, 0.035213247505465006, 0.09518379919820484, 0.031199789826430173, 0.014196754477282516, 0.0004655563978522462, 0.17702753168991042, 0.032560626879791485, 0.03971569132031204, 0.03794836576830288, 0.03832239243657794, 0.030994410645955223, 0.017394929806560457, 0.01174252771919042, 0.015441139727277163, 0.01069905067784429]
第1次循环, 总体误差:4.502181777491771
每个样本误差: [0.0012700060400311598, 0.016522932028595234, 0.04873826234382247, 0.01578181182447815, 0.16446427991054896, 0.009052891870907567, 0.0046455741360792455, 0.014714776862181924, 0.028015591383956564, 0.18911712820325752, 0.21649487788119498, 0.04010730316154194, 0.002558353127111011, 0.10532001857551229, 0.00017314513702910286, 0.03071919891592851, 0.0013513845595703915, 0.021195985095411138, 7.60127215852222e-05, 0.01749444569334864, 0.0031550579994367267, 0.19949585869752826, 0.05223257420408922, 0.0417197299413162, 0.08686706546262214, 0.01299495015986755, 0.09280958816426604, 0.004318972333369115, 0.04747375326598561, 0.04315580313118569, 0.20046168069988676, 0.02317144506803086, 0.0266865304413249, 0.03145201575664053, 0.18473230641262856, 0.007114805722550368, 0.038027579184671464, 0.0462206504982442, 0.04105797250936242, 0.054635540247132464, 0.10671281382208357, 0.04435911915976863, 0.006135732470001004, 0.014140025488489394, 0.14519131376861055, 0.029954652107043234, 0.03162772852037402, 0.05152001516713817, 0.014386648703932331, 0.037213255463416764, 0.03263048951500987, 0.018799039490834994, 0.02737442325507797, 4.123501152087079e-05, 0.01268597627455187, 0.020115097943397862, 0.04960741315184319, 0.0010404239358350034, 0.1268144157609311, 0.1518319562123025, 0.008087312809286505, 0.027341518921012054, 0.00327227163536747, 0.028860313154113998, 0.01695190177384341, 0.022886453498333095, 0.02357270215063361, 0.020874091245249703, 0.029894374890739202, 0.005130582064571118, 0.20387667683492292, 0.07884590736837074, 0.00244745416771797, 0.004864147316093057, 0.004302279963069986, 0.15485797495703155, 0.030259953401556322, 0.03337372381816873, 0.00283219935700535, 0.05191626277954449, 0.009157597210177964, 0.040329105615334315, 0.006611338760075727, 0.01657225383033488, 0.03483110284720921, 0.09604649256785981, 0.030722821645282852, 0.014587144602478806, 0.0005128340794891227, 0.17723263916390225, 0.03199221816014485, 0.03957062607913407, 0.03784490914124141, 0.0383469337352142, 0.031021299358769783, 0.017164763124022882, 0.011558059484844537, 0.015203126914957909, 0.010648774470268851]
第2次循环, 总体误差:4.50209383116374
每个样本误差: [0.0012613361798074847, 0.016417973459872656, 0.04908512932387356, 0.015618020505986492, 0.16513061624625497, 0.008986838938196488, 0.004590977958504538, 0.014615622732475526, 0.02804873047955616, 0.18950515084421546, 0.21604464159060585, 0.040126161146987506, 0.002594935175893824, 0.1049310486104809, 0.0001909681667250087, 0.03046133443512142, 0.0013533645182198894, 0.021364811471012272, 5.441763226454322e-05, 0.017191750162103933, 0.0031341360096008367, 0.1997365360170566, 0.052846285147174925, 0.04182230916103678, 0.08727490941594492, 0.013101958039012073, 0.09225713450144259, 0.004399952712020484, 0.047966486197536906, 0.04332982475338354, 0.20048906759708862, 0.023339642242776733, 0.02650109008242775, 0.0313896697160414, 0.18529602390861782, 0.007065434815815943, 0.03754474387310269, 0.04632472041556352, 0.04104463073032525, 0.054827735777764175, 0.10586087774363721, 0.044106573575446166, 0.0060355671493624455, 0.014174255071731109, 0.1445411381046562, 0.03003529991927018, 0.03150847213408243, 0.05183267184056195, 0.01417496919566876, 0.03727653075731824, 0.032616444023574595, 0.01856285056463817, 0.02761783499375432, 4.589630256098282e-05, 0.012626787336487049, 0.020027102418648714, 0.050022314902156896, 0.0010351952093687476, 0.1268115375676597, 0.15211976701145383, 0.007930939125049816, 0.027236569888979857, 0.003288915927744943, 0.028997837164734988, 0.016989120172959293, 0.02311919714829562, 0.023629605628414114, 0.020845232236497174, 0.030001153842067016, 0.005090276195857431, 0.20348598278457516, 0.07931714345418078, 0.002466416544404575, 0.004948341368436139, 0.0044362742963262175, 0.15488141747135703, 0.030195441045555643, 0.03310865308444844, 0.002816779505798389, 0.051563162223746295, 0.009057760959356531, 0.040292674356201014, 0.006698392461250337, 0.016421793521015193, 0.03476951591115415, 0.0964360409513888, 0.030541023647400686, 0.014802006199359244, 0.0005292385801232739, 0.17690600336443466, 0.03171344688103629, 0.039665652989046785, 0.03794360020655491, 0.038542853898324604, 0.030910921449707272, 0.017048461068700095, 0.011457070901732348, 0.015052076992633487, 0.01065863120095895]
第3次循环, 总体误差:4.501825250607882
每个样本误差: [0.0012715264518135426, 0.01637889142609892, 0.049326982351990054, 0.015518124378057436, 0.1656111202476661, 0.008876214211966228, 0.004571613539081073, 0.014573419085074172, 0.028148158137075344, 0.18972863181434513, 0.2154103472383741, 0.04024397029565818, 0.0026517284194097325, 0.10442820117896899, 0.00020292031830110606, 0.03028255511531193, 0.0013431576085319633, 0.021576011537984604, 3.511613570847874e-05, 0.016934367363186986,
0.003126131581331897, 0.19988284027198758, 0.05350570042592307, 0.041966620773010305, 0.08762022312536813, 0.013229082409332666, 0.09166187927265652, 0.004492309401356834, 0.048488379402446546, 0.04346780868541996, 0.20044099461407955, 0.023533476172840677, 0.026350125275669597, 0.03135507358714391, 0.1857945304937314, 0.00703150982854957, 0.037107185192333254, 0.046446534513531486, 0.04104645767896824, 0.05499975711601556, 0.10500538819677512, 0.04387936810787768, 0.005941912576150481, 0.014202421574417196, 0.1438862344223261, 0.030122543744766615, 0.03140049478151737, 0.05214552693159448, 0.013972451983760886, 0.037343265032153086, 0.032604862605521326, 0.018332972831179937, 0.027860569651754655, 5.0773206797848936e-05, 0.01256844516109681, 0.01994037886363708, 0.05043641144999795, 0.0010297239502908661, 0.12680846324374223, 0.15240143938663495, 0.007780854081136127, 0.027135126031150147,
0.0033057647891532034, 0.029135040826811256, 0.017027614051537573, 0.023352051574074877, 0.023685611297066266, 0.020818104342713837, 0.030106836382957535, 0.005051016190108512, 0.20310054805905026, 0.07977779332720433, 0.0024842819298944853, 0.005032660847039665, 0.004570611580484247, 0.1548969981156132, 0.03013824489937508, 0.03285472247512336, 0.002802177029993951, 0.05121627621567326, 0.00895978030747251, 0.040257482333418464, 0.006785226215576935, 0.01627500757618664, 0.03470915341357972, 0.09682317257912786, 0.030364107252577634, 0.015015169748170694, 0.0005457453961753774, 0.17658677253409513, 0.03144681111946729, 0.03975827203779853, 0.03803926142884098, 0.03873290681727725, 0.03080410198617534, 0.016932604019750566, 0.011357612574031097, 0.014901120691179673, 0.010665284152526213]
第4次循环, 总体误差:4.501655539751194
每个样本误差: [0.0012810007374459932, 0.016337727149017926, 0.049571041298433566, 0.0154191502905733, 0.16609264787818742, 0.008768909266432524, 0.004552488457587328, 0.014530356763192495, 0.028243567449200487, 0.1899538997249965, 0.21478620526585016, 0.040363510390739814, 0.0027080233274512575, 0.1039320359856473, 0.00021472997757682373, 0.030110990339776177, 0.0013331217097075224, 0.0217845065015839, 2.0218445720393362e-05, 0.016684289468013525,
0.003117702968915142, 0.2000333944015917, 0.05415818881273483, 0.042109904219724686, 0.08795942780769156, 0.013354300432984757, 0.09107492273245561, 0.004583878570047355, 0.04900386963359642, 0.04360260691061347, 0.20038797942859046, 0.023727101534126518, 0.026206849992983845, 0.03132398370941548, 0.1862807271337394, 0.007001577816649997, 0.036688872438779746, 0.046565762603440176, 0.04104660344529882, 0.05516893013333573, 0.10416794173615296, 0.043659364378941114, 0.005849145121009201, 0.01423189364754228, 0.1432476399587152, 0.030205279073125056, 0.03129316361736822, 0.0524480232730553, 0.013773321464879426, 0.03740635471515059, 0.032589848631052076, 0.01810495271497343, 0.02809878543723228, 5.570281107423547e-05, 0.012508438919783203, 0.019852024061128327, 0.050854459926390295, 0.0010234020614932032, 0.12681146146437114, 0.15268521575478844, 0.007634606906872306, 0.027033686416006033, 0.0033218447236874242, 0.029269381138154, 0.01706931277336085, 0.023583138711193517, 0.023738836081285963, 0.020794311802326007, 0.030209557965669407, 0.0050134647955839716, 0.20272439402664696, 0.08023209466032187, 0.002501708436255113, 0.005116362270518572, 0.004704651163655204, 0.15490883648993545, 0.030086185642819157, 0.03260943046547132, 0.0027879143891497726, 0.050876810645680434, 0.008864239336093371, 0.04022191599593699, 0.006872527606976508, 0.01613075556624254, 0.03464884151904811, 0.09721008479385566, 0.030190776468417658, 0.015227541664627206, 0.0005624770578042486, 0.17627613082306104, 0.031190774985234176, 0.03984790290819274, 0.038131398765409344, 0.038916767357488356, 0.030701124521315123, 0.01681668002737729, 0.011259248028938965, 0.014749802962360161, 0.010668595936147865]
第5次循环, 总体误差:4.5015829548229975
每个样本误差: [0.001289707804815632, 0.016294251592264976, 0.0498179888494067, 0.015320744457033126, 0.1665764756363082, 0.008664851160191172, 0.004533480028655703, 0.014486246505671457, 0.02833491527743062, 0.19018174659944698, 0.21417187304878257, 0.04048470025657677, 0.0027638093228498223, 0.10344228156077849, 0.00022639575275375302, 0.029946080266968017, 0.0013232713083051806, 0.021990382313238382, 9.534283354484233e-06, 0.016440829596439232, 0.0031088264052454404, 0.20018848845461928, 0.054804111079948745, 0.042252327240697274, 0.08829313496374984, 0.013477769874535766, 0.0904955682865953, 0.004674732907585553, 0.049513369834711905, 0.04373440560521315, 0.20032988587458264, 0.023920804613817834, 0.026070970021056673, 0.031296362704219295, 0.18675538794030394, 0.006975570154330517, 0.036288652352273146, 0.04668267883958205, 0.041045162145841745, 0.055335486873460946, 0.10334743545618993, 0.043446207022046886, 0.005757159353576704, 0.014262651992589496, 0.14262437173488227, 0.030283745465974423, 0.031186410722764273, 0.052740549336285705, 0.013577238257058288, 0.03746595082175152, 0.032571458296497754, 0.017878449086541782, 0.02833271534301041, 6.0668012074977177e-05, 0.012446745169993554, 0.019761973660390127, 0.051277051532142494, 0.0010162480439617743, 0.12682040037110226, 0.1529713515961217, 0.007491913608523561, 0.026932216186451207, 0.003337205945914931, 0.02940109177361338, 0.0171141921359362, 0.023812735723690553, 0.023789426280209164, 0.020773732687621796, 0.030309519005365406, 0.004977521571307425, 0.2023568380031319, 0.08068073005354169, 0.0025186892487028393, 0.005199567542217719, 0.004838489288318679, 0.15491677928373482, 0.03003925483928631, 0.032372309507056565, 0.0027739933840318486, 0.050544187014554105, 0.008770933757055885, 0.040186047480268106, 0.006960387899961615, 0.015988840446957513, 0.03458858146348635, 0.0975972572835421, 0.03002080958707491, 0.015439388643367516, 0.0005794402832418634, 0.17597329648825163, 0.03094487346669481, 0.0399348469256211, 0.038220265026402356, 0.039094749314543924, 0.03060176011035619, 0.016700581021201484, 0.01116184598282629, 0.014597971259511794, 0.010668642230823871]
第6次循环, 总体误差:4.501605217174395
每个样本误差: [0.0012976674664389796, 0.016248472522430987, 0.05006813492315359, 0.015222789266512155, 0.16706321585669012, 0.008563803752886194, 0.004514594148929642, 0.01444109897313673, 0.02842240056753414, 0.19041236877758896, 0.2135662906591705, 0.04060784353691085, 0.0028191557239606543, 0.10295819563195668, 0.0002378950953542833, 0.02978756011159985, 0.0013135754427695875, 0.022193901712782654, 2.8840748401707985e-06, 0.016203470193874393, 0.003099523341778673, 0.20034809804526668, 0.05544406703759431, 0.04239422289288286, 0.08862174227501203, 0.01359972423801866, 0.08992291129456452, 0.004764995087099896, 0.05001744922730403, 0.04386327657183403, 0.20026630904413786, 0.024114979746166165, 0.02594230717597944, 0.03127227012136621, 0.1872190905297509, 0.006953479319779926, 0.03590555073562455, 0.046797611548124146, 0.04104226245406527, 0.05549959605914291, 0.10254269118486779, 0.04323962023581409, 0.005665856773608033, 0.014294665731533129, 0.14201537932774344, 0.03035819467676156, 0.031080189569693436, 0.053023511434915925, 0.013383883041366404, 0.03752222463624934, 0.03254975134782228, 0.01765313092935447, 0.028562621434350993, 6.565461518314286e-05, 0.012383336476392365, 0.019670161629847322, 0.051704813292726255, 0.0010082768686936827, 0.12683515174470192, 0.15326011141660129, 0.007352508375506724, 0.026830675618222673, 0.0033519000816687305, 0.029530418106971063, 0.01716225368825092, 0.02404115057870026, 0.023837522145142, 0.02075626326523131, 0.03040692002403533, 0.004943092329208686, 0.20199719768866403, 0.08112435793880979, 0.0025352155426716827, 0.005282406077505768, 0.004972237506291855, 0.15492063697423974, 0.029997475000035894, 0.032142931188255806, 0.002760417035537887, 0.05021782956874063, 0.008679662360908611, 0.04014993701492093, 0.007048913127079878, 0.015849063855412772, 0.03452837139955961, 0.09798520337858595, 0.02985398926893116, 0.015650990040071398, 0.000596645019567768, 0.1756775221727273, 0.030708689909705423, 0.040019386897047836, 0.03830609792336132, 0.03926714564760046, 0.03050578945464535, 0.01658418280819489, 0.011065277441583615, 0.014445453502252577, 0.010665475666307296]
第7次循环, 总体误差:4.501720565786242
每个样本误差: [0.0013048976805702642, 0.016200381803095146, 0.05032181953240226, 0.01512516960501414, 0.16755351811219585, 0.008465544078056192, 0.0044958357028848095, 0.014394909616083184, 0.028506202362832182, 0.19064598861113768, 0.21296839621612515, 0.04073327021060456, 0.0028741343594354024, 0.10247900675462528, 0.00024920621309561594, 0.029635193516428737, 0.0013040033430204742, 0.022395338087843705, 1.0656406061364746e-07, 0.015971719944801625, 0.0030898109607830496, 0.20051225318153115, 0.056078705990982046, 0.042535933066167916, 0.08894564611881633, 0.013720391443042091, 0.0893560214982335, 0.004854792909237545, 0.05051670578919593, 0.04398930028432042, 0.20019683349247303, 0.02431003628472982, 0.025820700043307235, 0.03125177357052729, 0.18767241679477015, 0.006935317553843128, 0.03553867140384327, 0.046910868886840214, 0.041038004716504414, 0.05566142861417524, 0.10175252781672592, 0.043039339684716495, 0.005575124316998947, 0.014327920305230179, 0.14141961641858297, 0.030428849503286094, 0.03097443478051214, 0.053297295509539835, 0.01319293308195767, 0.037575342088494834, 0.03252476860731002, 0.017428658227656123, 0.02878878317962888, 7.065066545343095e-05, 0.012318171273479872, 0.01957650838057024, 0.052138433485686884, 0.0009994974794457548, 0.1268556149681578, 0.15355180342007035, 0.007216132960147398, 0.026729006248870705, 0.003365976721099383, 0.0296576094666496, 0.017213533585808533, 0.02426871873077312, 0.02388325104074532, 0.020741824358975265, 0.030501955615137348, 0.004910091245963403, 0.20164480219247866, 0.08156363293270721, 0.0025512789050975873, 0.005365013080816118, 0.0051060217655788795, 0.15492019652881422, 0.029960893057235906, 0.031920895018325644, 0.0027471878855784386, 0.04989716614810648, 0.008590227948700867, 0.04011362669699745, 0.007138228612640216, 0.015711220460328752, 0.03446820182449697, 0.09837448417794106, 0.029690096328545683, 0.015862644314314646, 0.0006141053557446362, 0.17538809535438518, 0.030481849176645783, 0.04010178611080003, 0.03838911909476024, 0.039434227942281797, 0.030413002888288007, 0.01646734171105202, 0.010969413184563953, 0.014292054548651812, 0.010659124455052668]
第8次循环, 总体误差:4.501927770761087
每个样本误差: [0.0013114142987468306, 0.016149953710192736, 0.05057941948383527, 0.015027770686962738, 0.1680480809114139, 0.008369860864801163, 0.00447720813110411, 0.014347657296975796, 0.02858647976223599, 0.19088286053887882, 0.2123771167329272, 0.04086134028122645, 0.0029288200324464886, 0.10200390666240929, 0.0002603078515135808, 0.029488769874384927, 0.0012945242563314585, 0.022594978316282776, 1.059492904946271e-06, 0.01574510801101386, 0.0030797019997853945, 0.20068104224123332, 0.05670873614506661, 0.04267781182354075, 0.08926524683965914, 0.013839995367979897, 0.088793933384569, 0.004944260437752274, 0.05101177331632998, 0.044112567727550456, 0.20012102933033998, 0.02450640313549718, 0.025706002706820066, 0.03123494955250261, 0.1881159603532619, 0.0069211177705223075, 0.03518718748138915, 0.04702274093179794, 0.041032460719649305, 0.05582115981688513, 0.10097575004057964, 0.04284511012364612, 0.005484833132017415, 0.014362418080179129, 0.14083603110541928, 0.030495904579465993, 0.03086906083800984, 0.053562269401922685, 0.013004058827293804, 0.03762546495103309, 0.03249653141393947, 0.017204678107319286, 0.02901150098652604, 7.564636564244785e-05, 0.012251192863786719, 0.019480919330383966, 0.052578671909193485, 0.000989912619705864, 0.12688171747589855, 0.1538467847653139, 0.007082534203182374, 0.026627129801592583, 0.0033794837748039037, 0.02978292172511706, 0.017268104420798555, 0.024495808003313947, 0.023926728061202336, 0.02073036148293005, 0.030594816024573313, 0.0048784402303073465, 0.20129898585991673, 0.0819992128532776, 0.00256687126266354, 0.005447531130792593, 0.0052399843996283725, 0.15491521811564474, 0.029929582100023056, 0.03170582503699299, 0.0027343080639013104, 0.049581622666546935, 0.00850243546303479, 0.040077140391033626, 0.007228481421535626, 0.01557509583919555, 0.034408055322642794, 0.09876571651129276, 0.029528907601203348, 0.016074673181954997, 0.0006318400241298109, 0.1751043316871342, 0.030264015352071003, 0.04018229052163691, 0.038469535787376614, 0.039596248059509016, 0.030323198611404575, 0.0163498926505776, 0.010874122411527135, 0.014137553795040479, 0.010649591715579906]
第9次循环, 总体误差:4.502226152227579
每个样本误差: [0.001317231067850271, 0.016097144054108482, 0.0508413538749347, 0.014930476818848865, 0.16854766104657656, 0.00827655257132664, 0.004458713497264518, 0.014299303623927758, 0.028663372678876628, 0.19112327502823778, 0.21179135634069013, 0.040992449235648334, 0.002983291328664909, 0.10153204091072036, 0.0002711789977922791, 0.029348103059564243, 0.0012851071217576694, 0.022793126280563174, 5.620598456722201e-06, 0.015523179215543425, 0.0030692047047643687, 0.2008546152492946, 0.05733493506496241, 0.04282022942430053, 0.08958095319930079, 0.013958757674356951, 0.08823563588353799, 0.005033539378533779, 0.051503329131960526, 0.04423318184614184, 0.20003844718009733, 0.0247045338906571, 0.025598084112693247, 0.03122188472178246, 0.18855033325514775, 0.006910934913182355, 0.03485033419744961, 0.04713350177928889, 0.041025673336913836, 0.05597897131696737, 0.10021113720899681, 0.04265668336858872, 0.005394837347377341, 0.014398179061585501, 0.140263556241528, 0.030559527008321936, 0.030763960744903965, 0.05381878494420242, 0.012816920662379975, 0.03767275203473241, 0.032465040925560955, 0.0169808210990911, 0.029231099915583897, 8.063402021866931e-05, 0.01218232833227656, 0.019383283395793656, 0.053026370770935374, 0.0009795186490412023, 0.1269134154340862, 0.1541454671363221, 0.0069514617946905555, 0.026524947042412123, 0.003392467837067336, 0.029906619986157637, 0.01732607742659765, 0.024722823839743658, 0.023968056552655905, 0.020721845221073874, 0.030685688707742086, 0.004848068378161524, 0.20095908220669773, 0.08243176556396757, 0.002581984774379569, 0.005530111896541403, 0.005374286335543998, 0.15490543139491386, 0.029903643536279768, 0.031497366878392836, 0.0027217793591948762, 0.04927061772376989, 0.008416090197014945, 0.0400404834642265, 0.007319843061622256, 0.015440464364690897, 0.03434790626544063, 0.09915958138932274, 0.029370193912026693, 0.01628742597820966, 0.0006498729463292356, 0.1748255685337651, 0.030054890239840263, 0.04026113080289452, 0.038547542381366935, 0.03975343952438599, 0.030236181073711337, 0.01623164707335592, 0.010779271413321228, 0.013981702655116505, 0.010636854580740627]

以上内容都是为了完成机器学习课程作业

#深度学习 #机器学习

神经网络BP算法求导-手动推导

手动推导

概念抽象

代码实现

统计咨询

赞助

神经网络BP算法求导-手动推导

手动推导

概念抽象

代码实现

统计咨询

赞助

赞助推荐

常用工具

深度学习

机器学习

友商赞助