DataNitro教程系列(python控制Excel自动化替代VBA)
2019年01月06日
这个系列的教程好像是在2014年录制的, 距今已经大概4年多过去, 当时教程放在百度传课上售卖, 不过百度传课倒闭了,
所以我把这个系列教程重新上传到我自己的网站来, 希望对一些人有用吧!
瑞文标准推理测验在线换算标准分,怎样计算百分等级
2019年01月04日
stata教程07-常见样本量估计
2018年12月13日
我们常用PASS做样本量的计算, 不过本人更熟悉Stata, 所以我把常用的样本量计算情景都用stata实现了一遍, 希望对大家有用, 对自己也是一个备忘录。
stata教程06-计算AIC和BIC决定变量个数
2018年12月12日
在回归分析中, 解释变量的个数应当是越少越好, 因为模型越简洁应用范围越广, 也越容易被理解。但是, 为了模型拟合好, 我们又期望模型越复杂越好, 解释变量越多, 模型的拟合优度$R^2$
数据介绍
使用上一篇教程用过的数据: icecream.dta, 具体内容可以参看上篇文章: stata教程05-自相关的检验和处理/
1 | use data/icecream.dta, clear |
进行回归分析
1 | reg consumption temp price income |
输出(stream):
Source | SS df MS Number of obs = 30 -------------+---------------------------------- F(3, 26) = 22.17 Model | .090250523 3 .030083508 Prob > F = 0.0000 Residual | .035272835 26 .001356647 R-squared = 0.7190 -------------+---------------------------------- Adj R-squared = 0.6866 Total | .125523358 29 .004328392 Root MSE = .03683 ------------------------------------------------------------------------------ consumption | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- temp | .0034584 .0004455 7.76 0.000 .0025426 .0043743 price | -1.044413 .834357 -1.25 0.222 -2.759458 .6706322 income | .0033078 .0011714 2.82 0.009 .0008999 .0057156 _cons | .1973149 .2702161 0.73 0.472 -.3581223 .752752 ------------------------------------------------------------------------------
Source | SS df MS Number of obs = 30 -------------+---------------------------------- F(3, 26) = 22.17 Model | .090250523 3 .030083508 Prob > F = 0.0000 Residual | .035272835 26 .001356647 R-squared = 0.7190 -------------+---------------------------------- Adj R-squared = 0.6866 Total | .125523358 29 .004328392 Root MSE = .03683 ------------------------------------------------------------------------------ consumption | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- temp | .0034584 .0004455 7.76 0.000 .0025426 .0043743 price | -1.044413 .834357 -1.25 0.222 -2.759458 .6706322 income | .0033078 .0011714 2.82 0.009 .0008999 .0057156 _cons | .1973149 .2702161 0.73 0.472 -.3581223 .752752 ------------------------------------------------------------------------------
计算AIC&BIC
1 | estat ic |
输出(stream):
Akaike's information criterion and Bayesian information criterion ----------------------------------------------------------------------------- Model | Obs ll(null) ll(model) df AIC BIC -------------+--------------------------------------------------------------- . | 30 39.57876 58.61944 4 -109.2389 -103.6341 ----------------------------------------------------------------------------- Note: N=Obs used in calculating BIC; see [R] BIC note.
Akaike's information criterion and Bayesian information criterion ----------------------------------------------------------------------------- Model | Obs ll(null) ll(model) df AIC BIC -------------+--------------------------------------------------------------- . | 30 39.57876 58.61944 4 -109.2389 -103.6341 ----------------------------------------------------------------------------- Note: N=Obs used in calculating BIC; see [R] BIC note.
加入temp的1阶滞后项
1 | reg consumption temp L.temp price income |
输出(stream):
Source | SS df MS Number of obs = 29 -------------+---------------------------------- F(4, 24) = 28.98 Model | .103387183 4 .025846796 Prob > F = 0.0000 Residual | .021406049 24 .000891919 R-squared = 0.8285 -------------+---------------------------------- Adj R-squared = 0.7999 Total | .124793232 28 .004456901 Root MSE = .02987 ------------------------------------------------------------------------------ consumption | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- temp | --. | .0053321 .0006704 7.95 0.000 .0039484 .0067158 L1. | -.0022039 .0007307 -3.02 0.006 -.0037119 -.0006959 | price | -.8383021 .6880205 -1.22 0.235 -2.258307 .5817025 income | .0028673 .0010533 2.72 0.012 .0006934 .0050413 _cons | .1894822 .2323169 0.82 0.423 -.2899963 .6689607 ------------------------------------------------------------------------------
Source | SS df MS Number of obs = 29 -------------+---------------------------------- F(4, 24) = 28.98 Model | .103387183 4 .025846796 Prob > F = 0.0000 Residual | .021406049 24 .000891919 R-squared = 0.8285 -------------+---------------------------------- Adj R-squared = 0.7999 Total | .124793232 28 .004456901 Root MSE = .02987 ------------------------------------------------------------------------------ consumption | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- temp | --. | .0053321 .0006704 7.95 0.000 .0039484 .0067158 L1. | -.0022039 .0007307 -3.02 0.006 -.0037119 -.0006959 | price | -.8383021 .6880205 -1.22 0.235 -2.258307 .5817025 income | .0028673 .0010533 2.72 0.012 .0006934 .0050413 _cons | .1894822 .2323169 0.82 0.423 -.2899963 .6689607 ------------------------------------------------------------------------------
再计算AIC&BIC
1 | estat ic |
输出(stream):
Akaike's information criterion and Bayesian information criterion ----------------------------------------------------------------------------- Model | Obs ll(null) ll(model) df AIC BIC -------------+--------------------------------------------------------------- . | 29 37.85248 63.41576 5 -116.8315 -109.995 ----------------------------------------------------------------------------- Note: N=Obs used in calculating BIC; see [R] BIC note.
Akaike's information criterion and Bayesian information criterion ----------------------------------------------------------------------------- Model | Obs ll(null) ll(model) df AIC BIC -------------+--------------------------------------------------------------- . | 29 37.85248 63.41576 5 -116.8315 -109.995 ----------------------------------------------------------------------------- Note: N=Obs used in calculating BIC; see [R] BIC note.
我们可以看到AIC和BIC都下降了。
再加入temp的2阶滞后项
1 | reg consumption temp L.temp L2.temp price income |
输出(stream):
Source | SS df MS Number of obs = 28 -------------+---------------------------------- F(5, 22) = 21.92 Model | .103722201 5 .02074444 Prob > F = 0.0000 Residual | .020822754 22 .000946489 R-squared = 0.8328 -------------+---------------------------------- Adj R-squared = 0.7948 Total | .124544954 27 .004612776 Root MSE = .03077 ------------------------------------------------------------------------------ consumption | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- temp | --. | .0047858 .0013502 3.54 0.002 .0019856 .007586 L1. | -.0010836 .0022905 -0.47 0.641 -.0058338 .0036666 L2. | -.0008022 .0013414 -0.60 0.556 -.0035841 .0019797 | price | -.7326035 .7214324 -1.02 0.321 -2.228763 .7635558 income | .0026704 .0011308 2.36 0.027 .0003252 .0050156 _cons | .1883478 .23949 0.79 0.440 -.3083241 .6850196 ------------------------------------------------------------------------------
Source | SS df MS Number of obs = 28 -------------+---------------------------------- F(5, 22) = 21.92 Model | .103722201 5 .02074444 Prob > F = 0.0000 Residual | .020822754 22 .000946489 R-squared = 0.8328 -------------+---------------------------------- Adj R-squared = 0.7948 Total | .124544954 27 .004612776 Root MSE = .03077 ------------------------------------------------------------------------------ consumption | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- temp | --. | .0047858 .0013502 3.54 0.002 .0019856 .007586 L1. | -.0010836 .0022905 -0.47 0.641 -.0058338 .0036666 L2. | -.0008022 .0013414 -0.60 0.556 -.0035841 .0019797 | price | -.7326035 .7214324 -1.02 0.321 -2.228763 .7635558 income | .0026704 .0011308 2.36 0.027 .0003252 .0050156 _cons | .1883478 .23949 0.79 0.440 -.3083241 .6850196 ------------------------------------------------------------------------------
再计算AIC&BIC
1 | estat ic |
输出(stream):
Akaike's information criterion and Bayesian information criterion ----------------------------------------------------------------------------- Model | Obs ll(null) ll(model) df AIC BIC -------------+--------------------------------------------------------------- . | 28 36.08382 61.12451 6 -110.249 -102.2558 ----------------------------------------------------------------------------- Note: N=Obs used in calculating BIC; see [R] BIC note.
Akaike's information criterion and Bayesian information criterion ----------------------------------------------------------------------------- Model | Obs ll(null) ll(model) df AIC BIC -------------+--------------------------------------------------------------- . | 28 36.08382 61.12451 6 -110.249 -102.2558 ----------------------------------------------------------------------------- Note: N=Obs used in calculating BIC; see [R] BIC note.
我们可以看到, 在增加了L2项滞后, 我们的AIC和BIC反而上升, 说明增加二阶滞后项导致模型复杂度上升, 但并没有带来模型的拟合优度较大的上升, 也就是得不偿失。
总结
AIC和BIC是两个常用的用户评估模型复杂性的指标, 但是他们略有不同, BIC是一致估计, 而AIC不是, 但现实样本不可能无限大, 而BIC可能导致模型过小, 所以我们通常是综合考虑两个指标。
注意
本文由jupyter notebook转换而来, 您可以在这里下载notebook
有问题可以直接在下方留言
或者给我发邮件675495787[at]qq.com
请记住我的网址: mlln.cn 或者 jupyter.cn
stata教程03-异方差的检验和处理
2018年12月11日
古典线性回归的假设是一种理想状态, 现实数据很难满足, 比如异方差就是一个常见的违背模型假设的情况。下面我们介绍一下如何检验数据是否存在异方差以及出现异方差的情况后如何处理。