<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>最小二乘法 &#8211; 编码无悔 /  Intent &amp; Focused</title>
	<atom:link href="https://www.codelast.com/tag/%E6%9C%80%E5%B0%8F%E4%BA%8C%E4%B9%98%E6%B3%95/feed/" rel="self" type="application/rss+xml" />
	<link>https://www.codelast.com</link>
	<description>最优化之路</description>
	<lastBuildDate>Sun, 03 May 2020 13:01:23 +0000</lastBuildDate>
	<language>zh-Hans</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.2</generator>
	<item>
		<title>[原创]最小二乘的理论依据</title>
		<link>https://www.codelast.com/%e5%8e%9f%e5%88%9b%e6%9c%80%e5%b0%8f%e4%ba%8c%e4%b9%98%e7%9a%84%e7%90%86%e8%ae%ba%e4%be%9d%e6%8d%ae/</link>
					<comments>https://www.codelast.com/%e5%8e%9f%e5%88%9b%e6%9c%80%e5%b0%8f%e4%ba%8c%e4%b9%98%e7%9a%84%e7%90%86%e8%ae%ba%e4%be%9d%e6%8d%ae/#comments</comments>
		
		<dc:creator><![CDATA[learnhard]]></dc:creator>
		<pubDate>Fri, 08 Oct 2010 14:44:32 +0000</pubDate>
				<category><![CDATA[Algorithm]]></category>
		<category><![CDATA[Math]]></category>
		<category><![CDATA[原创]]></category>
		<category><![CDATA[least square]]></category>
		<category><![CDATA[optimization]]></category>
		<category><![CDATA[最优化]]></category>
		<category><![CDATA[最小二乘法]]></category>
		<guid isPermaLink="false">http://www.codelast.com/?p=1027</guid>

					<description><![CDATA[<p class="MsoNormal">
	<span style="font-family: 微软雅黑;"><span style="font-size: 12pt; ">在做数据建模或者曲线拟合的时候，我们通常会用到最小二乘法。</span></span><br />
<span id="more-1027"></span>	<br />
	<span style="font-family: 微软雅黑;"><span style="font-size: 12pt; ">假设作为数学模型的函数为</span></span> <span class='MathJax_Preview'>\(y = f(x,S)\)</span><script type='math/tex'>y = f(x,S)</script> <span style="font-family: 微软雅黑; font-size: 12pt;">，其中 <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family: 微软雅黑; font-size: 12pt;">为参数集向量（即一系列的参数），</span><span lang="EN-US" style="font-family: 微软雅黑; font-size: 12pt;"> <span class='MathJax_Preview'>\(x\)</span><script type='math/tex'>x</script> </span><span style="font-family: 微软雅黑; font-size: 12pt;">为自变量。在这种情况下，为了求出</span><span style="font-family: 微软雅黑; font-size: 16px;"> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family: 微软雅黑; font-size: 12pt;">，需要对下式进行极小化：</span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img decoding="async" alt="" height="68" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_7.png" width="159" /></span></p>
<p class="MsoNormal">
	<span style="font-family:微软雅黑;"><span lang="EN-US" style="font-size:12.0pt">&#160;&#160;&#160;&#160;&#160;&#160; </span><span style="font-size: 12pt; ">即：对已知的一个数据集</span></span> <span class='MathJax_Preview'>\({x_i}(i = 1,2, \cdots ,n)\)</span><script type='math/tex'>{x_i}(i = 1,2, \cdots ,n)</script> <span style="font-family: 微软雅黑; font-size: 12pt;">，能极小化该式的 <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family: 微软雅黑; font-size: 12pt;">就是最优参数。但是这个式子是怎么来的呢？</span><br />
	<!--more--></p>
<p class="MsoNormal" style="text-indent:21.0pt">
	<span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">它是从最大似然估计方法得到的：对参数</span></span><span style="font-family: 微软雅黑; font-size: 16px;"> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">，能使已知数据集发生的概率越大，那么就说明我们取的</span></span><span style="font-family: 微软雅黑; font-size: 16px;"> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">越优良。注意，对于一组已知的数据集，参数</span></span><span style="font-family: 微软雅黑; font-size: 16px;"> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">几乎不可能使每个</span></span> <span class='MathJax_Preview'>\({x_i}\)</span><script type='math/tex'>{x_i}</script> <span style="font-family: 微软雅黑; text-indent: 21pt; font-size: 12pt;">都满足我们假设的数学模型，因此这里所说的&#8220;使已知数据集发生的概率越大&#8221;，这个&#8220;发生&#8221;，是指</span> <span class='MathJax_Preview'>\({y_i} \in \left[ {f({x_i},S) - \delta ,f({x_i},S) + \delta } \right]\)</span><script type='math/tex'>{y_i} \in \left[ {f({x_i},S) - \delta ,f({x_i},S) + \delta } \right]</script> <span style="text-indent: 21pt; font-family: 微软雅黑; font-size: 12pt;">，其中</span><span style="text-indent: 21pt; font-family: 微软雅黑;">&#948;</span><span style="text-indent: 21pt; font-family: 微软雅黑; font-size: 12pt;">为允许的误差。</span></p>
<p class="MsoNormal" style="text-indent:21.0pt">
	<span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">假设所有数据点的测量误差独立、符合正态分布，且标准差相等，则每一个数据点发生的概率为：</span></span><span lang="EN-US" style="font-size:12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img decoding="async" alt="" height="96" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_11.png" width="230" /></span></p>
<p class="MsoNormal" style="text-indent:21.0pt">
	<span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">整个数据集同时发生的概率为各数据点概率之积：</span></span><span lang="EN-US" style="font-size:12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img decoding="async" alt="" height="107" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_12.png" width="267" /></span></p>
<p class="MsoNormal">
	<span style="color: rgb(255, 255, 255);">文章来源：</span><a href="http://www.codelast.com/" rel="noopener noreferrer" target="_blank"><span style="color: rgb(255, 255, 255);">http://www.codelast.com/</span></a>&#8230; <a href="https://www.codelast.com/%e5%8e%9f%e5%88%9b%e6%9c%80%e5%b0%8f%e4%ba%8c%e4%b9%98%e7%9a%84%e7%90%86%e8%ae%ba%e4%be%9d%e6%8d%ae/" class="read-more">Read More </a></p>]]></description>
										<content:encoded><![CDATA[<p class="MsoNormal">
	<span style="font-family: 微软雅黑;"><span style="font-size: 12pt; ">在做数据建模或者曲线拟合的时候，我们通常会用到最小二乘法。</span></span><br />
<span id="more-1027"></span>	<br />
	<span style="font-family: 微软雅黑;"><span style="font-size: 12pt; ">假设作为数学模型的函数为</span></span> <span class='MathJax_Preview'>\(y = f(x,S)\)</span><script type='math/tex'>y = f(x,S)</script> <span style="font-family: 微软雅黑; font-size: 12pt;">，其中 <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family: 微软雅黑; font-size: 12pt;">为参数集向量（即一系列的参数），</span><span lang="EN-US" style="font-family: 微软雅黑; font-size: 12pt;"> <span class='MathJax_Preview'>\(x\)</span><script type='math/tex'>x</script> </span><span style="font-family: 微软雅黑; font-size: 12pt;">为自变量。在这种情况下，为了求出</span><span style="font-family: 微软雅黑; font-size: 16px;"> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family: 微软雅黑; font-size: 12pt;">，需要对下式进行极小化：</span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img loading="lazy" decoding="async" alt="" height="68" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_7.png" width="159" /></span></p>
<p class="MsoNormal">
	<span style="font-family:微软雅黑;"><span lang="EN-US" style="font-size:12.0pt">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span style="font-size: 12pt; ">即：对已知的一个数据集</span></span> <span class='MathJax_Preview'>\({x_i}(i = 1,2, \cdots ,n)\)</span><script type='math/tex'>{x_i}(i = 1,2, \cdots ,n)</script> <span style="font-family: 微软雅黑; font-size: 12pt;">，能极小化该式的 <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family: 微软雅黑; font-size: 12pt;">就是最优参数。但是这个式子是怎么来的呢？</span><br />
	<!--more--></p>
<p class="MsoNormal" style="text-indent:21.0pt">
	<span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">它是从最大似然估计方法得到的：对参数</span></span><span style="font-family: 微软雅黑; font-size: 16px;"> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">，能使已知数据集发生的概率越大，那么就说明我们取的</span></span><span style="font-family: 微软雅黑; font-size: 16px;"> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">越优良。注意，对于一组已知的数据集，参数</span></span><span style="font-family: 微软雅黑; font-size: 16px;"> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> </span><span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">几乎不可能使每个</span></span> <span class='MathJax_Preview'>\({x_i}\)</span><script type='math/tex'>{x_i}</script> <span style="font-family: 微软雅黑; text-indent: 21pt; font-size: 12pt;">都满足我们假设的数学模型，因此这里所说的&ldquo;使已知数据集发生的概率越大&rdquo;，这个&ldquo;发生&rdquo;，是指</span> <span class='MathJax_Preview'>\({y_i} \in \left[ {f({x_i},S) - \delta ,f({x_i},S) + \delta } \right]\)</span><script type='math/tex'>{y_i} \in \left[ {f({x_i},S) - \delta ,f({x_i},S) + \delta } \right]</script> <span style="text-indent: 21pt; font-family: 微软雅黑; font-size: 12pt;">，其中</span><span style="text-indent: 21pt; font-family: 微软雅黑;">&delta;</span><span style="text-indent: 21pt; font-family: 微软雅黑; font-size: 12pt;">为允许的误差。</span></p>
<p class="MsoNormal" style="text-indent:21.0pt">
	<span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">假设所有数据点的测量误差独立、符合正态分布，且标准差相等，则每一个数据点发生的概率为：</span></span><span lang="EN-US" style="font-size:12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img loading="lazy" decoding="async" alt="" height="96" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_11.png" width="230" /></span></p>
<p class="MsoNormal" style="text-indent:21.0pt">
	<span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">整个数据集同时发生的概率为各数据点概率之积：</span></span><span lang="EN-US" style="font-size:12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img loading="lazy" decoding="async" alt="" height="107" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_12.png" width="267" /></span></p>
<p class="MsoNormal">
	<span style="color: rgb(255, 255, 255);">文章来源：</span><a href="http://www.codelast.com/" rel="noopener noreferrer" target="_blank"><span style="color: rgb(255, 255, 255);">http://www.codelast.com/</span></a></p>
<p class="MsoNormal">
	<span style="font-family:微软雅黑;"><span lang="EN-US" style="font-size:12.0pt">&nbsp;&nbsp; &nbsp;&nbsp;</span><span style="font-size: 12pt; ">如前文所述：对参数</span></span> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> <span style="font-family: 微软雅黑;"><span style="font-size: 12pt; ">，能使已知数据集发生的概率越大，那么就说明我们取的</span></span> <span class='MathJax_Preview'>\(S\)</span><script type='math/tex'>S</script> <span style="font-family: 微软雅黑;"><span style="font-size: 12pt; ">越优良。因此，使上式最大化就是我们的目标。由于</span></span> <span class='MathJax_Preview'>\(\delta \)</span><script type='math/tex'>\delta </script> <span style="font-family: 微软雅黑;"><span style="font-size: 12pt; ">为正常数，</span></span> <span class='MathJax_Preview'>\(f(x) = {e^x}\)</span><script type='math/tex'>f(x) = {e^x}</script> <span style="font-size: 12pt; font-family: 微软雅黑;">为单调递增函数，因此，想要：</span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img decoding="async" alt="" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_14.png" /></span></p>
<p class="MsoNormal" style="text-indent:21.0pt">
	<span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">就等于：</span></span><span lang="EN-US" style="font-size:12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img loading="lazy" decoding="async" alt="" height="141" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_2.png" width="281" /></span></p>
<p class="MsoNormal">
	<span style="color: rgb(255, 255, 255);">文章来源：</span><a href="http://www.codelast.com/" rel="noopener noreferrer" target="_blank"><span style="color: rgb(255, 255, 255);">http://www.codelast.com/</span></a></p>
<p class="MsoNormal">
	<span style="font-family:微软雅黑;"><span lang="EN-US" style="font-size:12.0pt">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span style="font-size: 12pt; ">等同于：</span></span><span lang="EN-US" style="font-size:12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img decoding="async" alt="" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_3.png" /></span></p>
<p align="left" class="MsoNormal" style="text-align:left">
	<span style="font-family:微软雅黑;"><span lang="EN-US" style="font-size:12.0pt">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span style="font-size: 12pt; ">继续化简：</span></span><span lang="EN-US" style="font-size:
12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img decoding="async" alt="" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_4.png" /></span></p>
<p align="left" class="MsoNormal" style="text-align:left">
	<span style="font-family:微软雅黑;"><span lang="EN-US" style="font-size:12.0pt">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span style="font-size: 12pt; ">相当于：</span></span><span lang="EN-US" style="font-size:
12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img decoding="async" alt="" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_5.png" /></span></p>
<p class="MsoNormal">
	<span style="color: rgb(255, 255, 255);">文章来源：</span><a href="http://www.codelast.com/" rel="noopener noreferrer" target="_blank"><span style="color: rgb(255, 255, 255);">http://www.codelast.com/</span></a></p>
<p align="left" class="MsoNormal" style="text-align:left">
	<span style="font-family:微软雅黑;"><span lang="EN-US" style="font-size:12.0pt">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span style="font-size: 12pt; ">现在，由于</span></span> <span class='MathJax_Preview'>\(\sigma \)</span><script type='math/tex'>\sigma </script> <span style="font-family:微软雅黑;"><span style="font-size: 12pt; ">是常数，上式就等同于：</span></span><span lang="EN-US" style="font-size:12.0pt"><o:p></o:p></span></p>
<p align="center" class="MsoNormal" style="text-align:center">
	<span style="font-family:微软雅黑;"><img decoding="async" alt="" src="http://www.codelast.com/wp-content/uploads/2011/01/least_square_tb_6.png" /></span></p>
<p align="left" class="MsoNormal" style="text-align:left">
	<span style="font-family:微软雅黑;"><span lang="EN-US" style="font-size:12.0pt">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span style="font-size: 12pt; ">这就得到了我们要推导的结论。</span></span></p>
<p>	<span style="color: rgb(255, 255, 255);">文章来源：</span><a href="https://www.codelast.com/" rel="noopener noreferrer" target="_blank"><span style="color: rgb(255, 255, 255);">https://www.codelast.com/</span></a><br />
	<span style="color: rgb(255, 0, 0);">➤➤</span>&nbsp;版权声明&nbsp;<span style="color: rgb(255, 0, 0);">➤➤</span>&nbsp;<br />
	转载需注明出处：<u><a href="https://www.codelast.com/" rel="noopener noreferrer" target="_blank"><em><span style="color: rgb(0, 0, 255);"><strong style="font-size: 16px;"><span style="font-family: arial, helvetica, sans-serif;">codelast.com</span></strong></span></em></a></u>&nbsp;<br />
	感谢关注我的微信公众号（微信扫一扫）：</p>
<p style="border: 0px; font-size: 13px; margin: 0px 0px 9px; outline: 0px; padding: 0px; color: rgb(77, 77, 77);">
	<img decoding="async" alt="wechat qrcode of codelast" src="https://www.codelast.com/codelast_wechat_qr_code.jpg" style="width: 200px; height: 200px;" /></p>
<p align="left" class="MsoNormal" style="text-align:left">
	<span style="font-family:微软雅黑;"><span style="color:#fff;">NULL</span></span></p>
<p align="left" class="MsoNormal" style="text-align:left">
	<span lang="EN-US" style="font-size:12.0pt"><o:p></o:p></span></p>
]]></content:encoded>
					
					<wfw:commentRss>https://www.codelast.com/%e5%8e%9f%e5%88%9b%e6%9c%80%e5%b0%8f%e4%ba%8c%e4%b9%98%e7%9a%84%e7%90%86%e8%ae%ba%e4%be%9d%e6%8d%ae/feed/</wfw:commentRss>
			<slash:comments>1</slash:comments>
		
		
			</item>
	</channel>
</rss>
