首页 > 其他分享> > 浅谈范数正则化

浅谈范数正则化

2021-10-13 09:32:06 作者：互联网

<div id="page_begin_html">
    <style>

back-top {

 position: fixed;
 bottom: 5px;
 right: 10px;
 z-index: 99;

}

back-top span {

 width: 50px;
 height: 64px;
 display: block;
 background:url(http://images.cnblogs.com/cnblogs_com/shwee/1218109/o_toTop.bmp) no-repeat center center;

}

back-top a{outline:none}

凯鲁嘎吉
用书写铭记日常，最迷人的不在远方

posts - 245,comments - 244,views - 62万

</div>

浅谈范数正则化

</div>
<div id="cnblogs_post_body" class="blogpost-body blogpost-body-html"><a rel="nofollow" name="_labelTop"></a><div id="navCategory"><p style="font-size:18px"><b>阅读目录(Content)</b></p><ul class="first_class_ul"><li><a rel="nofollow" href="#_label0">浅谈范数正则化</a></li><ul class="second_class_ul"><li><a rel="nofollow" href="#_lab2_0_0">1. 向量范数与矩阵范数</a></li><li><a rel="nofollow" href="#_lab2_0_1">2. 为什么要添加正则项？</a></li><li><a rel="nofollow" href="#_lab2_0_2">3. <span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-1-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mn>0</mn></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-1" style="width: 1.47em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.133em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.13em, 2.699em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-2"><span class="msubsup" id="MathJax-Span-3"><span style="display: inline-block; position: relative; width: 1.11em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-4" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="mn" id="MathJax-Span-5" style="font-size: 70.7%; font-family: MathJax_Main;">0</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.29em; border-left: 0px solid; width: 0px; height: 1.227em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mn>0</mn></msub></math></span></span><script type="math/tex" id="MathJax-Element-1">L_0</script>范数</a></li><li><a rel="nofollow" href="#_lab2_0_3">4. <span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-2-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mn>1</mn></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-6" style="width: 1.47em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.133em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.13em, 2.683em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-7"><span class="msubsup" id="MathJax-Span-8"><span style="display: inline-block; position: relative; width: 1.11em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-9" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="mn" id="MathJax-Span-10" style="font-size: 70.7%; font-family: MathJax_Main;">1</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.271em; border-left: 0px solid; width: 0px; height: 1.208em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mn>1</mn></msub></math></span></span><script type="math/tex" id="MathJax-Element-2">L_1</script>范数</a></li><li><a rel="nofollow" href="#_lab2_0_4">5. <span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-3-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mn>2</mn></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-11" style="width: 1.47em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.133em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.13em, 2.683em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-12"><span class="msubsup" id="MathJax-Span-13"><span style="display: inline-block; position: relative; width: 1.11em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-14" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="mn" id="MathJax-Span-15" style="font-size: 70.7%; font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.271em; border-left: 0px solid; width: 0px; height: 1.208em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mn>2</mn></msub></math></span></span><script type="math/tex" id="MathJax-Element-3">L_2</script>范数</a></li><li><a rel="nofollow" href="#_lab2_0_5">6. <span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-4-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mn>1</mn></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-16" style="width: 1.47em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.133em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.13em, 2.683em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-17"><span class="msubsup" id="MathJax-Span-18"><span style="display: inline-block; position: relative; width: 1.11em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-19" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="mn" id="MathJax-Span-20" style="font-size: 70.7%; font-family: MathJax_Main;">1</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.271em; border-left: 0px solid; width: 0px; height: 1.208em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mn>1</mn></msub></math></span></span><script type="math/tex" id="MathJax-Element-4">L_1</script>范数与<span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-5-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mn>2</mn></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-21" style="width: 1.47em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.133em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.13em, 2.683em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-22"><span class="msubsup" id="MathJax-Span-23"><span style="display: inline-block; position: relative; width: 1.11em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-24" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="mn" id="MathJax-Span-25" style="font-size: 70.7%; font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.271em; border-left: 0px solid; width: 0px; height: 1.208em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mn>2</mn></msub></math></span></span><script type="math/tex" id="MathJax-Element-5">L_2</script>范数作为正则项的区别</a></li><li><a rel="nofollow" href="#_lab2_0_6">7. 用概率解释传统线性回归模型</a></li><li><a rel="nofollow" href="#_lab2_0_7">8. <span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-6-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mn>2</mn></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-26" style="width: 1.47em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.133em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.13em, 2.683em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-27"><span class="msubsup" id="MathJax-Span-28"><span style="display: inline-block; position: relative; width: 1.11em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-29" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="mn" id="MathJax-Span-30" style="font-size: 70.7%; font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.271em; border-left: 0px solid; width: 0px; height: 1.208em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mn>2</mn></msub></math></span></span><script type="math/tex" id="MathJax-Element-6">L_2</script>范等价于Gauss先验</a></li><li><a rel="nofollow" href="#_lab2_0_8">9. <span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-7-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mn>1</mn></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-31" style="width: 1.47em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.133em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.13em, 2.683em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-32"><span class="msubsup" id="MathJax-Span-33"><span style="display: inline-block; position: relative; width: 1.11em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-34" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="mn" id="MathJax-Span-35" style="font-size: 70.7%; font-family: MathJax_Main;">1</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.271em; border-left: 0px solid; width: 0px; height: 1.208em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mn>1</mn></msub></math></span></span><script type="math/tex" id="MathJax-Element-7">L_1</script>范数等价于Laplace先验</a></li><li><a rel="nofollow" href="#_lab2_0_9">10. 矩阵的<span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-8-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mn>2</mn><mo>,</mo><mn>1</mn></mrow></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-36" style="width: 2.137em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.667em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.67em, 2.821em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-37"><span class="msubsup" id="MathJax-Span-38"><span style="display: inline-block; position: relative; width: 1.66em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-39" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="texatom" id="MathJax-Span-40"><span class="mrow" id="MathJax-Span-41"><span class="mn" id="MathJax-Span-42" style="font-size: 70.7%; font-family: MathJax_Main;">2</span><span class="mo" id="MathJax-Span-43" style="font-size: 70.7%; font-family: MathJax_Main;">,</span><span class="mn" id="MathJax-Span-44" style="font-size: 70.7%; font-family: MathJax_Main;">1</span></span></span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.442em; border-left: 0px solid; width: 0px; height: 1.379em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mrow class="MJX-TeXAtom-ORD"><mn>2</mn><mo>,</mo><mn>1</mn></mrow></msub></math></span></span><script type="math/tex" id="MathJax-Element-8">L_{2, 1}</script>范数及<span class="MathJax_Preview" style="color: inherit;"></span><span class="MathJax" id="MathJax-Element-9-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><msub><mi>L</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msub></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-45" style="width: 2.137em; display: inline-block;"><span style="display: inline-block; position: relative; width: 1.667em; height: 0px; font-size: 125%;"><span style="position: absolute; clip: rect(1.45em, 1001.67em, 2.821em, -1000em); top: -2.333em; left: 0em;"><span class="mrow" id="MathJax-Span-46"><span class="msubsup" id="MathJax-Span-47"><span style="display: inline-block; position: relative; width: 1.634em; height: 0px;"><span style="position: absolute; clip: rect(3.117em, 1000.65em, 4.2em, -1000em); top: -4em; left: 0em;"><span class="mi" id="MathJax-Span-48" style="font-family: MathJax_Math; font-style: italic;">L</span><span style="display: inline-block; width: 0px; height: 4em;"></span></span><span style="position: absolute; top: -3.85em; left: 0.681em;"><span class="texatom" id="MathJax-Span-49"><span class="mrow" id="MathJax-Span-50"><span class="mi" id="MathJax-Span-51" style="font-size: 70.7%; font-family: MathJax_Math; font-style: italic;">p</span><span class="mo" id="MathJax-Span-52" style="font-size: 70.7%; font-family: MathJax_Main;">,</span><span class="mi" id="MathJax-Span-53" style="font-size: 70.7%; font-family: MathJax_Math; font-style: italic;">q<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.01em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 4em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 2.333em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -0.442em; border-left: 0px solid; width: 0px; height: 1.379em;"></span></span></nobr><span class="MJX_Assistive_MathML" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>L</mi><mrow class="MJX-TeXAtom-ORD"><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msub></math></span></span><script type="math/tex" id="MathJax-Element-9">L_{p, q}</script>范数</a></li><li><a rel="nofollow" href="#_lab2_0_10">11. 矩阵的核范数及Schatten范数</a></li><li><a rel="nofollow" href="#_lab2_0_11">12. MATLAB程序：Laplace分布与Gauss分布的概率密度函数图</a></li><li><a rel="nofollow" href="#_lab2_0_12">13. 参考文献</a></li></ul></ul></div>

回到顶部(go to top)

浅谈范数正则化

作者：凯鲁嘎吉 - 博客园 http://www.cnblogs.com/kailugaji/

这篇博客介绍不同范数作为正则化项时的作用。首先介绍了常见的向量范数与矩阵范数，然后说明添加正则化项的原因，之后介绍向量的 $L_{0}$ ， $L_{1}$ ， $L_{2}$ 范数及其作为正则化项的作用，对三者进行比较分析，并用贝叶斯观点解释传统线性模型与正则化项。随后，介绍矩阵的 $L_{2, 1}$ 范数及其推广形式 $L_{p, q}$ 范数，以及矩阵的核范数及其推广形式Schatten范数。最后，用MATLAB程序编写了Laplace分布与Gauss分布的概率密度函数图。有关矩阵范数优化求解问题可参考：一类涉及矩阵范数的优化问题 - 凯鲁嘎吉 - 博客园

1. 向量范数与矩阵范数

2. 为什么要添加正则项？

3. $L_{0}$ 范数

4. $L_{1}$ 范数

5. $L_{2}$ 范数

6. $L_{1}$ 范数与 $L_{2}$ 范数作为正则项的区别

7. 用概率解释传统线性回归模型

8. $L_{2}$ 范等价于Gauss先验

9. $L_{1}$ 范数等价于Laplace先验

10. 矩阵的 $L_{2, 1}$ 范数及 $L_{p, q}$ 范数

11. 矩阵的核范数及Schatten范数

12. MATLAB程序：Laplace分布与Gauss分布的概率密度函数图

12345678910111213141516171819202122232425 %% Demo of Laplace Density Function% x : variable% lambda : size para%miu: location paraclearclcx = -10:0.1:10;y_1=Laplace_distribution(x, 0, 1);y_2=Laplace_distribution(x, 0, 2);y_3=Laplace_distribution(x, 0, 4);y_4=Laplace_distribution(x, -5, 4);y_5=Laplace_distribution(x, 5, 4);y_6=normpdf(x,0,1);plot(x, y_1, 'r-', x, y_2, 'g-', x, y_3, 'c-', x, y_4, 'm-', x, y_5, 'y-', x, y_6, 'b-', 'LineWidth',1.2);legend('\mu =0, \lambda=1','\mu=0, \lambda=2','\mu=0, \lambda=4','\mu=-5, \lambda=4','\mu=5, \lambda=4', '\mu=0, \sigma=1'); %图例的设置xlabel('x');ylabel('f(x)');title('Laplace vs Gauss pdf');set(gca, 'FontName', 'Times New Roman', 'FontSize',11);saveas(gcf,sprintf('demo_Laplace_Gauss.jpg'),'bmp'); %保存图片 %% Laplace Density Functionfunction y=Laplace_distribution(x, miu, lambda) y = 1 / (2*lambda) * exp( -abs(x-miu)/lambda);end

13. 参考文献

[1] 证明核范数是矩阵秩的凸包络

EJ Candès, Recht B . Exact Matrix Completion via Convex Optimization[J]. Foundations of Computational Mathematics, 2009, 9(6):717.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.312.1183&rep=rep1&type=pdf

[2] 关于说明 $L_{1}$ 范数是 $L_{0}$ 范数的凸包络的文献及教案

Donoho D L , Huo X . Uncertainty Principles and Ideal Atomic Decomposition[J]. IEEE Transactions on Information Theory, 2001, 47(7):2845-2862.

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=00BC0C50CDECB265657379792F917FFE?doi=10.1.1.161.9300&rep=rep1&type=pdf

Learning with Combinatorial Structure Note for Lecture 12

http://people.csail.mit.edu/stefje/fall15/notes_lecture12.pdf

L1-norm Methods for Convex-Cardinality Problems

https://web.stanford.edu/class/ee364b/lectures/l1_slides.pdf

[3] 有关过拟合的教案及图片来源

2017 Lecture 2: Overfitting. Regularization

https://www.cs.mcgill.ca/~dprecup/courses/ML/Lectures/ml-lecture02.pdf

[4] 一些可供参考的资料

The difference between L1 and L2 regularization

https://explained.ai/regularization/L1vsL2.html

Why L1 norm for sparse models

https://stats.stackexchange.com/questions/45643/why-l1-norm-for-sparse-models

Why L1 regularization can “zero out the weights” and therefore leads to sparse models? [duplicate]

https://stats.stackexchange.com/questions/375374/why-l1-regularization-can-zero-out-the-weights-and-therefore-leads-to-sparse-m

What are L1, L2 and Elastic Net Regularization in neural networks?

https://www.machinecurve.com/index.php/2020/01/21/what-are-l1-l2-and-elastic-net-regularization-in-neural-networks/

Introduction. Sharpness Enhancement and Denoising of Image Using L1-Norm Minimization Technique in Adaptive Bilateral Filter.

https://www.ijsr.net/archive/v3i11/T0NUMTQxMzUy.pdf

作者：凯鲁嘎吉出处：http://www.cnblogs.com/kailugaji/ 本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须在文章页面给出原文链接，否则保留追究法律责任的权利。分类: MATLAB作图标签: 优化问题, 数值分析, MATLAB

<div id="blog_post_info">

好文要顶关注我收藏该文

凯鲁嘎吉
关注 - 10
粉丝 - 209 +加关注 0 0

<div class="clear"></div>
<div id="post_next_prev">

<a href="https://www.cnblogs.com/kailugaji/p/14613210.html" class="p_n_p_prefix">« </a> 上一篇：    <a href="https://www.cnblogs.com/kailugaji/p/14613210.html" title="发布于 2021-04-03 10:14">一类涉及矩阵范数的优化问题</a>
<br>
<a href="https://www.cnblogs.com/kailugaji/p/14682575.html" class="p_n_p_prefix">» </a> 下一篇：    <a href="https://www.cnblogs.com/kailugaji/p/14682575.html" title="发布于 2021-04-20 19:54">MATLAB数值实验：函数逼近法求方程的数值解</a>

posted on 2021-04-08 16:58 凯鲁嘎吉阅读(531) 评论(0) 编辑收藏举报

刷新评论刷新页面返回顶部发表评论编辑预览 f0ce0191-10df-476f-be07-08d7431a305b 自动补全

不改了退出订阅评论我的博客

[Ctrl+Enter快捷键提交]

—.—宅出新高度—.—

联系我： 2441040217@qq.com

Weibo： weibo.com/kailugaji

动态线条动态线条end 昵称：凯鲁嘎吉
园龄： 5年
粉丝： 209
关注： 10 +加关注

搜索

    </div>
</div>

常用链接

我的标签

</ul>

积分与排名

积分 - 372666
排名 - 1568

<h1 class="catListTitle">

随笔分类
(111)

</h1>
<ul class="catList">
        <li class="catListItem" data-category-list-item-visible="true" style="display: block">

Deep Learning(21)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

Distributed Networks(7)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

MATLAB: Clustering Algorithms(25)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

MATLAB作图(27)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

Meta Learning(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

Reinforcement Learning(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

聚类算法简介(25)

        </li>                <li class="catListItem"><a href="javascript:void(0)" onclick="sideColumnManager.loadMore(this)">更多</a></li>
</ul>
<h1 class="catListTitle">

随笔档案
(245)

</h1>
<ul class="catList">
        <li class="catListItem" data-category-list-item-visible="true" style="display: block">

2021年10月(2)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

2021年9月(7)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

2021年8月(10)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

2021年7月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

2021年5月(1)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2021年4月(4)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2021年3月(10)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2021年2月(1)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2021年1月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年12月(1)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年11月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年10月(6)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年9月(4)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年8月(1)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年7月(5)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年6月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年5月(1)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年4月(1)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年3月(2)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年2月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2020年1月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年12月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年11月(13)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年10月(13)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年9月(6)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年8月(2)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年7月(4)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年6月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年5月(3)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年4月(11)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年3月(6)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年2月(4)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2019年1月(2)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2018年12月(14)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2018年11月(14)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2018年10月(2)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2018年9月(6)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2018年6月(1)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2018年5月(8)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2018年3月(14)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2018年1月(6)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2017年9月(9)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2017年6月(9)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2017年5月(5)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2016年11月(7)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2016年10月(2)

        </li>                    <li class="catListItem" data-category-list-item-visible="false" style="display: none">

2016年9月(4)

        </li>                <li class="catListItem"><a href="javascript:void(0)" onclick="sideColumnManager.loadMore(this)">更多</a></li>
</ul>
<h1 class="catListTitle">

文章档案
(2)

</h1>
<ul class="catList">
        <li class="catListItem" data-category-list-item-visible="true" style="display: block">

2019年9月(1)

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

2019年2月(1)

        </li>                
</ul>
<h1 class="catListTitle">

我的其他博客

</h1>
<ul class="catList">
        <li class="catListItem" data-category-list-item-visible="true" style="display: block">

凯鲁嘎吉的微博

        </li>                    <li class="catListItem" data-category-list-item-visible="true" style="display: block">

GitHub-kailugaji

        </li>                
</ul>

浅谈范数正则化

back-top {

back-top span {

back-top a{outline:none}

浅谈范数正则化

1. 向量范数与矩阵范数

2. 为什么要添加正则项？

3. $L_{0}$ 范数

4. $L_{1}$ 范数

5. $L_{2}$ 范数

6. $L_{1}$ 范数与 $L_{2}$ 范数作为正则项的区别

7. 用概率解释传统线性回归模型

8. $L_{2}$ 范等价于Gauss先验

9. $L_{1}$ 范数等价于Laplace先验

10. 矩阵的 $L_{2, 1}$ 范数及 $L_{p, q}$ 范数

11. 矩阵的核范数及Schatten范数

12. MATLAB程序：Laplace分布与Gauss分布的概率密度函数图

13. 参考文献

搜索

常用链接

我的标签

积分与排名

阅读排行榜

评论排行榜

推荐排行榜

最新评论

浅谈范数正则化

back-top {

back-top span {

back-top a{outline:none}

浅谈范数正则化

1. 向量范数与矩阵范数

2. 为什么要添加正则项？

3. L0L0范数

4. L1L1范数

5. L2L2范数

6. L1L1范数与L2L2范数作为正则项的区别

7. 用概率解释传统线性回归模型

8. L2L2范等价于Gauss先验

9. L1L1范数等价于Laplace先验

10. 矩阵的L2,1L2,1范数及Lp,qLp,q范数

11. 矩阵的核范数及Schatten范数

12. MATLAB程序：Laplace分布与Gauss分布的概率密度函数图

13. 参考文献

搜索

常用链接

我的标签

积分与排名

阅读排行榜

评论排行榜

推荐排行榜

最新评论

3. $L_{0}$ 范数

4. $L_{1}$ 范数

5. $L_{2}$ 范数

6. $L_{1}$ 范数与 $L_{2}$ 范数作为正则项的区别

8. $L_{2}$ 范等价于Gauss先验

9. $L_{1}$ 范数等价于Laplace先验

10. 矩阵的 $L_{2, 1}$ 范数及 $L_{p, q}$ 范数