【数学】线性代数的直观理解An Intuitive Guide to Linear Algebra
<style>.entry-content h2 {color: #4682B4;} /*steelblue*/
blockquote {margin-left:1em; color:#000080;} /*navy*/
img {max-width:100%;}
p {margin-top: 10px;}
</style>
<!--以下2句不要封，否则显示x^2等不正常，显示成x^2而不是上标comment 230130-->
<script>
MathJax = {
tex: {
inlineMath: [
['$$$', '$$$'],
['$', '$'],
['\\(', '\\)']
],
processEscapes: true
},
svg: {
fontCache: 'global',
scale: 0.95
}
};
</script>
<script type="text/javascript" id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-svg.js">
</script>
<div class="prose entry-content overflow-clip">
<img src="https://tongyici.net:5556/static/data/la/la.jpg" style="width:100%;height:auto;">
<p>Despite two linear algebra classes, my knowledge consisted of “Matrices, determinants, eigen something something”.</p>
<p>Why? Well, let’s try this course format:</p>
<ul>
<li>Name the course <em>Linear Algebra</em> but focus on things called matrices and vectors</li>
<li>Teach concepts like Row/Column order with mnemonics instead of explaining the reasoning</li>
<li>Favor abstract examples (2d vectors! 3d vectors!) and avoid real-world topics until the final week</li>
</ul>
<p>The survivors are physicists, graphics programmers and other masochists. We missed the key insight:</p>
<p><strong>Linear algebra gives you mini-spreadsheets for your math equations.</strong></p>
<p>We can take a table of data (a matrix) and create updated tables from the original. It’s the power of a spreadsheet written as an equation.</p>
<p>Here’s the linear algebra introduction I wish I had, with a real-world stock market example.</p><br>
<h2>What’s in a name?</h2>
<p>“Algebra” means, roughly, “relationships”. Grade-school algebra explores the relationship between unknown numbers. Without knowing x and y, we can still work out that $(x + y)^2 = x^2 + 2xy + y^2$.</p>
<p>“Linear Algebra” means, roughly, “line-like relationships”. Let’s clarify a bit.</p>
<p><picture><source srcset="https://betterexplained.com/wp-content/webp-express/webp-images/uploads/linear-algebra/linear-vs-nonlinear.png.webp" type="image/webp"><img src="https://betterexplained.com/wp-content/uploads/linear-algebra/linear-vs-nonlinear.png"alt="" class="webpexpress-processed"></picture></p>
<p>Straight lines are predictable. Imagine a rooftop: move forward 3 horizontal feet (relative to the ground) and you might rise 1 foot in elevation (The slope! Rise/run = 1/3). Move forward 6 feet, and you’d expect a rise of 2 feet. Contrast this with climbing a dome: each horizontal foot forward raises you a different amount.</p>
<p>Lines are nice and predictable:</p>
<ul>
<li>If 3 feet forward has a 1-foot rise, then going 10x as far should give a 10x rise (30 feet forward is a 10-foot rise)</li>
<li>If 3 feet forward has a 1-foot rise, and 6 feet has a 2-foot rise, then (3 + 6) feet should have a (1 + 2) foot rise</li>
</ul>
<p>In math terms, an operation F is linear if scaling inputs scales the output, and adding inputs adds the outputs:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/09fa24ebc98dc2d051aa341bdf3af939.png' width='192' height='46' alt=' \begin{aligned} F(ax) &= a \cdot F(x) \\ F(x + y) &= F(x) + F(y) \end{aligned} ' class='tex'></p>
<p>In our example, $F(x)$ calculates the rise when moving forward x feet, and the properties hold:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/5daca54e0e1b24f95be0e19730c347ef.png' width='203' height='18' alt='\displaystyle{F(10 \cdot 3) = 10 \cdot F(3) = 10}' class='tex'></p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/aa091456dff3c4fca20a57b14c3de208.png' width='222' height='18' alt='\displaystyle{F(3+6) = F(3) + F(6) = 3}' class='tex'></p><br>
<h2>Linear Operations</h2>
<p>An operation is a calculation based on some inputs. Which operations are linear and predictable? Multiplication, it seems.</p>
<p>Exponents ($F(x) = x^2$) aren’t predictable: $10^2$ is 100, but $20^2$ is 400. We doubled the input but quadrupled the output.</p>
<p>Surprisingly, regular addition isn’t linear either. Consider the “add three” function $F(x) = x + 3$:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/44f95c15b7b9d5b5f155692a1458f7f5.png' width='88' height='46' alt=' \begin{aligned} F(10) &= 13 \\ F(20) &= 23 \end{aligned} ' class='tex'></p>
<p>We doubled the input and did not double the output. (Yes, $F(x) = x + 3$ happens to be the equation for an <em>offset</em> line, but it’s still not “linear” because $F(10) \neq 10 \cdot F(1)$. Fun.)</p>
<p>So, what types of functions are <em>actually</em> linear? Plain-old scaling by a constant, or functions that look like: $F(x) = ax$. In our roof example, $a = 1/3$.</p>
<p>But life isn’t <em>too</em> boring. We can still combine multiple linear functions ($A(x) = ax, B(x) = bx, C(x)=cx$) into a larger one, $G$:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/9ae605d8349f11e71a75b0c9cb849bc6.png' width='379' height='19' alt='\displaystyle{G(x,y,z) = A(x) + B(y) + C(z) = ax + by + cz }' class='tex'></p>
<p>$G$ is still linear, since doubling the input continues to double the output:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/495b6ea1b9badbcb0db15ceb21899159.png' width='558' height='19' alt='\displaystyle{G(2x, 2y, 2z) = a(2x) + b(2y) + c(2z) = 2(ax + by + cz) = 2 \cdot G(x, y, z)}' class='tex'></p>
<p>We have “mini arithmetic”: multiply inputs by a constant, and add the results. It’s actually useful because we can split inputs apart, analyze them individually, and combine the results:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/aaa2d86705d06b750e305ccacd68ce74.png' width='357' height='18' alt='\displaystyle{G(x,y,z) = G(x,0,0) + G(0,y,0) + G(0,0,z)}' class='tex'></p>
<p>If we allowed non-linear operations (like $x^2$) we couldn’t split our work and combine the results, since $(a+b)^2 \neq a^2 + b^2$. Limiting ourselves to linear operations has its advantages.</p><br>
<h2>Organizing Inputs and Operations</h2>
<p>Most courses hit you in the face with the details of a matrix. “Ok kids, let’s learn to speak. Select a subject, verb and object. Next, conjugate the verb. Then, add the prepositions…”</p>
<p><strong>No!</strong> Grammar is not the focus. What’s the key idea?</p>
<ul>
<li>We have a bunch of inputs to track</li>
<li>We have predictable, linear operations to perform (our “mini-arithmetic”)</li>
<li>We generate a result, perhaps transforming it again</li>
</ul>
<p>Ok. First, how should we track a bunch of inputs? How about a list:</p>
<pre><code>x
y
z
</code></pre>
<p>Not bad. We could write it (x, y, z) too — hang onto that thought.</p>
<p>Next, how should we track our operations? Remember, we only have “mini arithmetic”: multiplications by a constant, with a final addition. If our operation $F$ behaves like this:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/0e14e1462e4d71e33254bca3034f93f6.png' width='198' height='18' alt='\displaystyle{F(x, y, z) = 3x + 4y + 5z}' class='tex'></p>
<p>We could abbreviate the entire function as (3, 4, 5). We know to multiply the first input by the first value, the second input by the second value, the third input by the third value, and add the results.</p>
<p>Only need the first input?</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/7df6fe52f03a79f5d43aa7c581e741a4.png' width='278' height='18' alt='\displaystyle{G(x, y, z) = 3x + 0y + 0z = (3, 0, 0)}' class='tex'></p>
<p>Let’s spice it up: how should we handle multiple sets of inputs? Let’s say we want to run operation F on both (a, b, c) and (x, y, z). We could try this:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/366129689a962a39d13fc8bba305e1b4.png' width='150' height='18' alt='\displaystyle{F(a, b, c, x, y, z) = ?}' class='tex'></p>
<p>But it won’t work: F expects 3 inputs, not 6. We should separate the inputs into groups:</p>
<pre><code>1st Input2nd Input
--------------------
a x
b y
c z
</code></pre>
<p>Much neater.</p>
<p>And how could we run the same input through several operations? Have a row for each operation:</p>
<pre><code>F: 3 4 5
G: 3 0 0
</code></pre>
<p>Neat. We’re getting organized: inputs in vertical columns, operations in horizontal rows.</p><br>
<h2>Visualizing The Matrix</h2>
<p>Words aren’t enough. Here’s how I visualize inputs, operations, and outputs:</p>
<p><picture><source srcset="https://betterexplained.com/wp-content/webp-express/webp-images/uploads/images/linear-algebra-visualize-20121002-221620.png.webp" type="image/webp"><img src="https://betterexplained.com/wp-content/uploads/images//linear-algebra-visualize-20121002-221620.png" alt="linear algebra reference" class="webpexpress-processed"></picture></p>
<p>Imagine “pouring” each input through each operation:</p>
<p><picture><source srcset="https://betterexplained.com/wp-content/webp-express/webp-images/uploads/images/linear-algebra-pour-20121002-220550.png.webp" type="image/webp"><img src="https://betterexplained.com/wp-content/uploads/images//linear-algebra-pour-20121002-220550.png" alt="linear algebra pour in" class="webpexpress-processed"></picture></p>
<p>As an input passes an operation, it creates an output item. In our example, the input (a, b, c) goes against operation F and outputs 3a + 4b + 5c. It goes against operation G and outputs 3a + 0 + 0.</p>
<p>Time for the red pill. A matrix is a shorthand for our diagrams:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/3aa4f8c19764f6aed0969bee9b27bd57.png' width='334' height='66' alt='\text{I}\text{nputs} = A = \begin{bmatrix} \text{i}\text{nput1}&\text{i}\text{nput2}\end{bmatrix} = \begin{bmatrix}a & x\\b & y\\c & z\end{bmatrix}' class='tex'></p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/b6326fd70b2a885f537f6d96a1802b9d.png' width='361' height='44' alt='\text{Operations} = M = \begin{bmatrix}\text{operation1}\\ \text{operation2}\end{bmatrix} = \begin{bmatrix}3 & 4 & 5\\3 & 0 & 0\end{bmatrix}' class='tex'></p>
<p>A matrix is a single variable representing a spreadsheet of inputs or operations.</p>
<p><strong>Trickiness #1: The reading order</strong></p>
<p>Instead of an input => matrix => output flow, we use function notation, like y = f(x) or f(x) = y. We usually write a matrix with a capital letter (F), and a single input column with lowercase (x). Because we have several inputs (A) and outputs (B), they’re considered matrices too:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/175ec48c2ca59b2f32e22b646e2cb1c7.png' width='71' height='13' alt='\displaystyle{MA = B}' class='tex'></p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/b5d8ab037ca1679d48340a1b33940131.png' width='398' height='66' alt=' \begin{bmatrix}3 & 4 & 5\\3 & 0 & 0\end{bmatrix} \begin{bmatrix}a & x\\b & y\\c & z\end{bmatrix} = \begin{bmatrix}3a + 4b + 5c & 3x + 4y + 5z\\ 3a & 3x\end{bmatrix} ' class='tex'></p>
<p><strong>Trickiness #2: The numbering</strong></p>
<p>Matrix size is measured as RxC: row count, then column count, and abbreviated “m x n” (I hear ya, “r x c” would be easier to remember). Items in the matrix are referenced the same way: a<sub>ij</sub> is the ith row and jth column (I hear ya, “i” and “j” are easily confused on a chalkboard). Mnemonics are ok <em>with context</em>, and here’s what I use:</p>
<ul>
<li>RC, like Roman Centurion or RC Cola</li>
<li>Use an “L” shape. Count down the L, then across</li>
</ul>
<p>Why does RC ordering make sense? Our operations matrix is 2×3 and our input matrix is 3×2. Writing them together:</p>
<pre><code>
=
=
</code></pre>
<p>Notice the matrices touch at the “size of operation” and “size of input” (n = p). They should match! If our inputs have 3 components, our operations should expect 3 items. In fact, we can <em>only</em> multiply matrices when n = p.</p>
<p>The output matrix has m operation rows for each input, and q inputs, giving a “m x q” matrix.</p><br>
<h2>Fancier Operations</h2>
<p>Let’s get comfortable with operations. Assuming 3 inputs, we can whip up a few 1-operation matrices:</p>
<ul>
<li>Adder: </li>
<li>Averager: </li>
</ul>
<p>The “Adder” is just a + b + c. The “Averager” is similar: (a + b + c)/3 = a/3 + b/3 + c/3.</p>
<p>Try these 1-liners:</p>
<ul>
<li>First-input only: </li>
<li>Second-input only: </li>
<li>Third-input only: </li>
</ul>
<p>And if we merge them into a single matrix:</p>
<pre><code>
</code></pre>
<p>Whoa — it’s the “identity matrix”, which copies 3 inputs to 3 outputs, unchanged. How about this guy?</p>
<pre><code>
</code></pre>
<p>He reorders the inputs: (x, y, z) becomes (x, z, y).</p>
<p>And this one?</p>
<pre><code>
</code></pre>
<p>He’s an input doubler. We could rewrite him to <code>2*I</code> (the identity matrix) if we were so inclined.</p>
<p>And yes, when we decide to treat inputs as vector coordinates, the operations matrix will transform our vectors. Here’s a <a href="http://blog.stata.com/2011/03/03/understanding-matrices-intuitively-part-1/">few examples</a>:</p>
<ul>
<li>Scale: make all inputs bigger/smaller</li>
<li>Skew: make certain inputs bigger/smaller</li>
<li>Flip: make inputs negative</li>
<li>Rotate: make new coordinates based on old ones (East becomes North, North becomes West, etc.)</li>
</ul>
<p>These are geometric interpretations of multiplication, and how to warp a vector space. Just remember that vectors are <em>examples</em> of data to modify.</p><br>
<h2>A Non-Vector Example: Stock Market Portfolios</h2>
<p>Let’s practice linear algebra in the real world:</p>
<ul>
<li>Input data: stock portfolios with dollars in Apple, Google and Microsoft stock</li>
<li>Operations: the changes in company values after a news event</li>
<li>Output: updated portfolios</li>
</ul>
<p>And a bonus output: let’s make a new portfolio listing the net profit/loss from the event.</p>
<p>Normally, we’d track this in a spreadsheet. Let’s learn to think with linear algebra:</p>
<ul>
<li><p>The input vector could be (\$Apple, \$Google, \$Microsoft), showing the dollars in each stock. (Oh! These dollar values could come from <em>another</em> matrix that multiplied the number of shares by their price. Fancy that!)</p></li>
<li><p>The 4 output operations should be: Update Apple value, Update Google value, Update Microsoft value, Compute Profit</p></li>
</ul>
<p>Visualize the problem. Imagine running through each operation:</p>
<p><picture><source srcset="https://betterexplained.com/wp-content/webp-express/webp-images/uploads/images/linear-algebra-stock-20121002-230152.png.webp" type="image/webp"><img src="https://betterexplained.com/wp-content/uploads/images//linear-algebra-stock-20121002-230152.png" alt="linear algebra stock example" class="webpexpress-processed"></picture></p>
<p>The key is understanding <em>why</em> we’re setting up the matrix like this, not blindly crunching numbers.</p>
<p>Got it? Let’s introduce the scenario.</p>
<p>Suppose a secret iDevice is launched: Apple jumps 20%, Google drops 5%, and Microsoft stays the same. We want to adjust each stock value, using something similar to the identity matrix:</p>
<pre><code>New Apple
New Google
New Microsoft
</code></pre>
<p>The new Apple value is the original, increased by 20% (Google = 5% decrease, Microsoft = no change).</p>
<p>Oh wait! We need the overall profit:</p>
<p>Total change = (.20 * Apple) + (-.05 * Google) + (0 * Microsoft)</p>
<p>Our final operations matrix:</p>
<pre><code>New Apple
New Google
New Microsoft
Total Profit [.20-.05 0]
</code></pre>
<p>Making sense? Three inputs enter, four outputs leave. The first three operations are a “modified copy” and the last brings the changes together.</p>
<p>Now let’s feed in the portfolios for Alice \$1000, \$1000, \$1000) and Bob \$500, \$2000, \$500). We can crunch the numbers by hand, or use a Wolfram Alpha (<a href="https://www.wolframalpha.com/input/?i=++++%7B%7B1.2%2C0%2C0%7D%2C%7B0%2C0.95%2C0%7D%2C%7B0%2C0%2C1%7D%2C%7B.2%2C+-.05%2C+0%7D%7D+.+Transpose%28%5B%7B1000%2C+1000%2C+1000%7D%2C%7B500%2C2000%2C500%7D%5D%29">calculation</a>):</p>
<p><picture><source srcset="https://betterexplained.com/wp-content/webp-express/webp-images/uploads/images/matrix-stock-computation-20121002-231618.png.webp" type="image/webp"><img src="https://betterexplained.com/wp-content/uploads/images//matrix-stock-computation-20121002-231618.png" alt="matrix stock computation" class="webpexpress-processed"></picture></p>
<p>(Note: Inputs should be in columns, but it’s easier to type rows. The Transpose operation, indicated by t (tau), converts rows to columns.)</p>
<p>The final numbers: Alice has \$1200 in AAPL, \$950 in GOOG, \$1000 in MSFT, with a net profit of \$150. Bob has \$600 in AAPL, \$1900 in GOOG, and \$500 in MSFT, with a net profit of \$0.</p>
<p>What’s happening? <strong>We’re doing math with our own spreadsheet.</strong> Linear algebra emerged in the 1800s yet spreadsheets were invented in the 1980s. I blame the gap on poor linear algebra education.</p><br>
<h2>Historical Notes: Solving Simultaneous equations</h2>
<p>An <em>early</em> use of tables of numbers (not yet a “matrix”) was bookkeeping for linear systems:</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/d233925e50828fb082c4d36fad63a306.png' width='162' height='71' alt=' \begin{aligned} x + 2y + 3z &= 3 \\ 2x + 3y + 1z &= -10 \\ 5x + -y + 2z &= 14 \end{aligned} ' class='tex'></p>
<p>becomes</p>
<p><img src='https://betterexplained.com/wp-content/plugins/wp-latexrender/pictures/68fcbe0e409c55cafce74984fbae2ed2.png' width='211' height='66' alt=' \begin{bmatrix}1 & 2 & 3\\2 & 3 & 1\\5 & -1 & 2\end{bmatrix} \begin{bmatrix}x \\y \\ z \end{bmatrix} = \begin{bmatrix}3 \\ -10 \\ 14 \end{bmatrix} ' class='tex'></p>
<p>We can avoid hand cramps by adding/subtracting rows in the matrix and output, vs. rewriting the full equations. As the matrix evolves into the identity matrix, the values of x, y and z are revealed on the output side.</p>
<p>This process, called Gauss-Jordan elimination, saves time. However, linear algebra is mainly about matrix transformations, not solving large sets of equations (it’d be like using Excel for your shopping list).</p><br>
<h2>Terminology, Determinants, and Eigenstuff</h2>
<p>Words have technical categories to describe their use (nouns, verbs, adjectives). Matrices can be similarly subdivided.</p>
<p>Descriptions like “upper-triangular”, “symmetric”, “diagonal” are the shape of the matrix, and influence their transformations.</p>
<p>The <strong>determinant</strong> is the “size” of the output transformation. If the input was a unit vector (representing area or volume of 1), the determinant is the size of the transformed area or volume. A determinant of 0 means matrix is “destructive” and cannot be reversed (similar to multiplying by zero: information was lost).</p>
<p>The <strong>eigenvector</strong> and <strong>eigenvalue</strong> represent the “axes” of the transformation.</p>
<p>Consider spinning a globe: every location faces a new direction, except the poles.</p>
<p>An “eigenvector” is an input that doesn’t change direction when it’s run through the matrix (it points “along the axis”). And although the direction doesn’t change, the size might. The eigenvalue is the amount the eigenvector is scaled up or down when going through the matrix.</p>
<p>(My intuition here is weak, and I’d like to explore more. Here’s a <a href="https://en.wikipedia.org/wiki/File:Eigenvectors.gif">nice diagram</a> and <a href="https://www.youtube.com/watch?v=wXCRcnbCsJA">video</a>.)</p><br>
<h2>Matrices As Inputs</h2>
<p>A funky thought: we can treat the operations matrix as inputs!</p>
<p>Think of a recipe as a list of commands (<em>Add 2 cups of sugar, 3 cups of flour…</em>).</p>
<p>What if we want the metric version? Take the instructions, treat them like text, and convert the units. The recipe is “input” to modify. When we’re done, we can follow the instructions again.</p>
<p>An operations matrix is similar: commands to modify. Applying one operations matrix to another gives a new operations matrix that applies <em>both</em> transformations, in order.</p>
<p>If N is “adjust for portfolio for news” and T is “adjust portfolio for taxes” then applying both:</p>
<p>TN = X</p>
<p>means “Create matrix X, which first adjusts for news, and then adjusts for taxes”. Whoa! We didn’t need an input portfolio, we applied one matrix directly to the other.</p>
<p>The beauty of linear algebra is representing an entire spreadsheet calculation with a single letter. Want to apply the same transformation a few times? Use $N^2$ or $N^3$.</p><br>
<h2>Can We Use Regular Addition, Please?</h2>
<p>Yes, because you asked nicely. Our “mini arithmetic” seems limiting: multiplications, but no addition? Time to expand our brains.</p>
<p>Imagine adding a dummy entry of 1 to our input: (x, y, z) becomes (x, y, z, 1).</p>
<p>Now our operations matrix has an extra, known value to play with! If we want <code>x + 1</code> we can write:</p>
<pre><code>
</code></pre>
<p>And <code>x + y - 3</code> would be:</p>
<pre><code>
</code></pre>
<p>Huzzah!</p>
<p>Want the geeky explanation? We’re pretending our input exists in a 1-higher dimension, and put a “1” in that dimension. We <em>skew</em> that higher dimension, which looks like a <em>slide</em> in the current one. For example: take input (x, y, z, 1) and run it through:</p>
<pre><code>
</code></pre>
<p>The result is (x + 1, y + 1, z + 1, 1). Ignoring the 4th dimension, every input got a +1. We keep the dummy entry, and can do more slides later.</p>
<p>Mini-arithmetic isn’t so limited after all.</p><br>
<h2>Onward</h2>
<p>I’ve overlooked some linear algebra subtleties, and I’m not too concerned. Why?</p>
<p>These metaphors are helping me <em>think</em> with matrices, more than the classes I “aced”. I can finally respond to “Why is linear algebra useful?” with “Why are spreadsheets useful?”</p>
<p>They’re not, unless you want a tool used to attack nearly every real-world problem. Ask a businessman if they’d rather donate a kidney or be banned from Excel forever. That’s the impact of linear algebra we’ve overlooked: efficient notation to bring spreadsheets into our math equations.</p>
<p>Happy math.</p>
</div>
页:
[1]