Machine Learning for Hackers by Drew Conway and John Myles White
Author:Drew Conway and John Myles White
Language: eng
Format: mobi
Tags: COMPUTERS / Machine Theory
Publisher: O’Reilly Media
Published: 2012-02-12T16:00:00+00:00
Figure 6-4. Nonlinear data with smooth linear fit
By adding two more inputs, we went from an R2 of 60% to an R2 of 97%. That’s a huge increase. And, in principle, there’s no reason why we can’t follow this logic out as long as we want and keep adding more powers of X to our data set. But as we add more powers, we’ll eventually start to have more inputs than data points. That’s usually worrisome, because it means that we could, in principle, fit our data perfectly. But a more subtle problem with this strategy will present itself before then: the new columns we add to our data are so similar in value to the original columns that lm will simply stop working. In the output from summary shown next, you’ll see this problem referred to as a “singularity.”
df <- transform(df, X4 = X ^ 4) df <- transform(df, X5 = X ^ 5) df <- transform(df, X6 = X ^ 6) df <- transform(df, X7 = X ^ 7) df <- transform(df, X8 = X ^ 8) df <- transform(df, X9 = X ^ 9) df <- transform(df, X10 = X ^ 10) df <- transform(df, X11 = X ^ 11) df <- transform(df, X12 = X ^ 12) df <- transform(df, X13 = X ^ 13) df <- transform(df, X14 = X ^ 14) df <- transform(df, X15 = X ^ 15) summary(lm(Y ~ X + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X10 + X11 + X12 + X13 + X14, data = df)) #Call: #lm(formula = Y ~ X + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + # X10 + X11 + X12 + X13 + X14, data = df) # #Residuals: # Min 1Q Median 3Q Max #-0.242662 -0.038179 0.002771 0.052484 0.210917 # #Coefficients: (1 not defined because of singularities) # Estimate Std. Error t value Pr(>|t|) #(Intercept) -6.909e-02 8.413e-02 -0.821 0.414 #X 1.494e+01 1.056e+01 1.415 0.161 #X2 -2.609e+02 4.275e+02 -0.610 0.543 #X3 3.764e+03 7.863e+03 0.479 0.633 #X4 -3.203e+04 8.020e+04 -0.399 0.691 #X5 1.717e+05 5.050e+05 0.340 0.735 #X6 -6.225e+05 2.089e+06 -0.298 0.766 #X7 1.587e+06 5.881e+06 0.270 0.788 #X8 -2.889e+06 1.146e+07 -0.252 0.801 #X9 3.752e+06 1.544e+07 0.243 0.809 #X10 -3.398e+06 1.414e+07 -0.240 0.811 #X11 2.039e+06 8.384e+06 0.243 0.808 #X12 -7.276e+05 2.906e+06 -0.250 0.803 #X13 1.166e+05 4.467e+05 0.261 0.795 #X14 NA NA NA NA # #Residual standard error: 0.09079 on 87 degrees of freedom #Multiple R-squared: 0.9858, Adjusted R-squared: 0.9837 #F-statistic: 465.2 on 13 and 87 DF, p-value: < 2.2e-16
The problem here is that the new columns we’re adding with larger and larger powers of X are so correlated with the old columns that the linear regression algorithm breaks down and can’t find coefficients for all of the columns separately. Thankfully, there is a solution to this problem that can be found in the mathematical literature: instead of naively adding simple powers of x, we add more complicated
Download
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.
Computer Vision & Pattern Recognition | Expert Systems |
Intelligence & Semantics | Machine Theory |
Natural Language Processing | Neural Networks |
Algorithms of the Intelligent Web by Haralambos Marmanis;Dmitry Babenko(5472)
Hadoop in Practice by Alex Holmes(4502)
Jquery UI in Action : Master the concepts Of Jquery UI: A Step By Step Approach by ANMOL GOYAL(3980)
Functional Programming in JavaScript by Mantyla Dan(2802)
Life 3.0: Being Human in the Age of Artificial Intelligence by Tegmark Max(2539)
Dawn of the New Everything by Jaron Lanier(1579)
The Age of Surveillance Capitalism by Shoshana Zuboff(1514)
Blockchain Basics by Daniel Drescher(1512)
Applied Predictive Modeling by Max Kuhn & Kjell Johnson(1380)
The Rosie Effect by Graeme Simsion(1254)
Once Upon an Algorithm by Martin Erwig(1180)
The Art Of Deception by Kevin Mitnick(1152)
Networks: An Introduction by Newman Mark(1082)
The Artificial Intelligence Imperative by Anastassia Lauterbach(1076)
The Sentient Machine by Amir Husain(1070)
Python Natural Language Processing by Jalaj Thanaki(1056)
The Old New Thing by Raymond Chen(1041)
Rapid Viz: A New Method for the Rapid Visualization of Ideas by Kurt Hanks & Larry Belliston(1040)
Python Machine Learning Case Studies by Danish Haroon(1033)