Dominik Dominik - 9 months ago 29
R Question

fit exponential or hockey stick decay

I'm trying to find an equation to fit my data. I recognize the shape to be

y=-exp(x)
but
nls(y~-a*exp(x*b))
with various parameter start values fail.
y
is negative so the "easy" fit of log(y)~log(a)+bx doesn't work well. I tried log(y+2)~a+bx to make everything positive but that didn't result in the right shape.

Can someone provide some help for fitting this data? Thanks!

my data

y=swediff

x=avgdate

dat2=structure(list(swediff = c(0.0379635202678687, 0.0845477936160927,
0.146010217481196, 0.0416237104326292, 0.0659140490644253, 0.134535534695029,
0.0095147654468483, 0.238456044233877, 0.276025694437364, 0.29435448415394,
0.00301157777812485, 0.19171002685605, 0.277759059448242, 0.00400780564144798,
0.342605838471721, 0.236804884903432, 0.151048712082562, 0.188620966368049,
-0.0615972418208484, -0.00184933102124457, -0.0163171325413688,
0.00370250929658511, 0.30014673206306, 0.135354035472228, 0.00699671782210069,
0.0174510674253347, -0.0145499677497698, 0.0113155610814752,
-0.0683884523999768, 0.20157093417998, 0.186320361855075, -0.115609443650563,
0.069177592825418, -0.0161221161393796, 0.150181081582068, 0.0632121126749741,
0.0769960292118834, 0.061783685314432, 0.0442014176783082, -0.00990798027657931,
-0.00186219548019918, 0.0274216740478325, 0.118878480695049,
0.0592089915185285, -0.00823096478874009, 0.120750948230554,
0.278594307094423, -0.0111994006625954, 0.0379360193757585, 6.29460162030332e-05,
0.0602068958909111, 0.173755367986025, 0.135902420389977, 0.124863098282806,
0.0706487190649132, 0.284186487140106, 0.395475978216811, 0.207437141623493,
0.125297332063683, 0.173198938943319, 0.286364692763195, 0.187181622294097,
0.0155249739392275, 0.0284811383447792, 0.3302152196822, 0.279809724823687,
0.137578862533244, 0.116541887214791, 0.0092271882470954, -0.244303611220416,
-0.0553353805122039, 0.000651263069993122, 0.095154425644194,
0.208713883186232, 0.188079755592783, 0.205982363732731, 0.0411534316934769,
0.0245409797776046, 0.309614117523271, 0.355492037841145, 0.288275778246439,
0.321807979017761, 0.287947161484914, 0.156217353669998, 0.172890522219893,
-0.0179090707980069, 0.0179643473340213, 0.25572307647916, 0.129243837667331,
-0.122599542543602, -0.250851649402414, -0.590393411432316, -0.358595360363101,
-0.0550622410807962, 0.117220026030236, 0.164838184961244, -0.069042978776428,
0.167357636244551, 0.196797358553372, 0.0360344219433584, 0.104666349576576,
0.180262208126638, 0.310581785237572, -0.0391739945001232, 0.183767269585731,
-0.152544955460898, -0.158059621319118, 0.0295497815037277, 0.279152618815129,
0.449891855544769, 0.0647285940606174, 0.198100342057544, 0.180654262068054,
-0.0276951482994757, -0.265654834430295, -0.391084042220014,
-0.232183493444027, -0.350719544866729, -0.430609172084572, -0.360201519737938,
-0.150370469022476, 0.193858381273031, 0.0324799885766501, 0.00919239862196636,
0.0389874409767672, 0.0735492920963668, -0.0246395678120185,
-0.456275023145791, -0.454966261045632, -0.449668204190322, 0.109736227977039,
0.258795304502191, 0.097510631994483, 0.311711079270686, -0.214511654649866,
-0.369907957164848, -1.10256371343854, -0.540460735663232, -0.524390818371818,
-0.807243819575856, -0.821087579040873, -0.464518686689061, -0.00423706426563258,
-0.215993248122225, -0.250111278782723, -0.388687939032414, -0.360549487118099,
-0.650590724024303, -0.519445819136207, -0.202899217122569, 0.0586223761941996,
0.216084208573312, 0.0706260521616783, 0.0756847641524316, 0.0770749593558848,
0.117128900658644, 0.176013651395269, -0.0908276818873188, -0.264146913474664,
-0.397227204225366, -0.520243226629073, -0.315371107607145, -0.317452140016727,
-0.288852876616768, -0.341051691256037, -0.221530347691469, -0.459411134488964,
-0.637940256805166, -0.27643564304224, -0.0101249183839163, 0.286080813914053,
0.185985595857955, 0.0607903622684369, 0.0739324892640134, 0.0613910749095987,
0.081924014991581, 0.191652099453867, 0.0671457700438239, -0.239590260546166,
-0.153731321264683, -0.925001640233871, -0.768208799308804, -0.953874558691501,
-0.879388655106864, -0.731569006141193, -0.80662535340525, -0.546541080560938,
-0.0290726402048612, 0.0899201018225928, -0.427683149429864,
-0.642914036196173, -0.00975702612958795, 0.108679758607406,
0.115330348818079, 0.0636578050454333, 0.12389689549272, 0.0784572603799192,
0.0984567282801977, -0.192624142725435, -0.512093030098511, -0.478389465413485,
-0.621953473157772, -0.14258209361421, 0.215855068127397, 0.10933805943701,
0.0371862858446161, 0.147743925023497, 0.110106354151748, 0.167891959372529,
0.0869501603757304, 0.079154830708945, -0.0538071559599136, 0.329630218853886,
-0.069082249729998, -0.168466429438044, -0.0413130450215624,
-0.220680450761757, -0.778270577227636, -0.579688137311179, -0.377826042303253,
-0.172521105687602, 0.0123755037495739, 0.146581091763792, 0.142823263739513,
0.118485470250309, 0.123284157640724, 0.00262964701743046, -0.324425563039869,
0.0535619813150857, 0.172683623868712, 0.127449390525177, 0.268301759367799,
-0.00433449818031201, -0.371211977194441, -0.457630978639089,
-0.40157582709021, -0.303037159227197, -0.664551261534142, -0.619530450371125,
-0.310697158309397, -0.146761733349917, 0.0236095942741553, 0.00995413363968556,
-0.220037507344027, -0.298201547051362, -0.228524953982665, 0.139997538150043,
-0.0164089841500689, 0.0480316934068684, -0.0202200927067654,
0.210784261132325, 0.235666847665425, 0.187862324743091, -0.0954145478677223,
-0.198055375090692, -0.299881653134342, -0.693717473430792, -0.826874682094633,
-0.537211297019843, 0.0727237791693081, 0.252505366529933, -0.317726197889352,
-0.279117149104267, -0.0368974550397417, 0.292430819817608, 0.14796346005382,
0.037310174880905, 0.111369531415814, 0.222644841684763, 0.232955939088023,
0.0985704990997788, -0.045553014664072, -0.207731060229671, -0.229935960514404,
-0.686597725938266, -0.247064364067584, -0.371521331373238, -0.430762168759812,
-0.196945305918869, 0.215658033923907, 0.390228305203605, 0.300921043791892,
0.043076468096275, -0.00079320334809857, 0.252832538725812, -0.0267246881907509,
-0.0904732083287169, -0.254702182910177, -0.289199140111472,
-0.903663808758846, -0.730480757062446, -0.399450175857138, -0.204946388704593,
0.167241850751596, 0.108062173538109, 0.158279079258542, 0.110014597255262,
-0.0307183404739954, -0.105426447364233, 0.133090339859816, -0.178600042668314,
-0.271699090774754, -0.342264904547807, -1.09265736425287, -0.699172072891869,
-0.0979254975665448, 0.268596423677819, 0.155665651298694, 0.0992990830653245,
0.277629262139193, 0.176822125972084, -0.103178451026634, -0.461191269578747,
-0.216966326644849, -0.1084886076781, -0.154140082268591, -0.652188752264078,
-0.657623380576153, -0.0784736154290889, 0.0742726185810828,
0.0709763425947463, 0.294940653726301, 0.293652811592831, 0.0640793559532362,
-0.0367757730565124, -0.415018906491611, -0.538095246521145,
-0.18261089894835, -0.566401860686253, -0.194554300183061, -0.0327973145856545,
0.106726178908186, 0.0748878637317865, 0.0234611871797437, 0.218270627297708,
-0.15143333931606, -0.028720599313675, -0.604914220546465, -0.484120047066584,
-0.629905362306152, -0.363942052067613, -0.1558791539258, 0.0559939704995262,
-0.267565269947471, -0.507879399175005, -0.163173363661096, -0.0301926306852119,
0.0141783247953152, -0.078676708722438), avgdate = c(107.642857142857,
108.142857142857, 111.928571428571, 101.857142857143, 80.1428571428571,
78.1428571428571, 120.214285714286, 114.785714285714, 110.428571428571,
112.5, 111.571428571429, 103.214285714286, 68.1428571428571,
85, 79.6428571428571, 78.3571428571429, 100, 99.6428571428571,
78.2142857142857, 116.285714285714, 82.5714285714286, 84.4285714285714,
87, 65.7142857142857, 54.8571428571429, 77.2142857142857, 113.214285714286,
99.7142857142857, 98.4285714285714, 116.428571428571, 128.928571428571,
102.785714285714, 96.5, 121.071428571429, 115.214285714286, 116.5,
114.785714285714, 53, 45.2142857142857, 39.5, 72.5714285714286,
84.4285714285714, 112.571428571429, 74.3571428571429, 96.8571428571429,
118.571428571429, 159.642857142857, 164.214285714286, 20.2857142857143,
49.1428571428571, 59.7142857142857, 43.5, 60.8571428571429, 117,
64.6428571428571, 31.2857142857143, 16.5714285714286, 16.5714285714286,
32.3571428571429, 33.5, 42.0714285714286, 47.4285714285714, 68.7142857142857,
113.071428571429, 135, 154.214285714286, 159.714285714286, 167.5,
167.642857142857, 143.642857142857, 97.4285714285714, 40.0714285714286,
22.2142857142857, 18.5714285714286, 39.6428571428571, 66.2857142857143,
80.3571428571429, 56.6428571428571, 28.7142857142857, 30.8571428571429,
19.8571428571429, 18.7857142857143, 85, 138.142857142857, 121.357142857143,
104.785714285714, 64.4285714285714, 147.785714285714, 161.285714285714,
171.214285714286, 173.357142857143, 168.214285714286, 170.5,
172.5, 33.5, 47.2857142857143, 134.214285714286, 101.571428571429,
66.7857142857143, 62.8571428571429, 75.8571428571429, 57.3571428571429,
135.785714285714, 131.357142857143, 140.214285714286, 167.214285714286,
171.857142857143, 157, 120.928571428571, 92.2857142857143, 90.2142857142857,
127.5, 137.285714285714, 167.785714285714, 170.857142857143,
169.714285714286, 180.714285714286, 174.857142857143, 191.071428571429,
162.857142857143, 156.142857142857, 143.642857142857, 112.071428571429,
155.571428571429, 155.928571428571, 156.714285714286, 168.142857142857,
173.5, 180.428571428571, 175.642857142857, 152.642857142857,
144.214285714286, 119, 141.428571428571, 164.714285714286, 175.142857142857,
166.428571428571, 129.571428571429, 143.071428571429, 162.571428571429,
NA, 187.357142857143, 167.285714285714, 159.642857142857, 152.785714285714,
174.142857142857, 180.357142857143, 180.357142857143, 173.642857142857,
177.642857142857, 170.357142857143, 158.928571428571, 141.357142857143,
117.571428571429, 136.571428571429, 118.714285714286, 165.785714285714,
169.5, 171.357142857143, 196.785714285714, 198.428571428571,
195.428571428571, 193.642857142857, 185, 191.928571428571, NA,
195.642857142857, 169, 168.642857142857, 162.928571428571, 147,
136.928571428571, 93.6428571428571, 105.285714285714, 130.428571428571,
139.714285714286, 145.214285714286, 160.928571428571, 164.714285714286,
171.785714285714, 174.5, 185, 187.071428571429, 191.5, 190.214285714286,
NA, 173.357142857143, 171.285714285714, 165.214285714286, 186.642857142857,
178.714285714286, 169.928571428571, 132.428571428571, 133.357142857143,
137.5, 141.357142857143, 152.642857142857, 151.428571428571,
157.142857142857, 172.142857142857, 166.714285714286, 187.785714285714,
179.857142857143, 164.428571428571, 141.357142857143, 140.357142857143,
174.357142857143, 142.214285714286, 126.285714285714, 150.571428571429,
154.571428571429, 128.857142857143, 141.785714285714, 163.5,
169, 165.714285714286, 175.071428571429, 170.285714285714, 180.071428571429,
180.5, 178.357142857143, 167.285714285714, 152.357142857143,
136.714285714286, 154.5, 162.571428571429, 168.714285714286,
164.428571428571, 164, 129.214285714286, 134.142857142857, 152.428571428571,
169, 169.5, 174.857142857143, 174.857142857143, 177.285714285714,
188.071428571429, 191.214285714286, 184.5, 176.214285714286,
170.142857142857, 162.428571428571, 174, 173.428571428571, 167.785714285714,
165.785714285714, 137.714285714286, 147, 145.642857142857, 141.571428571429,
150.5, 158.428571428571, 172.428571428571, 174.285714285714,
171.285714285714, 185.285714285714, 190.857142857143, 177.214285714286,
166.571428571429, 167.571428571429, 169.428571428571, 159.357142857143,
156.357142857143, 142.571428571429, 136.928571428571, 158.714285714286,
159.071428571429, 146.928571428571, 149.214285714286, 152.285714285714,
159.928571428571, 162.785714285714, 174, 179.285714285714, 181.571428571429,
176.428571428571, 176.071428571429, 169.285714285714, 150.428571428571,
134.785714285714, 144, 152.714285714286, 163.785714285714, 159.428571428571,
172.714285714286, 164.642857142857, 167.857142857143, 165.642857142857,
179.571428571429, 179.142857142857, 144.428571428571, 176.357142857143,
152.285714285714, 132, 129.142857142857, 139.071428571429, 161.142857142857,
171.785714285714, 161.642857142857, 171.357142857143, 172.571428571429,
172.428571428571, 180.285714285714, 180.857142857143, 175.142857142857,
164.5, 135.357142857143, 132, 144, 147.714285714286, 166.928571428571,
178.571428571429, 176.071428571429, 173.785714285714, 169.857142857143,
183.785714285714, 193.857142857143, 169.714285714286, 160.928571428571,
144, 148.785714285714, 150, 160.428571428571, 157.785714285714,
174.357142857143, 175.571428571429, 171.785714285714, 180.142857142857,
180.857142857143, 172.928571428571, 168.214285714286, 158.5,
162.214285714286, 166.928571428571, 169.071428571429, 168.5,
178.428571428571, 188.285714285714, 178.142857142857, 175.785714285714,
170.214285714286, 154.571428571429, 170.142857142857, 165.214285714286,
168.642857142857, 167.214285714286, 154.071428571429, 168.357142857143
)), row.names = c(NA, -349L), class = c("tbl_df", "tbl", "data.frame"
), .Names = c("swediff", "avgdate"))

Answer

It's little wonder it doesn't converge. Your model doesn't allow the fit to go above 0.

You need an asymptote parameter in your nls model rather than forcing it to be 0.

If you try something like

nls(y~c-a*exp(x*b),start=list(a=.1,b=.1,c=0.2))

you get a reasonable fit:

Nonlinear fit to data with asymptote parameter added

A statistician will probably ask you why you need a functional form; typical nonparametric regression / smoothing approaches* should describe the relationship as well or slightly better; you could use cross-validation to choose the df.

* I'd expect that a natural cubic spline with say 3 df plus the constant term would do pretty well (it has an extra parameter so that would be little surprise if it does better than the nls exponential fit)


You can do a broken line fit easily with the package segmented:

 library(segmented)
 linmdl <- lm(y~x)
 segmentedmdl <- segmented(linmdl, seg.Z = ~x, psi=150)
 summary(segmentedmdl)

        ***Regression Model with Segmented Relationship(s)***

Call: 
segmented.lm(obj = linmdl, seg.Z = ~x, psi = 150)

Estimated Break-Point(s):
    Est.  St.Err 
154.173   1.962 

Meaningful coefficients of the linear terms:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.1518234  0.0427016   3.555  0.00043 ***
x           -0.0003317  0.0003784  -0.877  0.38134    
U1.x        -0.0191520  0.0015109 -12.676       NA    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1909 on 342 degrees of freedom
Multiple R-Squared: 0.5835,  Adjusted R-squared: 0.5799 

Convergence attained in 2 iterations with relative change 1.425652e-16 

As you see it placed the break at about 154, producing this fit:

fit of segmented regression