From 658bb069b1249953fd8d6a4af42d0e8c8a9a2fd5 Mon Sep 17 00:00:00 2001 From: Anthony Salemo Date: Wed, 28 Sep 2022 13:56:34 -0400 Subject: [PATCH] adding a blog post for ENOSPC file watcher limits --- ...mit-for-number-of-file-watchers-reached.md | 111 ++++++++++++++++++ media/2022/09/azure-oss-enospc-1.png | Bin 0 -> 7721 bytes 2 files changed, 111 insertions(+) create mode 100644 _posts/2022-09-28-ENOSPC-System-limit-for-number-of-file-watchers-reached.md create mode 100644 media/2022/09/azure-oss-enospc-1.png diff --git a/_posts/2022-09-28-ENOSPC-System-limit-for-number-of-file-watchers-reached.md b/_posts/2022-09-28-ENOSPC-System-limit-for-number-of-file-watchers-reached.md new file mode 100644 index 00000000..ef83ed4d --- /dev/null +++ b/_posts/2022-09-28-ENOSPC-System-limit-for-number-of-file-watchers-reached.md @@ -0,0 +1,111 @@ +--- +title: "Error: ENOSPC: System limit for number of file watchers reached on Azure App Service Linux" +author_name: "Anthony Salemo" +tags: + - Node + - Linux + - App Service + - Configuration + - ENOSPC +categories: + - Node + - Development +header: + teaser: /assets/images/nodejslogo.png +toc: true +toc_sticky: true +date: 2022-09-28 12:00:00 +--- + +When developing JavaScript-based applications on App Service Linux, notably ones that also come with a development server (e.x, with "hot-reloading"/"live-reloading", or ones that use webpack) you may encounter and error when trying to start the application: + +``` +Error: ENOSPC: System limit for number of file watchers reached, watch '/home/site/wwwroot/public' + at FSWatcher. (node:internal/fs/watchers:244:19) + at Object.watch (node:fs:2251:34) + at createFsWatchInstance (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/lib/nodefs-handler.js:38:15) + at setFsWatchListener (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/lib/nodefs-handler.js:81:15) + at FSWatcher.NodeFsHandler._watchWithNodeFs (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/lib/nodefs-handler.js:233:14) + at FSWatcher.NodeFsHandler._handleDir (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/lib/nodefs-handler.js:429:19) + at FSWatcher. (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/lib/nodefs-handler.js:477:19) + at FSWatcher. (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/lib/nodefs-handler.js:482:16) + at FSReqCallback.oncomplete (node:fs:199:5) + at FSReqCallback.callbackTrampoline (node:internal/async_hooks:130:17) +Emitted 'error' event on FSWatcher instance at: + at FSWatcher._handleError (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/index.js:260:10) + at createFsWatchInstance (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/lib/nodefs-handler.js:40:5) + at setFsWatchListener (/home/site/wwwroot/node_modules/webpack-dev-server/node_modules/chokidar/lib/nodefs-handler.js:81:15) + [... lines matching original stack trace ...] + at FSReqCallback.callbackTrampoline (node:internal/async_hooks:130:17) { + errno: -28, + syscall: 'watch', + code: 'ENOSPC', + path: '/home/site/wwwroot/public', + filename: '/home/site/wwwroot/public' +} +``` + +This is **especially** common with SPA's (Single Page Applications), such as React, Angular, Vue, and others that use the same approach as these libraries/frameworks. + +More than likely, preceeding this, you'll see that the **development** server is actually being ran. You can tell this is the case by making sure [App Service Logging](https://learn.microsoft.com/en-us/azure/app-service/troubleshoot-diagnostic-logs#enable-application-logging-linuxcontainer) is enabled: + +``` +npm start +npm info it worked if it ends with ok +npm info using npm@6.14.15 +npm info using node@v16.14.2 +npm info lifecycle somesite@8.0.37~prestart: somesite@8.0.37 +npm info lifecycle somesite@8.0.37~start: somesite@8.0.37 +> somesite@8.0.37 start /home/site/wwwroot +> react-scripts start + +ℹ ｢wds｣: Project is running at http://172.16.2.2/ +ℹ ｢wds｣: webpack output is served from /portal +ℹ ｢wds｣: Content not from webpack is served from /home/site/wwwroot/public +ℹ ｢wds｣: 404s will fallback to /portal/ +Starting the development server... +``` + +## Why does this happen? +The **ultimate** reason this is happening is because said person is deploying this application to be ran on it's development server. This is **NOT** recommended, for other reasons such as: +- Poor performance. These development servers are never intended to be ran in production (as described by the name, and also these libraries usually make sure to call this out when starting the development server) + - Very long start up times +- Security +- Development servers when using hot-reloading/live-reloading will watch directories and files within for changes. If any files happen to change, this will reload the server. + +There are system limits in places from an OS perpspective on the number of files that can be watched and monitored for changes. On Linux, [inotify](https://en.wikipedia.org/wiki/Inotify) is used under the hood when the development server is used to help monitor for these changes. + +For Ubuntu/Debian based distributions, this can be checked with the following through App Service Linux SSH or Kudu Bash: + +``` +cat /proc/sys/fs/inotify/max_user_watches +``` + +**IMPORTANT**: +Do not try to increase this as the resolution to the problem. The above is a simple way to check the current limit. + +## How to fix this +### Build for production +Depending on the library/framework you're using - **build the application for production**. This usually involves generating a folder with pure static/bundled files. + +If using React or Vue, you may use the following in your project: + +`npm run build` (or `yarn build`). This creates output like the below (depending on the library/framework, the build folder may be named `build`, `dist` or related). + +![React Static Build](/media/2022/09/azure-oss-enospc-1.png) + +> **NOTE**: The contents of the production build folder may look different for your project + +Angular projects would normally use `ng build` - [documentation](https://angular.io/guide/build). + +For more on creating a production build with React - see [here](https://create-react-app.dev/docs/production-build/). For Vue applications, see [here](https://vuejs.org/guide/best-practices/production-deployment.html). + +### Serve for production +This build folder should now be served with something like PM2 - see this [blog post](https://azureossd.github.io/2022/02/22/Using-PM2-on-App-Service-Linux/index.html#javascript-frameworks) on how to configure this. When serving this build folder, all the content is **static**, and will not change (unless it's decided to be rebuilt, normally during deployment). Using something like PM2 to serve the content, it is **not** looking for any changes like a development server. + +This is the difference between running a development server, where the server is watching for changes and what's being served is not built for production and may be dynamic still. + + + + + diff --git a/media/2022/09/azure-oss-enospc-1.png b/media/2022/09/azure-oss-enospc-1.png new file mode 100644 index 0000000000000000000000000000000000000000..7e6ccde82627b7518672a1465f64499024263a05 GIT binary patch literal 7721 zcmai3byQSsw;!ZL1`tG)5(x<%YDT)GQ(B~Z2BcHz5CQ2PIMSkY4!BoyWO&&T*$T=@2Os-!`!reOYx?+5 zDj1JLnCsmu_YJ6JH_F_{OKapN9Bkg|v5Y!=u`kHkele8uZ6|9lyY-|0QSi=o6APmK zcqeCS`_GQ+@Snr)JG=8g)h!rdw>gOnQJlB`!t)vDi_gTLUv+3&(EG+(W?Z)f0L#Cz zRtl!XXYg^x$$ScN=oRo?djsBL@h|;7(X=XV8vF%T8z?Ot!@MZ6IQpQt+2D}{>vJdl z(=v-&i=T^J>t^ipwlb}SjC#FdD*Ia&_Ax3z;Bra}Gw|3DhtpBRwZ@C;Y$~P}G4Ye! zahKG|xdM0ngJmh!{`?tS8oDl%<=u8hb~-M~3G?LFlf&(GtsF_QP4c#XW#7kJpG1>( zHTVa?;@LKC-bFxWb3=FVN@%|&jhuYz2?dLg!>G92Zpy9a}%ZSTNz z$v>H1u^%+6!TN##aPum2RLfbLDI1|Nu3hvOhdfS8M2fo$!SCaHb{Mzpb4^c$>}q-U zgL??ULYNJ(q2x~$k~ES&f8@ro|K1u!kVL?{$IF#F+6cG7t6`mDu<)s*LtBz}BMI2s zVpV-1Xy4u}-@pe5x(}rQ{qUiNv%Li_S)1)CJ0d#iefe+MAzHnj&i*znoTw5mqv_D= zG4B1CmZw6c5>BPBXr5?xT!w=+kI3LcI>67flJrra-dgb4GbO2?Z$QYoh7rc^r}tvG z(2=;Lcdru9uP^FVY`xYQl>?x_lTJ9|Q^(&RHwG;2H`*C8qyxo`h6KKaAW<(v|AG8T zcmlJ80uxHM-sX0mD>A1=VXpa*LGZnd&eY3eyR8R7?aslGPsk71XI~^tTTk+4c9+xZ zGmojB_@M5v7Q$vZ8eF*f_R$-W$?#L3qqcnO^`v=@UWfktm~P_6a@vAo@k};nApT3J zfIQisj&Im&WM=v(%Wo_9?3~eJSH)=&%5wZh7Ak87@L>F*w_LzU#^)CDVb0#MO~Du0 zh794v$%>fnbX2*<5j$UoM7ML%%Dq{+j2cZ1DSZ~3=#1u0my|oXKPAL}Lo%*OMU{)- z`_HCRY`S+?81J?Dta9Ry=NDM@O!_~qV@6p5GV{_V0f%#SuNuvr1QCR)#UZcW**3^Uo5F2=0`X%2 zX~#=8!X^~X!kq7Dp(q|O87Tk68p+FDId; zg+^ccET3LD7=+F;Azs9^r5&#!tp_6(*$CVCAJ>7ot!~M4Jxe2M5Z?Ghx5t12QZ{}WA>IyRF-1GsCa!!R1X#hayb9c|ZB~amyGDsMZw?#0C5;Pn&k9+@Gs!O(i-Ic_$bjzP=w9cCuh2 zXUKX~vG?UqcG|kT%uzicHnZ*U=J!EYJ0@UBcU7EmC27eqCBb;0WjIKF`NEXwqK2H~ zr=GLG2+1O9jWDo8_}YA_<9xY|IftuQ-=fJ&MH*~Ae;s=8t5)w<_T=xP=*lsVwIB}} zsVW7OnH@i|{OL23dB>qQfDVhJDTZR(HJ+;~+$u07HDXxMY#xzHnoW z+@8No_trJ?2;0d{)&1_;@MrZ2{i1ZNS9uZm=H&TsEn^F@kfA9tuJBWdn7)%g{;t<# z;>ZyDMK?blvao$ZDt1DT)zy|eU$i$HYNOOBZmFehkUxvK|7J7h*%vi)2No7A7bcml z3d9{Na~y0t4)G#3x!RZZ=YG7vY0#}@F#&OVgUU8{AP2p!cX{(<8p6cwQ#J@j+Ga<= z#Kn*HKZ)7-V+%cilIHuiJvJvDy?l2uB!Ak-he1|smZCYZWx;)R#SDnDC?_)0;7L-B zC`C5(reMrJ!h73e(6yGK&H%nm85I!ZF+0^xYCDP9k*Ajn>)XvMmXCKD=htqpC@iZS z{H5>qLFHK!9*fr130%zVa&gsugtNDZPG_=1>QA+S&C>KDY&Q|S>=qcNUUm%LzlJCw zeDa{=tJGyKNk$DeYo}}37VbcR!OT*|adT85L5Gv68{HPUA*xVPRmCd&hS|%15ZgaV z?(cQJJ3Lq1Gxb%ejbCO@f6LY+ne~PlT%+vbNTmQf+3ASK^X= zyVe%uTD)S38cKvK?(R7NckNR_5eq@SB>KO0Ai$pzKqIB30uO5V2|VC+e7Hw&KchyY z`hXf!S!D46*z)5{o+&i&k%*#>yBzjbbMijBShnMVaUdH5U`>?!U{_ZC7x{t_yO6o5@M*nFQpDKuAxzL|O<;}RYo)%dn-asPWnul&($!#f+zN4itdXW#jV>6i(k&0i zI^U14$ChMFvm+`IGij0rRHIjcsVXM^?r?sM6Dr30q0Zc4;-v{x2vEDGlu~fBJt=#f z5ow5OP3RFo?bgjDmZgG^qHGSX$0Z)eMRKK3`D&ARKNlOzd?<%Ay8tgNT#*xkt%~fn zs0_(!P($E9VcZxuN=Oo8p-G=gZm%BqX`k{_pQIOyAM{3A;?g@~+!e01MzSP(efmJL z=Yu7e-}J4tsQF+Etv0E1aofZfF<@40!4|-d@&HqJ3LCV(x+aRg=&}ss;;IEnV4h;) ztb5pbdXDgL=R}6C`cUELG7MJ{x~ypr&)cV9DMu+xR`zxS&_7#D*3+aFzY?8e1J$8Ks1In!|Gy?#RRQR+OVyGL3|D8PQns z{Th8(d_$mhzlRsB$#vfLC+Kv?MBQndYs(1d;iZf7Y!gy3^&PRSL2ce8d&Q(ZUkUci z`mzb2^$3y8RbrHoPI@`TC5^ZqE*4?t1#l?|UN9}?nTzC8=CUOTl{NfbcjZ-tGp~pw zq*J&q|D?GtQ4Fo!?=-MJ0L2s#(|{Q(&Q}-mq_A^$>eG$Lp^Kf;DityjV{m!Kxn3=5 zV9V&{QTP_rYFV`Uy6_GN!k^08*pi&%M`RcGt-|hAS76K1#ppP`M~FAM9(2pS&toL$ z{bmS9H)&m_lzIAc)I4mz{M(x|_=X>|+=pBg_p>np`HAUdu2W)Lg9Cv=htKfUfN+)@bBfu%IF4o<4P#CHS6vC(v7ia zr1F%sVJ;>=1>7ctey#E;KW0A_Y*np5o)?TNi?R{=^JVcXMr>KjWkxA!7(O&W74!3X z6(J**03)wtjt7(i!^)_Rd)}?U9|N=!7HXe=;nsD#fdBETt$}VOtzL!KV_u)+?Puje zvFn@TnyJYHwqsX;cK9AGz0A?awE%esXx;lB0e57N720d9 zrZ$jeq2)rfeYcNi%F>kJXGs|wR#~su{rc&pGDfut#U+W=aSaAmOpf;p36}`=n!;Fc z2(%kcHhgKgserPNQQUhyWsG_t$g-3MLsMt_*IvClGbsYx84JihOs@8Up_>r!qmR1* zHM2GIxBM**8{@mgL2f?Gy)usMd(33)c^dsr810R9 zeYg}6hyVU?@U6Sx-uHCgXqX3t4RF-^IS!eM76gZxsjcJdDj9qT%SfDUVlFXH$xO+LJkip7UZ%d zz&1R^h%RbF=~-a>V4fh{F{$MFif#OoMt~JQykQ=<62yQ|C)fq!=Le`O6jWUk?ODj4 zH6nT=r$U!Zy*=qyaVcWkncuw*4F}&-!|kG=scH0ya-X}2oRZ4j?w*c8pDw?PV~wWK zIzphV0wiD24kwBEF?;Ixd&N2d?2C+g2XO;p;F>>hFg3mVDksM;m#Yr9Wf#c-D=@QfaifnN_&uitzDI|Oo9#Zg zjcEJrJ+bnwF!28h)W7x}_z_h@s9(B{fUmc(-Ya8lECwggBA%IVUk++pn63@=JE6uE zMN6{rQv(JK#yJ|25rw00LgO{@%ArH#TOKQ^2aQ|$nifHbrID)uE28pQ8D{TY%)q&k zZ6GVa1Ta?^!*Xeh;~G+I1xjiv$V2 zzdh`-xb8&k4LiB5tu%lAwzXtN;1XlrdvG#NE?fD=vsiG@IGdT7w4ZgB22sIIO+(YD zzP|fgDXYwdZ)q+7-WxcUIdrG2>5}?!6+`jFvVlCZ%s+SW*L5Y8ackbY>rR)t3oOF~ z{#g)l zuFtk;w|#QFWELpK1>MJBPv~6ITxnzbF)ef`8)a&UYa_8eT;X1Qi5ya|K<0Q3v^60k z8uYVOv+}}zg}mqBPWYPGqb97jzV8%svOueL^}%vmT5kC%qwk35zCb)FrYVhfd5#v< zndE9(FsG({DRE7{3zW9;0ioR`@4?ee5ruN&>fC=7WamZ|_ris*i^#LAsn_dtd90CF zAQ6XSF-^)GfgTI)*=Wt9l+zN9D6pAQ{pwQQpSlVjl-eX1dt$i6X;{V6M7ZHr7TPmya=MP1;SVxCdO=68f!@54_p!bwl4;x4!r!m|3{22Unc+_ zi}?dHSkN}Ht4G0%suM1f#PLqxbCh}H|EENC;5##}7Wk7SJq~@TvDt@exO0EXo7OWA z$(;0@OM6titQ^#rI%CTy#Wmz64v3xmHEgHqIb7Qmff~XhYIwqFPh;04J|)@6h!AFM9~VN13E*M)RT`(O-qnhnb`Dq3f26GrBezT?kJ& z25rb5d6MYyGN2ix8=^~lY0o}v=kkz_&kkUAc(B((sDw@UAsve{Gc8n%a5kfK&g11P z$hU3ijJTRe8%caP`vJ||s1E6`^Lpi`%Ds^ChQ>VJCYuYpscixrXvE6s_23clDF>U- zKri~oPY@V}f?Y87?&*~_z&Gj15L{y10C!-+#j*I^t8c;jQ=g!&lNrhErIc)jo@m>= z{|es#XjH9KudprnO@Ghu#SdD3V>0Tz_r*Byvew@+L`BXFK&iY$*mX<@oK0kin&E`~ zChc9c3WH!wL6BD~o$--Yw?T2H?BMpk`{oPMdj8%w6gCBWX}xMk_^&5Etr`_95aoRq zz9lO)b7);?!}x8D9c@Q|zVPb5(~fgLDB%6?MNEDfoQmm1$ZY|a4+InOR=-Aaz?AgY zCR66@h5KCV#sYX!(kSdXvi36>d2iy@$19qv<@AYlsSiFCN7io4WC6;rd9cD!6^zm3 zfjElD6!@v(@8OiLbLE4r#V4fbCLR6jDDHfg5B^51`FjHa*!FDpNP3VN9T{tp7z4?ypVhe;TieWw3+FtWcnfKDWu9e_Xb0P-EZYxJx~>1+(m8UnP>U{a+By$r z>(w*7^?)s{vqF!i8=0D=a*Zcf(4^Cc9)l2L_d6`Hkq5oCh0WcDj+7mpp?!+gc3&+ZmDf z)vgiH+GX#3FGl(xtByZ>4|)bIwU8Za(;r(=W@FxA-~k zP0m&0{5&KT8^91+20r0OgAvV?L>&kw$f;@|Q6=UR&C2u?O^+yi!A5yCPG(Tx)OgjI zB{G2G+3O^9_ZFiZ16uFJ5Q&UJ4r>0st`9 zmCeY1#n&wUu^Ob7$o0G>W*}4zJ}!EjVRY~3E$Zep@W%E;jx=NqQ+*CLJot@#gs=jQ)7qrkw~o(zq~ z;)fu2E!_y$Uev5~Dgpt)2yT8JR?qoUkRtRGMNDzUcw9r)bBVbZLG}j|=`I#n+2g6Bc}Y zgY4CE<*f|b6mstkK1-Mz4*E@K&FWsocic{vcVkn(y(_&@CETt0NiYld~P_+ybc zRK}HQI>*X*6bDMa-Cy4fOV{K;9t!@P(dif=O*$_&2 z$#-QOf7s+!(~kE^hg2H!sf6fyelxPWOV`3o>I^w&i-P%$@6B&Cw7hLT*8i*VoNXuT zhz5IIviLOTylUsZ@2*REFb)_GY#MdiR&o=}-Q;c&e|N0U?>RIvEe%+4nqRCdxorv z+6E8JutXgU*PU3zS!?&n4uA+xw>lnAu_S_d5NbsX_Eoo^O>ZQ-5w3d-f(*1CqVQgF zZ7~#O%%QD|jd#P2`VD#q1-zh2z|^78uLUi2>4hqjvx`C1*97mV0xCgBRWW7U-@c9& z|7kC{9ebk-{j~AkDMX0mvwib@&j&p@0%0(_+tHUY{Xe&ui?hMiJJkult%#oV3K)%F z&ZOGTDQW_BW$=^B-@4NZ4moE9ViRWipom|5e^N)PzfH+um)Ttziq+&6mkUrj1{W|VJ2PMEfqn*R)ElVp>o|=1hy~~eBDcprC59m>UsevpDo!%NEiV~U zj`DhQ1--Ev!zxUfwY!+RZiMVuH6D{hSaIX7qT#~xw zw^ZokiaQWi>B$vw13q@RWgi0}%@a6c>5v!MaMF+k4Ax|vd6}Ueky9XQasaE3)qU}{ zDjZD~d?>UayjgI$&7DcFLm&6S83k@0t^gmH>J#ZZ5OsySu};`fnUdYWH`I+!UOG%~ z(Ld?_&nTmc{m~x8HnPydYrhdxHN)YARI9S-$))L}uw9 literal 0 HcmV?d00001