Skip to content

System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached. #8449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
VidyaKukke opened this issue Mar 12, 2019 · 15 comments
Labels
area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates

Comments

@VidyaKukke
Copy link

VidyaKukke commented Mar 12, 2019

Started to see this recently when I run it within docker. Code has been same forever and it used to work. Not really sure what has changed. Is there a workaround that I could use to get unblocked?

How we setup. .Build() is called only once

IConfiguration` configuration = new ConfigurationBuilder()
                // needed for file watcher to work correctly.
                .SetBasePath(Directory.GetCurrentDirectory())
                .AddJsonFile(CommonConstants.SettingsFileName, optional: false, reloadOnChange: false)
                .AddEnvironmentVariables()
                .Build();

On start we end up with this exception below. Seems to point to something in UseMvc()

Application startup exception: System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached.
   at System.IO.FileSystemWatcher.StartRaisingEvents()
   at System.IO.FileSystemWatcher.StartRaisingEventsIfNotDisposed()
   at System.IO.FileSystemWatcher.set_EnableRaisingEvents(Boolean value)
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.TryEnableFileSystemWatcher()
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.CreateFileChangeToken(String filter)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider.Watch(String filter)
   at Microsoft.AspNetCore.Mvc.RazorPages.Internal.PageActionDescriptorChangeProvider.GetChangeToken()
   at Microsoft.AspNetCore.Mvc.Internal.ActionDescriptorCollectionProvider.GetCompositeChangeToken()
   at Microsoft.Extensions.Primitives.ChangeToken.OnChange(Func`1 changeTokenProducer, Action changeTokenConsumer)
   at Microsoft.AspNetCore.Mvc.Internal.ActionDescriptorCollectionProvider..ctor(IEnumerable`1 actionDescriptorProviders, IEnumerable`1 actionDescriptorChangeProviders)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScoped(ScopedCallSite scopedCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitSingleton(SingletonCallSite singletonCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScoped(ScopedCallSite scopedCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitSingleton(SingletonCallSite singletonCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitTransient(TransientCallSite transientCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScoped(ScopedCallSite scopedCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitSingleton(SingletonCallSite singletonCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScoped(ScopedCallSite scopedCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitSingleton(SingletonCallSite singletonCallSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.AspNetCore.Builder.MvcApplicationBuilderExtensions.UseMvc(IApplicationBuilder app, Action`1 configureRoutes)
   at Microsoft.AspNetCore.Builder.MvcApplicationBuilderExtensions.UseMvc(IApplicationBuilder app)
   at Microsoft.Azure.EventGridEdge.Management.Api.WebHost.ManagementStartup.Configure(IApplicationBuilder app)

.NET Core SDK (reflecting any global.json):
Version: 2.1.500
Commit: b68b931422

Runtime Environment:
OS Name: Windows
OS Version: 10.0.17763
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\2.1.500\

Host (useful for support):
Version: 2.1.6
Commit: 3f4f8eebd8

.NET Core SDKs installed:
2.1.402 [C:\Program Files\dotnet\sdk]
2.1.500 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

@Eilon Eilon added the area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates label Mar 12, 2019
@pranavkm
Copy link
Contributor

Could you try setting RazorViewEngineOptions.AllowRecompilingViewsOnFileChange = false? It'll turn off the file watching that allows runtime recompilation of Razor views or pages.

@VidyaKukke
Copy link
Author

Thanks. In case someone else is facing this issue.
My setup was - Docker on windows (hosting Linux containers). In desperation, just restarted my machine and lo and behold the error went away. Nothing else. So please try that.

@pranavkm - If I hit this issue again I will look at your suggestion.

@lonix1
Copy link
Contributor

lonix1 commented Apr 18, 2019

@pranavkm I also have this problem (linux) - too many inotify handles.

I'm using 2.2 and a SPA (angular) so there is no razor. Yet I recall it's on by default? Where can I disable it?

@Hypnobrew
Copy link

Hypnobrew commented Apr 26, 2019

I experience this error as well on Linux since a week a go.
Running dotnet core 2.2.203.

Debugging native and not in a Docker container.

Edit.
Seem to be related to VS Code for me.
Experience the problem both when debugging on VS Code or via CLI, when the editor is open.
If i close VS Code running from the CLI work as expected.

@rmam-spinworks
Copy link

rmam-spinworks commented Apr 29, 2019

I can confirm this issue in Ubuntu 18.04. Running a brand new webapi project is enough to trigger the exception.

Steps to reproduce:

  • install dotnet 2.2 on Ubuntu 18.04
  • create a webapi project (dotnet new webapi)
  • build & run the project
  • exception.

See also: #7531

@devlnull
Copy link

devlnull commented May 1, 2019

Same here, while vs code is opened this problem occurs and when it's close, works well.

dotnet version -> 2.2.203

@paaland
Copy link

paaland commented May 2, 2019

I found a "fix" by increasing the fs.inotify.max_user_watches and fs.inotify.max_user_instances

mono/mono#12535 (comment)

@rtrimurthulu
Copy link

use command
export DOTNET_USE_POLLING_FILE_WATCHER=true
it will work

@lonix1
Copy link
Contributor

lonix1 commented May 13, 2019

@rtrimurthulu Polling tens of thousands of node/etc files is not a solution. It's better to just reload the IDE than use that option.

(Even if you're using aspnet, there's a good chance you have a node_modules in there somewhere.)

@stevef51
Copy link

Exact same problem - OSX Host, VSCode running Docker Dev-Containers, if I Debug from VSCode then I get the issue, if I run the application from terminal (not debugging) I get the same issue - I have to Quit VSCode to fix, which is painful as it occurs several times a day - will submit a VSCode issue I think

@lonix1
Copy link
Contributor

lonix1 commented Jul 12, 2019

I got used to doing this: ctrl+shift+p then type "reload" then enter.

@stevef51
Copy link

@Ionix1 thanks for the helpful work around, I am actually trialling using Visual Studio for Mac as an alternate IDE for VSCode - a real pitty since I really like VSCode and its dev-containers but this problem is killing my dev time at the moment, I am 99% sure its to do with VSCode since once it occurs then all Containers are affected by the issue, even none dev-containers and the only resolution is to quit VSCode

@paillave
Copy link

run this:
echo fs.inotify.max_user_instances=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
this will increase the amount of authorized instances as too many may be taken by vs code addons

@stevef51
Copy link

@paillave thanks for the effort - however on my Mac host this errors with

fs.inotify.max_user_instances=524288
sysctl: illegal option -- p
usage: sysctl [-bdehiNnoqx] name[=value] ...
sysctl [-bdehNnoqx] -a

and if I run it inside my Docker container (which is where I think it should be run), it also errors saying its a readonly file system, what does appear to work (in the middle of testing this) is to run

docker run -ti --privileged centos sysctl fs.inotify.max_user_instances=524288

which runs the container in privileged mode and changes the hosts (read Linux Hypervisor VM, not the Mac) sysctl setting which is then reflected in all containers - this appears promising

@paillave
Copy link

paillave commented Jul 24, 2019

It is normal that it is readonly in your docker as nothing should change system setups of the virtual machine. For this setup to be done in your docker, it must be done when preparing the image.
About the command I gave, indeed, it works only on linux as sysctl doesn't seem to work the same way on mac. My mistake.Still, try this command on your mac if you didn't do it yet:
echo fs.inotify.max_user_instances=524288 | sudo tee -a /etc/sysctl.conf
The only problem is that the setup will be lost once you restart your computer. The purpose of | sysctl -p is to save this setting for good. Try to find the equivalent command on mac.

@ghost ghost locked as resolved and limited conversation to collaborators Dec 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates
Projects
None yet
Development

No branches or pull requests