Скотт Хансельман предлагает простой и изящный способ генерации файла robots.txt
https://www.hanselman.com/blog/DynamicallyGeneratingRobotstxtForASPNETCoreSitesBasedOnEnvironment.aspx
https://www.hanselman.com/blog/DynamicallyGeneratingRobotstxtForASPNETCoreSitesBasedOnEnvironment.aspx
Hanselman
Dynamically generating robots.txt for ASP.NET Core sites based on environment
I'm putting part of older WebForms portions of my site that still run on bare ...