If you're creating a containerized application from an ASP.NET Core Empty project and you just want to serve static files your Program.cs can just look like below:
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.UseDefaultFiles();
app.UseStaticFiles();
app.MapFallbackToFile("index.html");
app.Run();
UseDefaultFiles() and MapFallbackToFile while similar are just a little different and the later is used a little more in SPAs (but can also direct to a general Not Found file as well).
For a containerized app serving static files, the project directory would look similar to the following: