Start With the Pipeline, Not the Player
Most teams begin a video feature by asking which player library to drop into a Razor view. That question is premature. A modern interactive video platform has four distinct concerns, and ASP.NET Core is only one part of that stack. Treat video as a single element pointing at a URL and you will hit a wall the first time you need adaptive bitrates, per-user access rules, generated clips that arrive asynchronously, or playback analytics that survive a content delivery network.
The mental model that scales is a pipeline with four stages:
Ingest. Uploads, generation requests, or external imports produce a source file plus metadata.
Process. Transcoding, thumbnailing, captioning, moderation, and enrichment run as background jobs.
Store and deliver. Object storage holds renditions, while a delivery network and a token service handle playback.
Present and interact. ASP.NET Core renders pages and APIs that let people watch, comment, trim, approve, or request a new version.
Each stage has its own failure modes and its own scaling curve. Keeping them separate in your head, and in your solution structure, is the most useful decision you can make early.
What interactive really means
Interactivity is not just a play button. It means a viewer can request a generated clip, leave a comment anchored to a specific second, ask for a different cut, approve a draft, or export a segment for use somewhere else. Every one of those actions is a write operation that eventually triggers video work downstream. That is why the architecture matters more than the player skin. A platform where comments, approvals, and regeneration all flow through one job system behaves predictably. A platform where each feature invents its own background thread does not, and the symptoms usually appear months later as duplicate processing, orphaned files, and statuses nobody trusts.
The four stages mapped to code
Ingest is an endpoint plus a durable record. Processing is a queue plus a worker. Delivery is a short-lived signed URL plus cache headers. Presentation is a Razor component plus a small JSON API. If you can point at the class or service responsible for each of those four things, you are already ahead of most implementations.
Choosing the Backend Shape for a Video Product
ASP.NET Core gives you a flexible toolkit: minimal APIs, controllers, Razor Pages, Blazor components, hosted services, and background workers. The design question is how to arrange them so video work does not strangle the request pipeline.
Start with a modular monolith
A single deployable application with clear internal boundaries is almost always the right starting point. Put the video domain in its own folder or project set: ingestion, processing, delivery, and API. Reference them from one host. You get simple transactions, one deployment, one log stream, and one place to look when something breaks at an inconvenient hour. Split into separate services only when you have a concrete reason: radically different scaling needs, a team boundary, or a runtime mismatch that cannot be solved inside one process.
Where background workers fit
Use a hosted service or a queue-triggered worker for anything that takes longer than a second or two. Transcoding a five-minute clip can take minutes on a shared machine. Generation requests can take longer still. Never do that work inside an HTTP request, even with async and await, because a caller who closes the connection does not stop the work but does stop your ability to report the result cleanly.
A pattern that holds up well: the API writes a job record in the same transaction as the user action, then publishes a message. A worker picks the message up, transitions the job state, and writes results and metrics. The browser polls a status endpoint or subscribes to a stream for updates.
When a second runtime earns its place
Some teams run a Node.js or Python process alongside ASP.NET Core because a particular media library or model SDK is better supported there. That is a reasonable trade, as long as you treat it as an internal service behind a narrow HTTP contract rather than a second source of truth. Define payload shapes once, version them, keep authentication between the two simple and boring, and make sure the .NET side still owns the job table.
Configuration, secrets, and startup validation
Video platforms accumulate keys: storage access secrets, signing secrets, delivery tokens, model API keys. Use the options pattern, bind strongly typed settings objects, and validate them at startup. A missing storage key should crash the application at boot, not at 2 a.m. when the first upload arrives from a customer.
Data Modeling for Videos, Assets, Jobs, and Sessions
Relational databases are an excellent fit for video metadata. PostgreSQL or SQL Server gives you constraints and query power for dashboards, filters, and permission checks.
Core tables
At minimum you want Videos with an identifier, owner, title, slug, status, duration, and timestamps. Add VideoAssets with identifier, video identifier, kind, storage key, size, format, and duration. Add Jobs with identifier, video identifier, type, status, attempts, error text, and timestamps. Add PlaybackSessions with identifier, video identifier, user identifier, start time, bytes served, and device hints. Add Comments with identifier, video identifier, user identifier, an offset in seconds, body text, and visibility.
Index on owner plus status, and on video plus asset kind. Those two cover the majority of list and detail queries you will write, and they keep the dashboard fast as the table grows.
Storing generation parameters without schema churn
Model parameters change constantly: new providers, new aspect ratios, new style presets, new voices. Do not add a column every time. Keep normalized columns for the fields you filter and sort by, and store the rest in a JSON column. In EF Core, map it to a plain object, and include a schema version inside the payload so you can migrate lazily instead of running a painful one-shot data migration under pressure.
Storage keys and signed URLs
Never expose raw storage paths in rendered pages or JSON. Use deterministic keys such as videos/{identifier}/{kind}/{version}.{extension} and generate short-lived signed URLs on demand. Versioned keys let you re-transcode without invalidating caches in confusing ways, and they make it obvious which asset a log line refers to when you are debugging at speed.
Deletion is a workflow, not a statement
When a video is deleted, mark it soft-deleted first, hide it from listings, purge storage on a schedule, and only then remove or archive the row. People ask for undo far more often than you expect, and a hard delete that also removes the source file is unrecoverable. Build the undo path before someone needs it.
Building Interactive Views with Razor Pages and MVC
Server-rendered pages with targeted JavaScript remain the most maintainable approach for content-heavy video sites. You get fast first paint, real URLs, sensible SEO, and pages that still communicate something when scripts fail.
Server rendering plus progressive enhancement
Render the page, metadata, comment thread, and player shell on the server. Then hydrate the player with JavaScript. If a script fails, the viewer still sees the title, description, transcript, and chapter list. That fallback is not a nice-to-have; it is the difference between a degraded page and a blank one.
For the player itself, a thin wrapper component beats a heavy framework binding. Expose a small contract: play, pause, seek, a current-time getter, and an event subscription for time updates. Let the rest of the interface depend on that contract rather than on a specific library, so swapping players later is a contained change instead of a rewrite.
Designing the player component
Keep the player dumb. It receives a manifest URL, a poster image, and caption tracks. It emits events. Anything requiring a network call, such as saving progress, refreshing a playback token, or posting a comment at a timestamp, belongs in a separate service class that the page calls. This separation also makes the player trivial to unit test, because its inputs and outputs are all plain data.
Showing long-running jobs honestly
Generation and transcoding take time, so design for waiting. A pattern that works: an optimistic card appears immediately in a pending state, a status endpoint is polled every few seconds with backoff, polling pauses when the tab is hidden, and a failure shows a real message with a retry action. Never leave a spinner that spins forever. If the worker can report a percentage, show it, but only if the percentage is honest. Fake progress is worse than no progress because it destroys trust in every other status message on the page.
Captions and keyboard access
Captions are not optional. Generate them during processing, store them as an asset of kind caption, and expose them as a track. Add keyboard controls for play, pause, seek, and volume, and make sure focus order in a comment list makes sense. Retrofitting accessibility into a finished player is far more expensive than building it in from the start, and in many markets it is also a legal requirement rather than a preference.
Adding AI Generation Without Losing Control
This is where the platform stops being a media library and starts being a product. The engineering goal is to make generation feel like an ordinary, reliable operation rather than a mysterious external dependency.
Prompt templates belong in data, not code
Do not hardcode prompts in controllers. Store templates with named placeholders and fill them from structured input: subject, tone, duration, aspect ratio, voice, style preset, and constraints. That gives you versioning, A/B testing, and the ability to fix a bad template without a deployment. Keep the rendered prompt alongside the job record so you can always answer what was actually sent, even months later.
Validate inputs before submission. A request rejected in milliseconds is far cheaper than one that fails after two minutes of processing, and it produces a better error message for the person waiting.
A job state machine with a timeline
Model generation explicitly: queued, submitted, processing, needs review, approved, published, with failed and cancelled as terminal branches. Persist every transition with a timestamp and an actor, whether that actor is a person or a worker process. When someone asks what happened to their clip, you want a timeline, not a guess.
Review and revision loops
Generated output usually needs a human pass. Build review as a first-class screen: side-by-side playback of versions, a diff of the prompt, approve and reject actions with reasons, and a one-click regenerate with changes applied. Capture rejection reasons as structured data. Those reasons become the most valuable improvement signal you have, far more useful than aggregate satisfaction scores because they tell you which specific parameter to change.
Quotas and a circuit breaker
Track usage per account and per period in your own database, and enforce limits before submitting work to an external model, not after the bill arrives. Add a global circuit breaker so a bug in a retry loop cannot quietly generate thousands of clips overnight. Log every request with its parameters and an estimated cost so expensive patterns are visible during review rather than during a finance meeting.
Delivery: Adaptive Streaming, Caching, and Real Networks
Delivery is where user experience is won or lost, especially on mobile networks with variable throughput.
HLS versus progressive MP4
Progressive MP4 is simple and perfectly fine for short clips under a minute or two. An adaptive format such as HLS or DASH is the right answer for anything longer, because it adapts bitrate to the viewer connection. Transcode each upload into a ladder, for example 240p, 480p, 720p, and 1080p, and let the player switch rungs as bandwidth changes.
Store playlists and segments in object storage, serve them through a delivery network, and set long cache lifetimes on segment files because they are immutable. Immutability plus long cache lifetimes is what keeps delivery cheap.
Bitrate ladders and bandwidth budgets
Assume a meaningful share of your audience is on mobile data with intermittent connectivity. Keep the initial manifest small, use a poster image instead of autoplaying the first segment, choose a sensible audio bitrate, and offer an explicit low-data mode. These choices affect perceived quality more than any player skin. A clean 480p stream that starts in a second beats a stuttering 1080p stream every time.
Playback tokens and caching rules
Sign playback URLs with a short expiry and bind them to a session or user. If you need tighter control, sign per segment or use edge authorization at the delivery provider. Keep expiry long enough to cover the full video plus buffering, typically one to four hours, and refresh proactively in the player at around eighty percent of the token lifetime instead of waiting for a failed segment request.
Thumbnails and scrub previews
Generate a sprite sheet or a set of preview thumbnails during processing and store them as assets. Hover previews and scrub previews are cheap once the images exist, and they noticeably improve perceived quality, especially for long-form content where people want to find one moment quickly.
Security, Access Control, and Upload Safety
Video content is often the most valuable asset on the site, so treat authorization as a core feature rather than a later hardening pass.
Identity choices
ASP.NET Core Identity works when you want full control and a local user store. External providers reduce friction and move password handling elsewhere. Whichever you pick, keep a single canonical user identifier in your own database and map external identities to it. Never key your video tables on an external provider subject claim.
Authorization at the resource level
Route-level attributes are not enough. You need per-resource checks: is this person the owner, a collaborator, a subscriber at the right tier, or an administrator? Centralize that logic in a policy service and call it from every read and write path. Repeating the rule in three controllers guarantees drift, and drift in authorization is how content leaks.
For multi-tenant products, consider enforcing isolation in the database as well. A tenant identifier on every row, plus filtered queries as default behavior, is far safer than relying on developers to remember a filter clause under deadline pressure.
Watermarking and hotlink resistance
For premium content, overlay a per-session watermark or burn one into the stream. Combine that with short-lived signed URLs, referrer checks at the edge, and rate limits on manifest requests. None of these stops a determined attacker, but together they raise the cost of casual redistribution, which is usually the realistic threat.
Upload safety
Validate file type by inspecting content, not the extension. Scan uploads, cap sizes, process in an isolated worker, and reject anything that does not match expected container formats. Treat every uploaded file as hostile until proven otherwise, because the one time you skip that rule is the time it matters.
Observability and Cost Discipline
A video platform fails in interesting ways: a queue backs up, a transcoder stalls, a token expires early, a model endpoint starts returning rate-limit responses. You want to see all of it before users do.
Tracing across asynchronous boundaries
Attach a correlation identifier to every job and propagate it through logs, storage metadata, and outbound requests. When someone reports a broken video, one identifier should give you the whole history: upload, transcode, generation, review, publish. Log the duration of every pipeline stage as a metric, not just a text line. Percentiles on transcode time tell you more about perceived reliability than raw error counts.
Cost levers that actually matter
Lifecycle rules that move rarely played originals to colder storage tiers. Deleting intermediate renditions after the final ladder is published. Aggressive edge caching so origin egress stays low. Batching generation requests where the provider supports it. Alerting on anomalies, such as a sudden spike in transcode minutes. Review these monthly, because storage and egress costs creep quietly and nobody notices until the invoice doubles.
Failure modes worth pre-building for
A webhook that never arrives, so keep a reconciliation job that polls status on a schedule. Duplicate processing, so make workers idempotent and guard on state transitions. Partial uploads, so use multipart uploads with resume support and clean up abandoned parts. Poison messages, so after a set number of attempts move the job to a dead-letter state and surface it in an admin view. Token expiry during a long video, so refresh proactively rather than reacting to errors.
Mistakes That Cost Teams Months
Doing heavy work inside the request pipeline. If an endpoint regularly takes more than a couple of seconds, that work belongs in a job.
Storing video bytes in the database. It feels convenient until backups grow to terabytes. Use object storage and keep keys and metadata in the relational database.
Serving one format to every viewer. A single high-bitrate file wastes bandwidth and frustrates mobile viewers who abandon the page before the first frame.
Hardcoding prompts and model names. Move them into data so you can iterate without shipping code.
Skipping human review. Automatically publishing generated content is a reputational risk, and approval is cheap compared to a bad clip going live.
No lifecycle policy. Old renditions, orphaned uploads, and abandoned drafts accumulate forever. Schedule cleanup from day one.
Ignoring accessibility. Captions, keyboard controls, and focus management are requirements, not backlog items.
Over-engineering the start. Few platforms need five services on day one. A well-structured monolith with one worker process will take you surprisingly far.
FAQ
Do I need a separate media server?
Only for live streaming or strict DRM requirements. For on-demand content, pre-transcoded adaptive segments served from object storage through a delivery network are simpler, cheaper, and easier to operate.
How should uploads from slow connections be handled?
Use chunked or multipart uploads with resume support, and send files directly to object storage using pre-signed URLs rather than streaming them through the application server. Your API only issues URLs and records metadata.
How long should playback URLs stay valid?
Long enough to cover the full video plus buffering, typically one to four hours. Refresh proactively in the player well before expiry rather than waiting for a failed segment request.
Should I use Blazor or Razor Pages for the video interface?
Either works. Razor Pages with a little JavaScript is usually simpler for content-heavy public pages, while Blazor is attractive for highly interactive admin and review screens. Many teams run both: server-rendered public pages and a Blazor-based internal tool.
How do I keep generation costs predictable?
Track usage per account in your own database, enforce quotas before submitting, add a global circuit breaker, and log every request with its parameters so expensive patterns are visible during review.
What is the smallest viable schema?
Videos, assets, jobs, and playback sessions. Comments, playlists, and analytics can be layered on once the core pipeline is stable and observable.
How do I choose a bitrate ladder?
Start from your audience and your content. Talking-head footage at 720p often looks better than poorly encoded 1080p, and adding an intermediate 360p rung helps viewers on weak mobile connections far more than adding a 4K rung helps anyone.
What about moderation for user uploads?
Run automated checks during processing, store moderation results as flags or assets, and route ambiguous cases to a human queue. Treat moderation as part of the pipeline rather than a separate feature bolted on later, because retrofitting it means reprocessing everything you already have.
