A topic-grouped index of interview and review questions across .NET, computer science, architecture, AI, data, networks, security, and engineering practice. Each answer links back to the note that establishes the underlying mechanism and tradeoffs.
A topic-grouped index of interview and review questions across .NET, computer science, architecture, AI, data, networks, security, and engineering practice. Each answer links back to the note that establishes the underlying mechanism and tradeoffs.
383 questions
When is targeting .NET Standard still justified?
Targeting .NET Standard is justified when one library build must be usable on more than one .NET implementation, which in practice usually means .NET Framework consumers remain.
netstandard2.0is the highest version .NET Framework can consume, so it is the target that keeps that reach. When every consumer runs modern .NET, a modernnetX.0target is the better default..NET Standard is a versioned API contract, not a runtime, and meeting the contract does not by itself guarantee that a library runs on a given host. Targeting a version fixes the baseline reference API surface the library compiles against. Each .NET implementation supports particular .NET Standard versions starting from particular releases, so a consumer can use the library only on an implementation release that supports the targeted version. Compatible NuGet packages can add APIs beyond that baseline, and every package dependency must then also support the consumer’s platform. After .NET 5 unified the target frameworks, no new .NET Standard versions are planned beyond 2.1.
Target Consumers that can use it Cost netstandard2.0onlyReleases of .NET Framework, .NET Core, Mono, Xamarin and modern .NET that implement 2.0 Limited to the 2.0 baseline plus APIs from compatible packages, with no modern-target APIs or optimizations netstandard2.1Implementations and releases that support 2.1, including modern .NET, but not .NET Framework Gives up .NET Framework reach without the full modern API set Modern netX.0onlyApps on that .NET version or later .NET Framework and older runtimes cannot load it Multi-target, such as net8.0;netstandard2.0Both groups, each receiving the assembly built for it More builds, tests, conditional code and package metadata, justified only when distinct consumer groups need distinct assets Modern .NET implements .NET Standard 2.1 and earlier, so an existing
netstandard2.0library can keep serving modern consumers, provided its package dependencies also support those runtimes. Retargeting is worthwhile when the library needs APIs or features that only a newer target provides.Sources: Microsoft Learn: .NET Standard · .NET Blog: The future of .NET Standard
What are the main layers of .NET, and why does it help to separate them?
A practical model splits .NET into languages and compilers, SDK tooling, the runtime, the base libraries and application frameworks. Separating them helps because each owns different behavior, so naming the owning layer turns a vague symptom into a testable hypothesis.
Layer What it owns Example of a problem it owns Languages and compilers Translate C#, F# or Visual Basic source into IL stored in assemblies. A compile error, or unexpected behavior of a language feature. SDK and tools Restore packages, then build, test and publish projects, mainly through the dotnetCLI.A package-restore, build or publish failure. Runtime (CLR) Load assemblies and run managed code: JIT compilation, garbage collection, exception handling and threading. GC pauses, or slow startup while methods are compiled. Base libraries Shared APIs such as collections, I/O, networking and threading primitives. The behavior or cost of a specific API, such as a collection or stream. Application frameworks Application models such as ASP.NET Core, WPF or .NET MAUI, including pipelines like HTTP middleware. A slow or misordered request pipeline. A symptom often appears in a different layer from its cause. Rising memory shows up as garbage-collection activity in the runtime even when application code is keeping objects reachable. A slow HTTP request can come from the framework pipeline, synchronous I/O in application code or the generated machine code. “.NET is slow” is not a useful hypothesis until the owning part is named.
This is a conceptual map rather than an official fixed taxonomy, and component and version boundaries change between releases.
Sources: Microsoft Learn: Introduction to .NET · Microsoft Learn: Common Language Runtime overview
What must an API establish before it trusts a bearer JWT?
Before trusting a bearer JWT, an API must confirm all of the following against its own configured trust settings:
- Signature: the token was signed with a key the API accepts, so its contents were not changed after signing.
- Issuer: the authority the API expects issued it.
- Audience: it was issued for this API rather than for another service.
- Lifetime: it has not expired.
Each check answers a different question, so none can stand in for another. A valid signature proves only that an accepted key signed the token. It does not show that the expected authority issued the token, that this API was the intended recipient, or that the token is still valid. Any further claims the API requires depend on its own trust configuration.
In ASP.NET Core, the JWT bearer handler performs these checks according to its
TokenValidationParameters. Disabling issuer, audience or lifetime validation because the token is signed removes the protection that check provides.Microsoft: Configure JWT bearer authentication in ASP.NET Core
What is the difference between
GetService<T>andGetRequiredService<T>?
GetService<T>returnsnullwhen the requested registration is absent.GetRequiredService<T>throwsInvalidOperationExceptionat resolution. Microsoft: GetRequiredService · Microsoft: service lifetimes · Microsoft: DI guidelines
What is the normal execution order of ASP.NET Core MVC filter types?
After routing selects an MVC action, the filter stages run in this order: authorization, resource, action, then result. Exception filters are not a sequential stage; they run only when an unhandled exception occurs.
- Authorization filters run first and can short-circuit an unauthorized request.
- Resource filters run their before logic next, ahead of model binding, and their after logic last, once the rest of the pipeline has completed.
- Action filters run immediately before and after the action method.
- Result filters run immediately before and after the action result executes.
Full flow: authorization → resource (before) → action (before) → action method → action (after) → result (before) → result execution → result (after) → resource (after).
Action filters finish before result filters start, so an action filter’s after logic can still replace the result. Only resource filters wrap both the action and the result stages.
Which exceptions can ASP.NET Core MVC exception filters handle, and what happens after they run?
MVC exception filters handle only unhandled exceptions raised while MVC creates the controller, binds the model, runs action filters or runs the action method. They are not a before-and-after stage; they run only when such an exception occurs.
Not covered:
- Exceptions from authorization filters and resource filters, which run outside that part of the pipeline.
- Exceptions from result filters or from executing the action result.
- Exceptions from middleware, routing or non-MVC endpoints, which run outside MVC.
After an exception filter runs:
- Handled: the filter sets
ExceptionHandledtotrueor assigns aResult, and the exception stops propagating. A suppliedResultis executed as the response. Ordinary result filters do not run around it; only always-run result filters, such asIAlwaysRunResultFilterimplementations, do.- Not handled: the exception leaves the MVC pipeline and continues to outer middleware, such as exception-handling middleware.
Because of these gaps, exception-handling middleware is the broader boundary for a uniform API error response.
Within one ASP.NET Core MVC filter type, how do scope and the
Orderproperty determine execution order?By default, scope decides the order among filters of one type: global filters wrap controller filters, which wrap action filters. An explicit
Ordervalue, exposed throughIOrderedFilter, takes precedence over scope.
- Equal
Ordervalues: before logic runs global → controller → action, and after logic runs action → controller → global.- Different
Ordervalues: filters are sorted byOrderfirst and by scope second. A lower value runs its before logic earlier and its after logic later, so it wraps filters with higher values regardless of their scope.For example, an action-level action filter with
Order0 runs its before logic ahead of a global action filter withOrder1.This ordering only arranges filters of the same type. It cannot move an action filter ahead of an authorization filter, because each filter type runs at its own stage of the MVC pipeline.
How does middleware differ from an MVC action filter?
Middleware runs in the general ASP.NET Core request pipeline for every request that reaches it, and it can run before an endpoint is selected. An MVC action filter runs only inside MVC, around a controller action after that action has been selected. It can read and change the action’s arguments and result.
Aspect Middleware MVC action filter Position A pipeline component that runs in registration order and can run before routing selects an endpoint. Runs inside MVC after action selection, immediately before and after the action method. Requests covered Every request that reaches it, including static files, Minimal API endpoints and requests that never enter MVC. An earlier component that short-circuits, or a branch that bypasses it, removes that coverage. Only the controller actions the filter is applied to. Context HttpContextand the rest of the pipeline. It has no model-bound arguments or action result.Action context: model-bound arguments, which it can read or replace before the action runs, and the result, which it can inspect or replace afterward. Fits Concerns that span endpoint types, such as exception handling, request logging, CORS and authentication. Behavior that depends on action arguments or results for a set of controllers or actions. Minimal API endpoints do not run MVC action filters. Their endpoint-scoped equivalent is endpoint filters.
Microsoft: Filters in ASP.NET Core · Microsoft: Middleware in Minimal API apps
How can an ASP.NET Core application handle unhandled exceptions consistently?
A single exception-handling middleware, such as
UseExceptionHandler, is registered near the start of the pipeline so that it wraps every later component. It turns any unhandled exception from those components into one safe, consistent error response. In ASP.NET Core 8 and later, a registeredIExceptionHandlerimplementation can keep the mapping from exception to response in one place.
- A later component or the endpoint throws. The exception propagates back out through each
nextcall until it reaches the handler.- A safe server-side diagnostic records the failure. This diagnostic is arranged deliberately, not assumed. In ASP.NET Core 10, when an
IExceptionHandlerreports an exception as handled, the middleware’s own diagnostics are suppressed by default. The handler therefore records what an investigation needs.- The handler writes the application’s error contract: an error status code and a body. With
AddProblemDetails()registered, that body can be a standard Problem Details response.- Production responses omit stack traces and other sensitive details.
Limits:
- Coverage: components registered before the handler are outside it, so their exceptions are not caught.
- Started responses: once the response has started, the status code and headers may already be sent. The handler then cannot reliably replace them with an error response.
- Swallowing: logging the exception but returning a success status hides the failure from clients. The boundary either writes a real error response or lets the exception continue.
How does a request move through the ASP.NET Core middleware pipeline?
A request passes inward through middleware in registration order until something produces a response, normally the endpoint. The response then passes back outward through the same components in reverse order.
- Each component receives
HttpContextand anextdelegate. Code beforeawait next(context)runs on the way in.- A component either calls
nextto pass control inward, or returns without calling it to short-circuit. After a short-circuit, later components and the endpoint do not run.- In a typical API pipeline, routing selects the endpoint. Authentication then tries to establish
HttpContext.Userand normally continues even without a valid credential. Authorization evaluates the endpoint’s policy. If the caller fails it, authorization triggers the scheme’s challenge for an unauthenticated caller, or forbid for an authenticated one. The scheme decides the response, for example401or403for bearer tokens, or a redirect for cookies.- The endpoint runs and produces the response.
- Control returns through earlier components in reverse order, so code after
nextruns on the way out.Code after
nextcan change the status code or headers only while the response has not started. OnceResponse.HasStartedis true, they may already be on the wire. Branches such asMapandUseWhenchange which components a request passes through, so order within a branch matters as well.Microsoft: Middleware in Minimal API apps · Microsoft: ASP.NET Core authentication overview · Microsoft: Handle errors in ASP.NET Core
How do asynchronous waiting and multithreading differ?
Asynchronous waiting is about not occupying a thread while an operation is pending; multithreading is about running work on more than one thread. An
asyncmethod does not create a thread by itself.
Aspect Asynchronous waiting Multithreading What it addresses Time spent waiting for I/O or another pending operation. Executing several pieces of work on separate threads. Thread use While true asynchronous I/O is pending, the method is suspended and no thread runs it. A thread is needed again only when completion makes the continuation runnable. Each piece of work occupies its thread while it runs. The threads run in parallel only when enough processors are available; otherwise the scheduler interleaves them. CPU-bound work No gain by itself, because computation still needs a thread to execute. Several pieces of CPU-bound work can make progress at the same time when processors allow. The two ideas are independent: asynchronous code can wait without extra threads, and multithreaded code can still block its threads while waiting.
Microsoft:
awaitoperator · Microsoft: ASP.NET Core best practices · Microsoft: Task-based asynchronous programming
What changes when an incomplete task is consumed with
awaitinstead ofTask.Result?
awaitsuspends the method and frees the current thread until the task completes, whileTask.Resultblocks the calling thread for the entire wait. That difference also changes deadlock risk and how failures surface.
Aspect awaitTask.ResultCalling thread Returns to its caller while the task is pending, so the thread can run other work. Stays blocked and idle until the task completes. Resumption Runs the rest of the method later as a continuation, through the captured SynchronizationContextor non-defaultTaskSchedulerwhen one applies.Continues on the same thread once it is released; no continuation is involved. Deadlock risk Avoids the context deadlock, because the waiting code does not hold the context’s thread. Can deadlock when the task’s continuation needs a single-threaded context, such as a UI thread, that the blocked caller occupies. Each side then waits for the other. Failures Rethrows the task’s exception itself; when a task holds several exceptions, awaitrethrows one of them.Throws an AggregateExceptionthat wraps the task’s exception or exceptions.Microsoft: Task-based asynchronous programming · Microsoft: Synchronous wrappers for asynchronous methods · Microsoft:
Task<TResult>.Result
What does
ConfigureAwait(false)change, and when is it useful?
ConfigureAwait(false)tells anawaitthat its continuation does not need to resume through the capturedSynchronizationContextor non-defaultTaskScheduler. It is mainly useful in general-purpose library code that never needs its caller’s context.What changes:
- By default, awaiting an incomplete task captures the current
SynchronizationContext, or a non-default currentTaskScheduler, and resumes through it. In a UI app, that returns the continuation to the UI thread.- With
ConfigureAwait(false), that scheduling context is not captured. The continuation can run on a ThreadPool thread or directly on the thread that completed the operation.What it does not do:
- It does not force a thread switch. If the awaited task has already completed, the method continues synchronously on the current thread, still inside the original context.
- It does not stop
ExecutionContextflow. Ambient state such asAsyncLocal<T>values still reaches the continuation by default.- It is not a reliable fix for code that blocks on asynchronous work. Every
awaitalong the path would need it; keeping the call chain asynchronous is the durable fix.Where it fits:
- Reusable libraries: library code stays independent of whatever context its caller has and avoids the cost of queuing back to it.
- UI code: usually keeps the default, because code after the
awaitoften updates controls that belong to the UI thread.- ASP.NET Core: normally has no custom
SynchronizationContext, so the setting usually changes little.HttpContextlifetime and thread safety remain separate concerns:HttpContextis not thread-safe and is unsafe to use outside the processing of its request, whichever thread resumes the method.Microsoft: Task.ConfigureAwait · .NET Blog: ConfigureAwait FAQ · Microsoft: HttpContext in ASP.NET Core
Why can true asynchronous I/O improve server concurrency without keeping a thread for every wait?
Asynchronous I/O improves concurrency because a pending operation needs no thread. At the
await, the request code gives its worker thread back to the ThreadPool, and a thread is needed again only to run the continuation after the I/O completes.
- Request code starts an I/O operation, such as a database query, and awaits the incomplete task.
- The method suspends and returns, and the worker thread goes back to the pool.
- While the I/O is pending, that thread can serve other requests.
- When the I/O completes, the continuation becomes runnable and a thread resumes the method. Whether the continuation is queued or runs directly on the completing thread depends on completion timing and the scheduling context.
A synchronous blocking call instead keeps its thread idle for the entire wait. Under load, blocked threads pile up, the ThreadPool runs short of free threads, and response times degrade.
Async I/O removes threads as the limit on how many waits can be in flight. It does not guarantee a particular capacity or throughput: CPU work, remaining blocking calls, connection pools, downstream services and other resource limits still constrain the system.
Microsoft:
awaitoperator · Microsoft: ASP.NET Core best practices · .NET Blog: ConfigureAwait FAQ
When is
Task.Runuseful, and why is it usually wrong around asynchronous I/O?
Task.Runis useful for moving CPU-bound work off a thread that must stay responsive, typically a UI thread. Around an API that is already asynchronous it gains nothing: the I/O does not finish sooner, and the wrapper only adds a ThreadPool scheduling hop.
- What it does: queues a delegate to the ThreadPool and returns a task that represents that work.
- Useful case: a UI event handler awaits
Task.Run(() => BuildReport(data)), so the UI thread keeps handling input during a long calculation. The calculation still consumes CPU and occupies a pool thread while it runs.- Already-asynchronous I/O:
await Task.Run(() => client.GetStringAsync(url))only starts the same request from a pool thread. The wait was already threadless, so awaitingclient.GetStringAsync(url)directly gives the same result with less overhead.- Synchronous blocking I/O: wrapping a blocking call does not make it asynchronous. The block moves to a pool thread, which stays occupied for the whole wait.
On a server, request code already runs on ThreadPool threads, so wrapping work in
Task.Runand immediately awaiting it frees nothing and adds scheduling and pool pressure.Microsoft: Task-based asynchronous programming · Microsoft: ASP.NET Core best practices
What is a
Channel<T>, and what reliability does it provide?
Channel<T>is an in-process, in-memory queue that hands items from producers to consumers, with asynchronous waiting on both sides. Its reliability ends at the process boundary. Queued items are not persisted, so a crash loses them, and code on other hosts cannot read them.How it works:
- Writer and reader: producers write through
channel.Writer, for example withWriteAsync. Consumers read throughchannel.Reader, often withawait foreachoverReadAllAsync(). A reader waiting on an empty channel, or a writer waiting on a full bounded channel, waits without blocking a thread.- Capacity:
Channel.CreateBounded<T>(capacity)has a fixed buffer and a full-mode policy that either makes producers wait or drops an item.Channel.CreateUnbounded<T>()keeps growing.- Completion:
Writer.Complete()signals that no more items will arrive, so readers drain the buffer and their read loop ends.A typical use is an HTTP endpoint that queues a job while a
BackgroundServicein the same process handles it.Reliability limits:
- No persistence: buffered items exist only in that process’s memory and are lost if it crashes.
- No cross-host delivery: only code inside the same process can read the channel.
- No durable retry: reading an item removes it from the channel, and nothing redelivers it if processing then fails.
Work that must survive a crash or be retried durably needs a database, message broker or other persistent store.
What does a bounded
Channel<T>provide thatSemaphoreSlimdoes not?A bounded
Channel<T>stores the work itself and hands it to separate consumers, using a fixed buffer and an overload policy.SemaphoreSlimstores no work. It only counts permits, which limits how many callers run an operation at the same time.
Aspect Bounded Channel<T>SemaphoreSlimWhat it holds Up to capacityqueued itemsOnly a count of free permits Who does the work A producer writes an item and continues; a consumer reads and processes it later The caller that gets a permit runs the operation itself, then releases the permit At the limit In Waitmode,WriteAsyncwaits for buffer space, which slows producers; drop modes discard an item insteadWaitAsyncwaits for a permit; nothing is stored or droppedOrder Items are read in the order they were queued; with several consumers, they can finish in a different order Waiting callers have no guaranteed order, such as FIFO, for receiving permits A channel therefore fits producer-consumer handoff, such as a request queuing a job for a background worker. A semaphore fits admission control, such as capping concurrent calls to a dependency.
Why is
Channel.CreateUnbounded<T>()a risky default?An unbounded channel does not apply capacity backpressure. If producers continue to outpace consumers, queued items accumulate and consume memory until the process is pressured. A bounded channel forces an explicit overload choice: wait for capacity or drop according to a chosen policy. Microsoft: Channels
When is
BoundedChannelFullMode.DropOldesta reasonable policy?
DropOldestis reasonable when each newer item makes older buffered items obsolete, so only the latest values matter. Examples are progress snapshots, current status updates and sampled metrics.When the buffer is full, the channel removes the oldest buffered item and accepts the incoming one. The write still succeeds, so the producer is not slowed, and the write result does not report the discard; only an
itemDroppedcallback, when configured for the channel, signals that an item was discarded. Losing a stale snapshot is harmless when a newer one is already queued, but counting or logging drops keeps unexpected loss visible.It is the wrong policy for independent jobs, commands or business events that must each be processed. Dropping one of those silently loses work. The default
Waitmode instead makes producers wait for space and discards nothing.
What is the difference between concurrency and parallelism in practice?
Concurrency means several operations are in progress during the same period, even if only one runs at any instant. Parallelism means several operations run at the same instant on different cores. Parallel execution is always concurrent, but concurrent work need not be parallel.
Aspect Concurrency Parallelism What it describes How work is structured: operations overlap in time, and their steps can interleave How work executes: operations run simultaneously Hardware required Possible on a single thread or core Requires multiple cores Typical .NET use Asynchronous I/O: a thread starts an operation, yields at awaitand runs other work while the operating system waitsCPU-bound work split across cores, such as Parallel.Forover independent itemsWhat it improves Progress continues during waits, without a thread held for each wait Elapsed time of computation that divides into independent parts For example, one thread can start two HTTP requests, yield at each
awaitand resume each request when its response arrives. Both requests are in flight together, yet the thread runs one continuation at a time. Extra threads would not make the remote service respond sooner. Parallelism pays off only when there is enough independent CPU work to repay the cost of partitioning and scheduling it.Microsoft: Asynchronous programming · Microsoft: Parallel programming in .NET · Microsoft: Task-based asynchronous programming
What should be checked before choosing
Task,lock,Parallel, orChannel?Three checks drive the choice: whether the work waits on I/O or uses CPU, whether threads share mutable state, and which code owns completion, cancellation and failure.
Check Fits Reason The work mostly waits on I/O TaskwithawaitThe thread is released while the operation waits; more threads would not make the I/O finish sooner The work is independent CPU computation large enough to split Parallelwith a measured degree of parallelismSeveral cores compute at once, but small or contended work may not repay partitioning and scheduling Threads change shared mutable state in a short synchronous step lockOnly one thread at a time performs the change; a lockcannot be held acrossawaitProducers hand work to separate consumers Bounded Channel<T>It buffers items and fixes an overload policy: producers wait or items are dropped Ownership applies to every option. Some code must await or otherwise observe each task so that its exceptions surface. That code also passes a cancellation token into the operation and, for a channel, calls
Completeafter the last write. Without such an owner, a task can fail unnoticed and a channel reader can wait forever for items that will never arrive.Microsoft: Asynchronous programming · Microsoft: Task-based asynchronous programming · Microsoft: Parallel programming in .NET · Microsoft:
lockstatement · Microsoft: Channels
What are the four Coffman conditions for a resource deadlock?
The four conditions are mutual exclusion, hold and wait, no preemption and circular wait. A classical resource deadlock occurs only when all four hold at the same time.
- Mutual exclusion: a resource can be held by only one participant at a time, as with a lock.
- Hold and wait: a participant keeps the resources it holds while waiting for another.
- No preemption: a resource cannot be taken from its holder; only the holder can release it.
- Circular wait: the waiting participants form a closed chain, each waiting for a resource held by the next.
For example, thread T1 holds lock A and waits for lock B, while T2 holds B and waits for A. Both locks are exclusive, each thread holds one while waiting, neither lock can be taken away, and the two waits close a cycle.
Because all four conditions are required, breaking any one prevents this kind of deadlock. Which condition is practical to break depends on the resources and design. For locks, a single global acquisition order that every code path follows makes circular wait impossible.
Coffman, Elphick and Shoshani: System Deadlocks · Microsoft: Dynamic-link library best practices
How can lock-based deadlocks be prevented in .NET code?
Lock-based deadlocks are prevented by making a closed wait cycle impossible, or by ensuring that a thread does not wait forever while it holds locks.
- One global lock order: every code path that needs several locks acquires them in the same order, such as A before B. No thread can then hold B while waiting for A, so circular wait cannot form.
- Bounded acquisition: a timed
Monitor.TryEnterlets a thread give up instead of waiting forever. On failure, the thread releases any locks it already holds, then reports an error or retries after a randomized delay. The timeout does not take the lock from its owner, and identical retry timing could make two threads keep colliding (livelock).- Short critical sections: I/O, blocking waits and calls into unknown library code stay outside the lock. Code called while a lock is held may take its own locks, which adds an ordering dependency that is hard to see.
- Private lock objects: a
private readonlylock object keeps the lock under the component’s control. Locking onthis, aTypeor a string lets unrelated code take the same lock and join the ordering.For example, if
Firstlocks A then B whileSecondlocks B then A, two threads can each hold one lock and wait for the other. ChangingSecondto lock A then B removes the cycle.Microsoft: Dynamic-link library best practices · Microsoft:
lockstatement
When does blocking on an incomplete task deadlock a UI thread?
Blocking deadlocks a UI thread when the task cannot complete without running code on that same UI thread. This happens when an
awaitinside the task captured the UISynchronizationContext, which runs continuations only on the UI thread.The cycle forms in five steps:
- A UI event handler calls an async method, such as
LoadDataAsync().- Inside it,
awaiton incomplete I/O captures the UI context, and the method returns an incomplete task.- The handler calls
.Resultor.Wait()on that task, blocking the UI thread.- The I/O completes, and the rest of
LoadDataAsync()is queued to run on the UI thread.- The UI thread waits for the task, while the task waits for the UI thread. Neither can proceed.
Awaiting through the whole call chain breaks the cycle, because the handler yields the UI thread instead of blocking it.
ConfigureAwait(false)in library code avoids capturing the context for those awaits, but it does not make blocking on asynchronous code a safe general bridge.Without a single-threaded context, as in ASP.NET Core, this particular cycle does not form. Blocking still holds a thread-pool thread for each waiting call and can starve the pool under load.
Microsoft: Common async bugs · Microsoft: Synchronous wrappers for async methods
How can a production .NET deadlock be diagnosed?
A production deadlock is diagnosed from a process dump by finding a closed wait-for cycle, in which each blocked thread or task waits for something that another member of the cycle holds or must run.
- Dump:
dotnet-dump collectsaves the memory and thread state of the hung process.- Blocked threads: in
dotnet-dump analyze,clrstack -allshows every managed stack. Deadlocked threads sit in lock acquisitions such asMonitor.Enter, or in synchronous task waits such as.Resultor.Wait().- Monitor owners:
syncblklists which thread owns each monitor. Matching each waiting thread to the lock it wants, and that lock to its owner, shows whether the waits close a cycle.- Async waits: for a thread blocked on a task, the analysis also needs the task’s pending continuation and the context or scheduler it must run on. For example, a continuation bound to the UI context cannot run while the UI thread itself is blocked on that task.
The cycle is the evidence. A high thread count, or many blocked threads without a cycle, points to a different problem such as thread-pool starvation.
What is a race condition, and how does
lockprevent it?A race condition occurs when the result depends on the timing of threads that access shared state concurrently.
lockprevents it by letting only one thread at a time run code protected by the same lock object, so one protected update cannot interleave with another.For example, two threads run
_count++on a shared field. The increment is three steps: read the value, add one and write the result back.
- Thread A reads 5.
- Thread B reads 5 before A writes.
- A writes 6.
- B writes 6.
Two increments ran, but one was lost. With
lock (_gate) { _count++; }in both threads, B cannot read until A has written and released the lock, so the result is 7.The protection covers only code that takes the same lock. Every read and write of the shared state must use that lock object; an unprotected access elsewhere can still interleave with a protected update.
Microsoft: Managed threading best practices · Microsoft:
lockstatement
When
locktargets an ordinary object, what does the compiler ensure around the critical section?The compiler acquires the monitor with
Monitor.Enter(obj, ref lockTaken), runs the body insidetry, and exits infinallyonly if acquisition succeeded. This releases ownership on normal and exceptional exits without exiting a monitor the thread never acquired. The specializedSystem.Threading.Lockpath in C# 13 usesEnterScopeinstead. Microsoft:lockstatement · Monitor.Enter
Why can’t a
lockblock containawait?
Monitorownership belongs to the thread that entered it, and only that thread can exit it. An awaited continuation may resume on a different thread, which would then try to release a monitor it does not own. The C# compiler therefore rejectsawaitinside alockblock.Holding a lock across
awaitwould also make other threads wait for the whole asynchronous operation, turning I/O latency into lock contention.Two alternatives remain:
- Restructure: the awaited work runs outside the lock, and only the resulting state change runs inside a short
lock. This fits most cases.- Async-aware gate: when an invariant must stay protected across an
await, aSemaphoreSlimwith one permit andawait WaitAsync()provides the gate. It has no thread ownership, so a continuation on another thread can release it.Release()belongs infinally.Microsoft:
lockstatement · Monitor.Enter · SemaphoreSlim API
What does
System.Threading.Lockchange aboutlockin C# 13 and .NET 9+?
System.Threading.Lockgives the compiler a dedicated synchronization type instead of an arbitrary object. In C# 13,lockon that type lowers to anEnterScope()guard that releases when the scope exits, rather than the ordinaryMonitor.Enter/Exitpattern. It remains synchronous and cannot spanawait; the type does not promise a universal speedup. Microsoft:lockstatement
How does reentrancy differ between
lock/MonitorandSemaphoreSlim?
lock/Monitoris reentrant for its owning thread.SemaphoreSlimis not reentrant, because it has no owner.
lock/Monitor: records which thread owns the monitor and how many times it has entered. The owner can enter again without blocking, and other threads can acquire the monitor only after the owner has exited the same number of times.SemaphoreSlim: records only how many permits remain, not which thread or asynchronous flow holds them. A second wait from the same code is just another request for a permit.With a one-permit
SemaphoreSlim, a method that holds the permit and then calls code that waits on the same semaphore ends up waiting for itself. Without a timeout or cancellation,Wait()blocks forever andWaitAsync()never completes. The permit therefore belongs at one boundary in the call chain, withRelease()infinallyso that each acquisition is released exactly once.Microsoft:
lockstatement · Monitor.Enter · SemaphoreSlim API
When is a named
Mutexappropriate in .NET?A named
Mutexfits when separate processes on the same machine must not run a critical section at the same time. Examples are a single-instance application guard and several processes writing one local file. Every process that opens the same name shares one operating-system mutex, and only one thread across those processes can own it at a time.
- Thread ownership: the thread that acquires the mutex must release it with
ReleaseMutex; a call from any other thread throws. Acquisition and release therefore stay in one synchronous scope, not around anawait.- Inside one process:
lockfor synchronous code, orSemaphoreSlimfor asynchronous code, provides exclusion without an operating-system mutex.- Across machines: a named mutex cannot coordinate them. Ownership across machines needs a lease or lock managed by a shared database or coordination service.
- Platform: name visibility and access rules depend on the operating system. On Windows, the
Global\andLocal\prefixes control whether other Terminal Services sessions see the name.
What does
AbandonedMutexExceptionsignal?The previous owner thread terminated without releasing the mutex. The waiting thread receives
AbandonedMutexExceptionbut acquires ownership, so it must still release the mutex. The protected state may have been left halfway through an update and should be validated or recovered before other threads enter. Microsoft: Mutex class · Microsoft: Mutexes
Why can adding more parallel workers reduce performance?
Extra workers help only while there is independent work for them and spare capacity to run it. Past that point, each added worker contributes more overhead and contention than useful work, so throughput can fall and latency can rise.
- More CPU-bound workers than cores: the extra workers cannot run at the same time. The operating system time-slices them, which adds switching overhead without adding compute.
- Contention on shared resources: locks, shared writes and thread-safe collections make workers wait for one another. Memory bandwidth and caches are shared as well, so memory-heavy workers slow each other down.
- Coordination overhead: partitioning the input, scheduling the pieces and merging the results cost more as work is spread across more workers.
- Downstream limits: when each worker calls a database or remote service, extra workers queue on that dependency’s connections or rate limits, adding waiting and possibly timeouts.
Worker count is therefore a value to measure against the real workload, not a direct measure of speed.
Microsoft: Potential pitfalls with PLINQ · Microsoft: Understanding speedup in PLINQ
How should
MaxDegreeOfParallelismbe chosen?The limit should come from the resource that actually constrains the work and be confirmed by measurement under representative load.
Environment.ProcessorCountis a reasonable starting point for CPU-bound work, not a universal default.
- CPU-bound work: values near the core count are a starting point. Higher values add time-slicing overhead without adding compute.
- Memory- or cache-heavy work: memory bandwidth can saturate before every core is busy, so the best value can be lower than the core count.
- Work that waits on I/O or a dependency: the limit follows what the database or remote service can handle, such as its connection pool or rate limits. It can be higher or lower than the core count.
The chosen value is the one that meets throughput and tail-latency goals without exhausting those resources.
MaxDegreeOfParallelismis only an upper bound on concurrent operations. It does not force that many to run.Microsoft: ParallelOptions.MaxDegreeOfParallelism · Microsoft: Introduction to PLINQ · Microsoft: Understanding speedup in PLINQ
When is PLINQ a poor fit?
PLINQ is a poor fit when the query’s work is not independent and large enough per element, or when running its delegates concurrently would break correctness.
- Shared mutable state: delegates that write to shared variables or call non-thread-safe code can corrupt data. Locks or thread-safe calls that prevent this make workers wait on each other and can erase the speedup.
- Order-dependent side effects: elements are processed on several threads, so side effects happen in no predictable order.
AsOrdered()only preserves the order of the query’s results, at extra merge cost.- Thread-affine code: work that must run on a specific thread, such as Windows Forms or WPF UI access or single-threaded-apartment COM objects, cannot safely run inside PLINQ delegates.
- Too little work per element: partitioning and merging overhead can outweigh the parallel gain.
PLINQ fits pure, CPU-bound transforms over in-memory data, where each element does enough work and measurement shows a speedup.
Microsoft: Potential pitfalls with PLINQ · Microsoft: Introduction to PLINQ · Microsoft: Understanding speedup in PLINQ
Why can a parallel query be slower than a sequential query for small inputs?
Parallel execution pays to partition input, schedule work, and merge results. With few items or little work per item, the sequential loop can finish before those fixed costs are repaid. Parallel execution is useful only when independent work is large enough to outweigh the setup and coordination. Microsoft: Understanding speedup in PLINQ
When is
SemaphoreSlima better fit thanlock?
SemaphoreSlimis the better fit when waiting must be asynchronous, such as protected work that containsawait, or when up to N callers may proceed at once.lockremains the simpler fit for a short synchronous section that one thread at a time may enter.
Aspect lockSemaphoreSlimCallers inside at once One thread, so it only provides mutual exclusion. Up to the configured count. A count of 1 gives mutual exclusion, and a higher count throttles concurrency. How a caller waits The waiting thread blocks until the lock is free. WaitAsyncreturns a task, so the caller waits without holding a thread.Waitblocks like a lock.awaitinside the sectionNot allowed by the compiler, because the lock belongs to the thread that entered it. Allowed, because a permit is not tied to any thread. Release Automatic when the block exits, including after an exception. Only through an explicit Releasecall, which any code can make.Because a semaphore tracks no owner, each successful wait is paired with exactly one
Release, normally in afinallyblock. A missed release permanently removes a permit. An extra release adds a permit beyond the intended limit. AmaxCountdoes not reliably prevent this:ReleasethrowsSemaphoreFullExceptiononly when it would raise the count above that maximum, so an extra release made while permits are still held succeeds silently.Microsoft: SemaphoreSlim · Microsoft: SemaphoreSlim.Release · Microsoft: C# lock statement
Why is
Tasknot equivalent to a thread?A
Taskis an object that represents an operation and its eventual outcome: a result, a failure or cancellation. A thread is an execution resource that runs code. A task uses a thread only while code belonging to it is actually running.
- CPU-bound tasks: with the default scheduler, which
Task.Runuses, their delegates are queued to the ThreadPool, where a limited set of reusable threads runs many tasks over time instead of one thread per task. A customTaskSchedulerdecides for itself where its tasks run, and theLongRunninghint typically leads the default scheduler to use a separate non-pool thread.- I/O-bound tasks: while the operating system performs the I/O, the pending task holds no managed thread. A thread is needed only to start the operation and to run the code that processes its completion.
Creating many tasks therefore does not mean one thread per task, and the number of pending tasks does not show how many threads are busy.
Microsoft: Task-based asynchronous programming · Microsoft: TaskScheduler · Microsoft: TAP I/O-bound tasks
Why does awaiting
Task.Runinside an ASP.NET Core request usually not help?ASP.NET Core already runs request code on ThreadPool threads. When the request awaits
Task.Run, its own thread returns to the pool while another pool thread runs the work. The request therefore gains no CPU capacity and frees no net thread. It only adds a scheduling hop.
- Blocking work: inside
Task.Runit still blocks a pool thread, just a different one. Synchronous I/O does not become asynchronous I/O.- Already asynchronous APIs: awaiting them directly gives the same non-blocking wait without the extra hop.
- Long-running or CPU-heavy work: it competes with other requests for the same pool threads. It belongs outside the request path, for example in a background service or an out-of-process worker fed by a queue, with the result reported to the client asynchronously.
Microsoft: ASP.NET Core best practices · Microsoft: Task-based asynchronous programming
Why can
Task.WhenAllreduce elapsed time for independent asynchronous calls?The time saving comes from starting the calls before waiting, so their waits overlap. Awaiting each call in turn makes the total roughly the sum of the call times. Starting all of them first and then awaiting
Task.WhenAllmakes the total approach the slowest call.// Sequential: the second call starts only after the first finishes var user = await GetUserAsync(id); var orders = await GetOrdersAsync(id); // Overlapped: both calls are in flight before the wait var userTask = GetUserAsync(id); var ordersTask = GetOrdersAsync(id); await Task.WhenAll(userTask, ordersTask);
Task.WhenAllitself only returns a task that completes when every supplied task has completed. It does not start tasks that were never started, and it does not make synchronous work run in parallel. Awaiting the already-started tasks one by one would also overlap them.WhenAlladds one wait for all of them that also records every failure.The gain also requires the calls to be independent and the downstream service to handle them concurrently. If it queues or throttles them, the total moves back toward the sum.
Microsoft: Task.WhenAll · Microsoft: Task-based asynchronous programming
How does
Task.WhenAllreport failures when several of its tasks fail?The task returned by
Task.WhenAllcompletes only after every supplied task has finished. If any of them faulted, it ends faulted, and itsExceptionholds anAggregateExceptionwith the failures from all faulted tasks. Awaiting it, however, rethrows only one of those exceptions.
- The other failures: after
awaitthrows, the full set is still available on the combined task’sException.InnerExceptionsand on each input task’s ownException.- No early exit: one failure does not stop or cancel the remaining tasks, and
WhenAllstill waits for all of them.- Cancellation: when no task faulted but at least one was canceled, the combined task ends canceled.
Task all = Task.WhenAll(a, b, c); try { await all; } catch (Exception) { // await rethrew one failure; the combined task keeps all of them var failures = all.Exception?.InnerExceptions; Log(failures); throw; }
When is
ValueTaskworth using instead ofTask?
ValueTaskis worth it only when profiling shows thatTaskallocations on a hot path matter. The typical case is a method called very often that usually completes synchronously, such as a read served from an internal buffer.Taskremains the default.
- What it saves: when the result is already available,
ValueTask<T>carries it directly instead of allocating aTask<T>. It can also wrap a reusableIValueTaskSource, so some asynchronous completions avoid an allocation too.- What it costs: the struct holds more fields than a single
Taskreference, so returning and copying it costs more. It also has stricter consumption rules: it is awaited only once, never awaited concurrently, and itsResultis not read before it completes. A pooled source can be reused for another operation after the first consumption, so breaking these rules gives undefined behavior, such as reading another operation’s result.A caller that must store, share or await the result more than once calls
AsTask()once and uses the resultingTask.
What is the .NET ThreadPool for, and what work does it not suit?
The ThreadPool is a per-process set of reusable background worker threads that runs short pieces of queued work, such as Task Parallel Library tasks, timer callbacks and asynchronous I/O completions. Reusing threads avoids creating and destroying a thread for each small job, and the runtime adjusts the number of workers to optimize throughput.
Pool threads are shared and interchangeable, so the pool does not suit work that holds a thread for a long time or needs control over the thread:
- Long blocking work: the pool has a maximum size and, above its minimum, may wait before adding threads. Many blocked workers can keep queued work from starting, which is ThreadPool starvation.
- Thread-specific needs: a foreground thread that keeps the process alive, a particular priority, a single-threaded apartment for COM, or a stable thread identity.
- A long-lived dedicated job: a component that owns a loop for the process’s lifetime fits a dedicated
Threadbetter than a borrowed pool worker.
What is ThreadPool starvation, and how does it usually start?
ThreadPool starvation is a state in which queued work items and continuations wait because no pool worker is available to run them. It usually starts when workers are blocked rather than busy, through synchronous blocking on pool threads: most often sync-over-async calls such as
.Resultor.Wait()on a task, or synchronous I/O in request code.A typical progression:
- Many pool threads each block, waiting for an asynchronous operation to finish.
- When an operation’s I/O completes, its continuation, the code that finishes the operation, is queued to the ThreadPool.
- All workers are blocked, so that continuation stays queued while the blocked workers keep waiting for it.
- Above its minimum, the pool may wait before adding workers, so the queue can grow faster than new workers arrive.
In this blocking-driven case, the typical symptoms are a growing queue and rising latency or timeouts while CPU stays well below saturation. That pattern separates it from CPU saturation, where every core is busy with real work and more threads do not help.
Microsoft: Debug ThreadPool starvation · Microsoft: The managed thread pool
When is it appropriate to call
ThreadPool.SetMinThreads?Raising the minimum is appropriate only after measurements show that the pool’s gradual thread injection is delaying work. That covers a burst of work arriving faster than the pool adds workers. It also covers blocking-related starvation, where a higher minimum can serve as a temporary mitigation while the blocking code is fixed. It is a late tuning step: it does not remove the cause of blocking, and it does not help when the CPU is already saturated.
- What it changes: the minimum is the number of workers the pool creates on demand without its usual injection delay. It does not create those threads in advance, and an idle pool can stay below it.
- What it costs: the pool can create many threads quickly, which adds memory use, contention and context switching, and can oversubscribe the CPU.
- What it does not fix: when workers block on synchronous waits, extra threads can relieve the backlog for a while, but the blocking remains the cause, and enough load can block the new threads too. When all cores are already busy, extra threads add switching overhead rather than throughput.
Queue length, thread count, CPU and latency compared before and after the change show whether it helped.
Microsoft: ThreadPool.SetMinThreads · Microsoft: The managed thread pool
When should a .NET method throw an exception instead of returning a failure result?
A method throws when it cannot deliver what it promises and callers are not expected to branch on that outcome in normal flow. It returns a failure result when the outcome is common and expected, so callers routinely check it.
Aspect Throwing an exception Returning a failure result Fits when The failure is unexpected for this call, such as an invalid argument, an invalid object state or an unavailable dependency. The outcome is part of normal flow, such as a cache miss, unparsable user input or a rejected validation. If the caller ignores it A synchronous throw propagates up the call stack until a handler catches it, so the caller cannot skip it by not checking. A Task-returning async method stores the exception on thatTask; it surfaces only when the code that owns the task awaits or otherwise observes it, so an ignored faulted task can go unnoticed.The failure can be silently ignored if the caller never checks the result. Cost when it happens Throwing and unwinding are expensive, which matters when the outcome occurs on every request. Reporting the outcome costs about the same as success. Typical .NET shape A specific exception type such as ArgumentExceptionorInvalidOperationException.A Trymethod such asint.TryParsereturningbool,nullor default for a common miss, or an explicit result type.Many APIs offer both forms:
int.Parsethrows on bad input, whileint.TryParsereturnsfalse. The caller then picks the form that matches how often failure is expected.Microsoft: Best practices for exceptions · Microsoft: TaskScheduler.UnobservedTaskException
What is the difference between
throw;andthrow ex;inside acatchblock?
throw;rethrows the current exception while preserving its original stack trace.throw ex;starts the visible stack at the rethrow site and loses the original path. Microsoft: exception handling statements
When should an exception be wrapped instead of rethrown directly?
An exception should be wrapped when a boundary must add meaning that the original lacks: context about the operation that failed, or translation into a stable failure type that callers of that layer are expected to handle. The original goes into the new exception’s
InnerException, so its type, message and stack trace stay available.For example, a data-access layer can translate a database provider’s exception into a domain-level
OrderStoreUnavailableException. Callers then handle one stable type and do not depend on the provider.When the catch block only logs, cleans up or decides it cannot handle the failure,
throw;rethrows the original with its stack trace intact. Wrapping without adding meaning only lengthens the exception chain. It can also change which outer handlers match. Acatchmatches its declared type and types derived from it, so a handler that caught the original stops matching when the wrapper’s type is neither that type nor derived from it.Microsoft: Best practices for exceptions · Microsoft: Exception-handling statements
Why is throwing from
finallydangerous?An exception that escapes
finallywhile another exception is already propagating replaces that exception. The original stops propagating, so callers, outer handlers and any later logging see only the cleanup error, which usually says little about the real cause.For example, a
tryblock fails with a timeout, and closing a connection infinallythen throws an exception that escapes the block. The caller receives the close failure, and the timeout no longer appears in the propagated exception.Cleanup in
finallyis therefore kept simple, and any failure it can raise is caught and logged inside thefinallyblock so the original exception keeps propagating.Microsoft: Exception-handling statements · Microsoft: Best practices for exceptions
When might
finallynot execute?
finallyruns when control leaves itstryblock while the process keeps executing: normal completion, a jump such asreturnorbreak, or an exception unwinding through it. It does not run when the process ends without unwinding or when control never leaves thetryblock.
- Immediate process termination:
Environment.FailFastends the process without running pendingfinallyblocks or finalizers. A stack overflow, or the process being killed by the operating system or an orchestrator, ends it just as abruptly.- Unhandled exceptions: cleanup is reliable when a handler exists and the stack unwinds to it. If no handler is found, whether
finallyblocks run depends on how the runtime and operating system end the process.- Control never leaves the
tryblock: an infinite loop or a deadlock inside it meansfinallyis never reached.
finallyis therefore in-process cleanup, not crash recovery. State that must survive a crash relies on durable mechanisms such as transactions or recovery logic that runs on restart.Microsoft: Exception-handling statements · Microsoft: Environment.FailFast
What must a type provide to work with
foreach?A type works with
foreachin either of two ways. It implementsIEnumerable<T>orIEnumerable, or it matches the enumerator pattern that the compiler looks for. The interface is the usual reusable contract, but it is not required.The pattern has two parts:
- The source has a public parameterless
GetEnumerator()method. That method can also be an extension method.- The returned enumerator has a public
Currentproperty and a public parameterlessMoveNext()method that returnsbool.The compiler binds to these members rather than to an interface, so a type can be iterated without implementing either interface. Arrays also receive specialized compiler handling.
How does
foreachwork under the hood?The compiler rewrites
foreachinto lower-level code, and the result depends on the source type. For the general enumerator pattern, the rewrite is roughly:var enumerator = source.GetEnumerator(); try { while (enumerator.MoveNext()) { var item = enumerator.Current; // loop body } } finally { // Disposes the enumerator when it requires disposal. }
GetEnumerator()is called once, before the first element.- Each
MoveNext()call advances to the next element and returnsfalsewhen the sequence is exhausted.Currentsupplies that element, which is assigned to the iteration variable before the body runs.- When the enumerator requires disposal, the
finallyblock disposes it, including afterbreakor an exception. A handwrittenwhile (MoveNext())loop has to provide that cleanup itself, or the enumerator’s cleanup does not run.No single generated shape fits every source. Arrays, for example, can be lowered to index-based code without an enumerator.
When is
yield returnuseful compared with a materialized collection?
yield returnproduces values on demand, so a consumer can stop early without first building the whole result. The iterator body runs during enumeration; enumerating again runs it again and may observe changed state. A materialized collection is preferable when a stable snapshot or repeated traversal is needed. Microsoft: yield statement
What do generics add to a C# API?
Generics let one implementation take a type as a parameter. The same code then works with many types, while the compiler still checks the type of every value that goes in or comes out.
Compared with an API that accepts
object:
- Type safety: an
object-based API needs casts on the way out, so a wrong type fails only at runtime. AList<string>instead rejects anintat compile time and returns strings without casts.- No storage boxing for value types: an
object-based collection boxes eachintit stores, whileList<int>stores the integers directly. Generic code can still box a value-typeTthrough conversions, such as converting it toobjector to an interface type.Constraints such as
where T : IComparable<T>narrow which type arguments are allowed. In return, the generic code may use the members that the constraint guarantees, such asCompareTo.Microsoft: generic types and methods · Microsoft: constraints on type parameters · C# specification: boxing conversions
Why does
IEnumerable<string>assign toIEnumerable<object>, butList<string>does not assign toList<object>?
IEnumerable<T>declares its type parameter covariant (out T), so the conversion is allowed.List<T>is invariant, so it is not.
IEnumerable<string>asIEnumerable<object>: the interface only handsTvalues out. Every string read from the sequence is also an object, so reading strings as objects is safe.List<string>asList<object>:List<T>also takesTvalues in, for example throughAdd. If the conversion were allowed, code could add anintthrough theList<object>reference, and the originalList<string>would then hold a non-string.Variance also has to be declared, and C# allows it only on generic interfaces and delegates. A generic class such as
List<T>is therefore always invariant, whatever its members do.
When should a generic interface type parameter be marked
outorin?A type parameter can be marked
outwhen the interface only producesT, andinwhen it only consumesT. WhenTflows both in and out, it stays invariant.
Marking Where Tmay appearConversion it enables out(covariant)Only in output positions, such as method return types A more derived type argument can stand in for a less derived one: IEnumerable<string>converts toIEnumerable<object>in(contravariant)Only in input positions, such as method parameter types A less derived type argument can stand in for a more derived one: IComparer<object>converts toIComparer<string>None (invariant) In both input and output positions No conversion, because either direction would allow an unsafe read or write The compiler enforces these positions, so it accepts a variant marking only when the interface’s members keep it safe.
Variance conversions work only when the type arguments are reference types.
IEnumerable<int>does not convert toIEnumerable<object>, because a value-type argument makes the constructed type invariant.
A generic method uses
default(T)as a fallback. Why can this be dangerous?
default(T)is not one fixed value. It isnullwhenTis a reference type, and a zero-initialized value such as0orfalsewhenTis a value type. Those values can also be legitimate data, so returningdefault(T)to signal that nothing was found or that something failed makes absence look like a real result.For example, a
GetOrDefault<T>method that returnsdefault(T)for a missing key returns0whenTisint. A caller then cannot tell a missing entry from a stored0.When that distinction matters, the API reports absence separately from the value:
Trymethod: returnsboolfor success and the value through anout Tparameter.- Result type: carries the success or failure state together with the value.
- Nullable contract: returns
T?, withnullmeaning absence. This works only whennullis never legitimate data and the return type can actually holdnullfor the type argument. For a value type that requiresNullable<T>, for exampleT?under awhere T : structconstraint, whoseHasValueseparates a missing entry from a stored0. With an unconstrainedT,T?adds no null state for a value-type argument, soGetOrDefault<int>would still return0.Microsoft: default keyword · Microsoft: nullable value types
How do
ref,out, andinparameters differ?All three pass the argument by reference instead of copying its value. For
refandout, the parameter always aliases the caller’s variable. Forin, it aliases the caller’s variable when the argument is a variable of the parameter’s type. Aninargument that is not a variable, or that needs a conversion, can instead be copied into a temporary, and the parameter then refers to that temporary. The three modifiers differ in which direction data may flow through the reference.
Modifier How data flows through the reference Typical use refBoth ways. The caller must initialize the variable, and the method can read it and assign it. The caller sees any assignment Updating or replacing the caller’s variable outOutward. The caller can pass an unassigned variable. The method cannot read it before assigning it and must assign it before every normal return Returning an extra result, as in Trymethods such asDictionary.TryGetValueinInward and readonly. The caller must initialize the argument, and the method can read the parameter but cannot assign it Avoiding copies of a large struct on a measured hot path Without a modifier, the parameter is a copy, so assigning it never changes the caller’s variable.
Why would a method need
refwhen the argument is already a reference type?A reference-type argument passes a copy of the reference. The method can mutate the shared object through that copy, but assigning a different object changes only its local copy.
refaliases the caller’s variable, allowing the method to replace the reference stored there. Microsoft: method parameters
What does an
inparameter do, and when is it useful?An
inparameter passes a value by readonly reference, so the method cannot assign through that parameter. It may avoid copying a large struct in a measured hot path, but conversions and non-variable arguments can still require a temporary. Microsoft: method parameters
How does C# apply an optional parameter's default value, and why can already-compiled callers keep an old default?
The default is applied at the call site. When an ordinary compiled call omits an optional argument, the compiler inserts the declared default value into the caller’s code, and the method receives it like any other argument.
The declaration rules fit that mechanism:
- The default must be a compile-time value: a constant expression,
default(S), ornew S()whereSis a value type. The compiler has to be able to write the value into each call.- Optional parameters come after all required parameters. Named arguments let a call supply a later optional argument while omitting earlier ones.
Versioning consequence: the default lives in each caller’s compiled code. If a library changes a default, callers compiled against the old version keep passing the old value until they are recompiled. A public default value is therefore part of the library’s compatibility contract.
Microsoft: named and optional arguments · C#: 12.6.2.3 Run-time evaluation of argument lists
When
Animal a = new Dog();callsa.Category(), what makes the call runDog's implementation, and what does that commit theAnimalAPI to?
Animal.Category()must be declaredvirtual, andDogmust implementCategory()withoverride. The call is then resolved by the runtime type of the object,Dog, rather than by the declared type of the variable,Animal.Methods are non-virtual by default. A non-virtual
a.Category()binds toAnimal.Category()becauseais declared asAnimal, even whenDogdeclares its ownCategory()withnew.Commitment for the base API: a virtual member is an extension point. Derived classes can replace its behavior, unless an override along the hierarchy is marked
sealed, which stops classes below it from overriding the member again. Ordinary calls to the member on such an object run the most-derived override, including calls made from code insideAnimal. An explicitbase.Category()call in a derived class is the exception: it runs the base class’s implementation rather than the most-derived override.Animalcan therefore no longer assume that its own implementation of that member is the one that runs.Microsoft: virtual keyword · Microsoft: new modifier · Microsoft: base keyword · Microsoft: sealed modifier
A base method is not virtual, but a derived class needs different behavior. What are the options, and which ones work through a base-class reference?
Only virtual dispatch makes a call through a base-class reference run derived code. Making the base member
virtualand overriding it is therefore the only option that works through such a reference. Hiding withnewaffects only derived-typed references, and composition moves the variation out of inheritance.
Option Call through a base reference Fits when Base member made virtual, derived member markedoverrideRuns the derived implementation, chosen by the object’s runtime type The base type can be changed and derived objects are meant to substitute for base objects Derived member declared with new(hiding)Still runs the base implementation. Only references typed as the derived class reach the new member Hiding itself is the intended behavior, such as a deliberate compatibility split, not polymorphism Composition No inheritance dispatch is involved. The class delegates the varying step to an object it holds The base cannot be changed, or the variation is not a natural subtype relationship Hiding is the common trap. The same object behaves differently depending on whether it is referenced as the derived type or as the base type.
What do a C# namespace, an assembly, and an access modifier control?
A namespace organizes qualified names, an assembly is a compiled unit used for deployment and loading, and access modifiers control which code can see a type or member. These are separate boundaries: sharing a namespace does not place types in one assembly or grant access. Microsoft: namespaces · Microsoft: assemblies · Microsoft: access modifiers
What is reflection in .NET, and what is it used for?
Reflection is the ability of running code to inspect .NET metadata, such as assemblies, types, members and attributes, and to use that metadata to create objects or call members that the calling code does not name at compile time.
It works in two phases:
- Inspect: code obtains a
Type, for example throughtypeof(T), an object’sGetType()or a loaded assembly. Objects such asMethodInfo,PropertyInfoandConstructorInfothen describe its members, signatures and attributes.- Act (optional): code creates an instance of a discovered type or invokes a discovered member late-bound, for example with
Activator.CreateInstanceorMethodInfo.Invoke.Typical uses are reading custom attributes, examining and instantiating types from an assembly, such as plug-ins that implement a known interface, and late binding to members selected at runtime.
The trade-off is a weaker contract. A misspelled member name or an incompatible signature is discovered only when the code runs, not by the compiler.
Microsoft: reflection overview · Microsoft: attributes and reflection
Why is reflection often a poor default in performance-critical code?
Reflection replaces a call that the compiler binds once with work done while the program runs. A direct call is resolved during compilation. Reflection finds a member by inspecting metadata, often by name, and
MethodInfo.Invokethen receives its arguments as anobject[]and checks them against the member’s signature on each call.On a hot path, repeating that lookup and late-bound invocation adds work to every call. The cost depends on the operation and how often it runs, so there is no universal threshold.
When reflection is needed on a repeated path, the usual mitigation is to discover the member once, cache it, and bind it to a strongly typed delegate. Later calls then go through the delegate without per-call reflection. Measurement shows whether that setup cost pays off.
How do attributes become behavior at runtime?
An attribute does nothing by itself. The compiler stores it as metadata on the marked type or member, and it affects runtime behavior only when some code reads that metadata and acts on it.
At runtime the flow is:
- The attribute and its arguments were recorded in the assembly’s metadata when the code was compiled.
- A framework or the application inspects the type or member through reflection, for example with
GetCustomAttribute<T>().- The consumer acts on what it finds, such as registering a handler or selecting a method to invoke.
An attribute that no code reads has no runtime effect. A source generator is the build-time alternative. It reads attributes during compilation and emits ordinary code, so the behavior no longer depends on runtime reflection.
Microsoft: attributes and reflection · Microsoft: source generators overview
When do interfaces or generics, source generators, and reflection each fit best?
The deciding question is whether the code can depend on a compile-time contract or has to work from metadata. Interfaces and generics fit when the code can name the types or operations it needs. When the work is driven by metadata instead, timing separates the other two: a source generator reads metadata visible during compilation and emits ordinary code, while reflection reads metadata while the program runs. Reflection therefore covers both types that are unknown until runtime and runtime inspection of known types, such as reading their attributes.
Mechanism Fits when Consequence Interface or generic contract The calling code can state the types or operations it needs The compiler checks every use against the contract, and trimming can see what is used. An interface or virtual call can still select its implementation at runtime, but only among types that satisfy the compiled contract Source generator Repetitive code can be derived from types and attributes in the compilation, such as registration code for the project’s own attributed types The work moves to build time, and the generated code is compiled, checked and trimmed like handwritten code. Types that appear only at runtime, such as plug-ins loaded later, are invisible to it Reflection Code has to discover or read metadata at runtime, such as plug-ins loaded from a folder, members named by data, or attributes that a framework reads from known types It is the most flexible option, but mistakes surface at runtime and repeated lookup and invocation cost more. In trimmed deployments, which include Native AOT, members reached only dynamically can be removed unless annotations such as DynamicallyAccessedMembersor explicit registration preserve themMicrosoft: reflection overview · Microsoft: attributes and reflection · Microsoft: source generators overview · Microsoft: prepare libraries for trimming · C# specification: function member invocation
What is the difference between an abstract class and an interface with default members, and when is an abstract class a better fit?
An abstract class is a partial implementation that derived classes extend through single inheritance, and it can hold per-object state. An interface is a contract that a type can implement alongside any number of other interfaces. Default members let it supply behavior, but not instance state. An abstract class is the better fit when the derived types genuinely share instance state, construction rules or
protectedextension points.
Aspect Abstract class Interface with default members Inheritance limit A class can derive from only one base class, so an abstract base uses that single slot. A class or struct can implement many interfaces, so independent capabilities combine freely. Instance state Can declare fields and auto-implemented properties that every derived object carries. Cannot declare instance fields or auto-implemented properties, so default members can build only on other members, not on stored per-object data. Construction and hooks Can declare constructors and protectedmembers, so the base controls how derived objects are initialized and which extension points they get.Cannot declare instance constructors, so it cannot control how implementers are initialized. Coupling Derived classes depend on the base’s constructors, protected members and the order in which it calls them, so changes to the base can break them. Implementers depend only on the contract, which is easier to evolve when no shared state is needed. For example, a
Shapebase that stores aColorand declares an abstractArea()shares state across all shapes, so an abstract class fits. A capability such asIComparable<T>, implemented by otherwise unrelated types, fits an interface.Microsoft: abstract · Microsoft: interface · C# specification: classes
Why can’t a static class implement an interface?
A static class never has instances, but an interface’s instance members are always reached through an object. A variable, parameter or field of the interface type must refer to something that implements it. A static class can supply no such object, so C# forbids a static class declaration from listing a base class or any implemented interfaces.
As a result, behavior that callers must reach through an interface has to live on an instance type rather than in a static class. A test that substitutes a fake is one such caller.
Static abstract interface members, available from C# 11 and .NET 7, do not change this. A non-static class or struct implements them. Generic code calls them on a type parameter constrained to the interface, as in
T.Create()withwhere T : IFactory<T>, rather than through an instance.Microsoft: static · C# specification: classes (static classes) · Microsoft: static abstract interface members
A
sealed overridestops further overriding. Can a derived class usenewto hide it, and what happens at runtime?Yes.
newcan declare a separate hidden member, but it does not reopen the sealed virtual slot. A call through the derived type can select the hidden member by compile-time lookup; a call through the base type still dispatches to the sealed override. Microsoft: class types
How does a
partialtype let a source generator add code to a handwritten class?All declarations of a type marked
partialare parts of a single type, and at compile time the compiler combines every part into one type. A source generator adds a new source file to the compilation containing anotherpartialdeclaration of the class. The members, interfaces and attributes in that generated part become part of the handwritten class, while the handwritten file stays unchanged.// Order.cs (handwritten) public partial class Order { public int Id { get; set; } } // Order.g.cs (generated) public partial class Order : IValidatable { public bool Validate() => Id > 0; }Consequences of the parts forming one type:
- One member set: members declared in any part are available to all parts. If the handwritten and generated parts both declare
Validate()with the same signature, compilation fails with a duplicate-member error.- Combined declaration: the type implements the interfaces listed on every part, and
abstractorsealedon any part applies to the whole type.- Matching parts: every part must use
partial, have the same accessibility and be compiled into the same assembly and module.- Additive generation: the generator contributes new source instead of editing the handwritten file. Generated members are still part of the type’s behavior, so they need the same review and tests as handwritten members.
Microsoft: partial classes and members · Microsoft: partial types · Microsoft: source generators
When does the C# compiler remove a partial method and the calls to it?
The compiler removes a partial method, together with every call to it, when no part of the type supplies an implementing declaration and the method qualifies as optional. It qualifies only if its declaration has:
- no access modifier, not even an explicit
private;- a
voidreturn type;- no
outparameters;- none of the modifiers
virtual,override,sealed,neworextern.Partial methods gained access modifiers, non-
voidreturns andoutparameters in C# 9. Those extended forms, such aspublic partial bool Validate();, always require an implementation, so leaving it out is a compile error rather than a silent removal.Optional partial methods suit hooks in generated code. A generator can declare
partial void OnNameChanged();and call it, and the handwritten part may implement it. When no part implements it, the calls are removed at compile time, including evaluation of their argument expressions, so behavior expected from the hook never runs.Microsoft: partial members · C# specification: classes (partial methods)
When a class does not overload
==, why do two separately created instances with equal fields compare unequal, and how can value equality be added?For a class,
==without an overload compares references: it istrueonly when both operands refer to the same object. Two separately created instances are two objects, so they compare unequal even when every field matches.Equals(object)compares references in the same way only when neither the class nor any of its base classes overrides it, so the implementation inherited fromObjectapplies; the absence of an==overload alone does not guarantee this.Value equality is added by deciding which fields make two instances equal and applying that rule consistently:
- implement
IEquatable<T>.Equals(T)and overrideEquals(object)to compare those fields;- override
GetHashCodefrom the same fields, so equal instances produce equal hash codes and hash-based collections such asDictionary<TKey,TValue>andHashSet<T>treat them as the same key;- overload
==and!=to callEqualswhen operator comparison belongs to the type’s API. OverridingEqualsalone leaves==comparing references.A
record classgenerates all of these members, comparing the runtime type and every field, so it is the shorter choice when that generated equality matches the domain.
What is a delegate, and what does a delegate value carry?
A delegate is a type that represents references to methods with a particular signature. The delegate type fixes the parameter and return types. A delegate value packages a compatible method so it can be stored, passed and invoked later like any other value.
A delegate value carries:
- Method: the method to call, which must be compatible with the delegate’s signature.
- Target (optional): a delegate closed over an instance method also stores the object the method runs on. A delegate with no bound object, such as one created from an ordinary static method, has no target. Reflection through
Delegate.CreateDelegatecan also create open instance delegates, which receive the instance as an argument, and static delegates closed over their first argument.- Invocation list: a delegate can combine several methods, for example with
+=. Invoking it calls each in order.var tenPercent = new Discount(0.10m); PriceCalculator calc = tenPercent.Apply; // closed instance delegate: method Apply, target tenPercent decimal total = calc(3, 20m); // runs tenPercent.Apply: 54.00 public delegate decimal PriceCalculator(int quantity, decimal unitPrice); public sealed class Discount { private readonly decimal _rate; public Discount(decimal rate) => _rate = rate; public decimal Apply(int quantity, decimal unitPrice) => quantity * unitPrice * (1 - _rate); }Delegate values are immutable: adding or removing a method produces a new delegate. Lambdas and method groups convert to delegate types, which is how callbacks, LINQ operators and events pass behavior around.
Microsoft: delegates · C# specification: delegates · Microsoft: Delegate.CreateDelegate
When a multicast delegate is invoked, in what order do its methods run, and what happens to return values and exceptions?
Its methods run synchronously, one after another, in invocation-list order, which is the order in which they were combined. For a non-
voiddelegate, the caller receives only the last method’s return value. If any method throws, the exception propagates to the caller and the remaining methods are not invoked.Func<int> f = () => 1; f += () => 2; f += () => 3; int result = f(); // 3; the results 1 and 2 are discardedConsequences:
- Lost results: every result is available only by enumerating
GetInvocationList()and invoking each delegate separately.- One failure skips later handlers: an event raised with a single call stops at the first subscriber that throws. Isolating subscribers requires invoking each entry from
GetInvocationList()inside its owntry/catch.
What is the difference between an event and a public delegate field?
Outside code can only subscribe to and unsubscribe from an event, and the declaring type alone decides when it is raised. A public delegate field exposes the delegate value itself, so any outside code can also invoke it, overwrite it or clear it.
What outside code does public event EventHandler? Changed;public EventHandler? Changed;(field)Add or remove a handler: feed.Changed += OnChanged;Allowed, through the event’s addandremoveaccessors.Allowed, by storing a new combined delegate in the field. Raise it: feed.Changed?.Invoke(feed, EventArgs.Empty);Compile error, so only the publisher can announce the change. Allowed, so any caller can send a notification for a change that never happened. Overwrite it: feed.Changed = OnChanged;or= nullCompile error. Allowed, so one careless =silently removes every other subscriber.Inside the declaring type, a field-like event behaves like its backing delegate, so the publisher can still invoke it and check it for
null. Theeventkeyword therefore restricts access to the same delegate mechanism rather than introducing a different one.
Why can event subscriptions keep an otherwise unused object alive?
Subscribing stores a delegate in the publisher, and a delegate for an instance method keeps a reference to the subscriber as its target. The garbage collector reclaims only objects that are unreachable from roots. As long as the publisher is reachable, the subscriber is reachable through it, even when nothing else uses the subscriber.
Reference path: GC root → publisher → event delegate → subscriber.
- Long-lived publisher: a static event or an application-wide service stays reachable for the life of the process. Every subscriber that never unsubscribes stays in memory, and its handler keeps being called.
- Short-lived publisher: references point from the publisher to its subscribers, not the reverse. Once the publisher becomes unreachable, it no longer keeps its subscribers alive.
The fix is to unsubscribe with
-=when the subscriber’s lifetime ends, commonly inDispose.Microsoft: subscribe to and unsubscribe from events · Microsoft: events · Microsoft: GC fundamentals
What is a C# record, and what does the compiler generate for it?
A record is a class or struct declared with
record, whose equality is based on its data rather than its identity. The compiler generates the members that make it behave as a data value:
- Value equality:
Equals,GetHashCode,==and!=treat two records as equal when they have the same runtime type and equal field values, each field compared with its own type’s equality. Two separately createdrecord classinstances with equal data compare equal, unlike two ordinary class instances.- Nondestructive mutation:
p with { Age = 31 }copies the record and changes the listed members. The copy is shallow, so referenced objects are shared.- Readable
ToString: prints the type name and public members, such asPerson { Name = Alice, Age = 30 }.- Positional members:
record Person(string Name, int Age)also generates a primary constructor, a public property for each parameter andDeconstruct. These properties are init-only forrecord classandreadonly record struct, but settable for a plainrecord struct.
recordalone meansrecord class, a reference type that can inherit from another record class, whilerecord structis a value type. Records fit DTOs, messages and value objects. Types whose identity matters more than their current data, such as Entity Framework Core entities, fit ordinary classes better.
In
record Wrapper(List<int> Items), ifvar b = a with { };and an item is added tob.Items, doesaobserve the change, and why?Yes. A
withexpression makes a shallow copy: it copies each field ofainto the new record, and for a reference-type field that copies the reference, not the object it points to.a.Itemsandb.Itemstherefore refer to the sameList<int>, so an item added throughb.Itemsis visible througha.Items.var a = new Wrapper(new List<int> { 1, 2 }); var b = a with { }; b.Items.Add(3); Console.WriteLine(a.Items.Count); // 3An independent copy needs its own list, for example
a with { Items = new List<int>(a.Items) }.
Given
record Wrapper(List<int> Items)andvar b = a with { };, areaandbequal before and after an item is added tob.Items? Why?They are equal both before and after the item is added. Generated record equality compares the runtime type and each field using that field’s own equality, and
List<int>compares by reference. Becausewithcopied the reference,a.Itemsandb.Itemsrefer to the same list. Adding an item changes that list in place without replacing it. The comparison therefore sees the same reference both times and never looks at the list’s contents.The converse follows from the same rule: two
Wrapperinstances holding separate lists with identical items compare unequal.
How can a record compare a collection member by its elements instead of by reference?
There are two ways: override the record’s equality so it compares the collection’s elements, or store the elements in a type whose own
Equalscompares elements. Generated record equality compares each field with that field type’s equality, andList<T>compares references, so the default never looks at the elements.Overriding equality in the record replaces the generated
Equals(T?)and keepsGetHashCodeconsistent with it:public sealed record Basket(string Owner, List<string> Items) { public bool Equals(Basket? other) => other is not null && Owner == other.Owner && Items.SequenceEqual(other.Items); // element-wise and order-sensitive public override int GetHashCode() { var hash = new HashCode(); hash.Add(Owner); foreach (var item in Items) hash.Add(item); return hash.ToHashCode(); } }The generated
==and!=call thisEquals, so they also compare elements.GetHashCodehashes the same components thatEqualscompares, so records that compare equal produce equal hash codes, as hash-based collections require.Trade-offs:
- Cost:
SequenceEqualcompares elements pairwise in order and stops at the first mismatch, so a comparison examines up to all n items, O(n), instead of comparing one reference. TheGetHashCodeshown always visits every item, so it is O(n).- Mutation: this equality and hash read the list’s current contents. If the list changes while the record is stored in a
HashSet<T>or used as a dictionary key, the record’s current hash can stop matching the hash the collection recorded, and its equality can change, so lookups become unreliable. A different hash and a failed lookup are both possible but not guaranteed, because different contents can produce the same hash code. This equality therefore fits collections that are not changed while the record serves as a key.- Immutability alone is not enough: an immutable collection prevents such changes, but its type’s own
Equalsstill decides whether elements are compared.Microsoft: records · Microsoft: define value equality · Microsoft: Object.GetHashCode · Microsoft: Dictionary<TKey,TValue> · .NET runtime: SequenceEqual source
When is
record classa better choice thanreadonly record struct?A
record classis the better choice when the value is large, needs inheritance, usesnullas an ordinary ‘no value’ state, or is frequently handled asobjector an interface. Areadonly record structfits small immutable values where avoiding a separate heap object matters more than those features. Both generate value equality and supportwith, so equality alone does not decide.
Aspect record classreadonly record structAssignment and passing Copies a reference, so the cost is the same however many fields the record has. Copies every field, which is cheap for a few small fields and grows with the size of the value. Storage Each instance is a separate heap object that the garbage collector must track. Stored inline in its variable, field or array element. Converting it to objector an interface boxes it into a heap copy.Absence A variable of the type can be null.Needs Nullable<T>(T?) to represent a missing value.Inheritance Can derive from another record class, and equality accounts for the runtime type. Cannot take part in inheritance: it neither derives from a record nor can be derived from. When inheritance or
nullas absence decides the choice, no measurement is needed. When the justification rests on performance, the size at which copying outweighs allocation depends on the workload, so allocation and copy costs drive the decision only when measurement shows they matter.Microsoft: records · Microsoft: value types · Microsoft: boxing
If record equality is customized, why must
GetHashCodestay consistent with it?Hash-based collections such as
Dictionary<TKey,TValue>andHashSet<T>useGetHashCodeto decide where to look, and then callEqualsonly on entries with a matching hash code. Objects that are equal must therefore return the same hash code. Otherwise a lookup never reachesEqualsfor the matching entry.For example, suppose a custom
Equalsignores anUpdatedAtproperty, but the generatedGetHashCodestill includes every field:
- Record
ais added to aHashSetunder a hash code that includesa.UpdatedAt.- Record
bdiffers only inUpdatedAt, soa.Equals(b)istrue, butb’s hash code is computed from a differentUpdatedAt.- A lookup for
bsearches byb’s hash code and never compares it witha.Contains(b)returnsfalse, andAdd(b)stores a second entry equal to the first, unless the two hash codes happen to collide.
GetHashCodetherefore must not depend on anything that can differ between two equal records, such as a property thatEqualsignores. Hashing exactly the components thatEqualscompares is the usual choice. A coarser hash over a subset of them is also valid, because equal records agree on that subset; it only causes more collisions. The reverse is not required: unequal objects may share a hash code, which only costs extraEqualscalls. For records, the compiler reports CS8851 when a record defines its ownEqualsbut notGetHashCode.Microsoft: Object.GetHashCode · Microsoft: record declaration errors · Microsoft: define value equality
Can a record struct be used safely as a
Dictionarykey, and what can make it unsafe?Yes, as long as the equality and hash code the dictionary actually uses cannot change while the key is stored. That is either the key type’s own equality or the
IEqualityComparer<TKey>supplied to the dictionary. A dictionary stores its own copy of a struct key, so later assignments to the original variable do not affect the stored key. The risk comes from referenced objects: copying a struct copies references, not the objects they point to. Suppose the effective equality or a supplied comparer observes a referenced object’s mutable state. Changing that object can make the key’s current hash differ from the hash the dictionary recorded and can change its equality. The dictionary does not notice, so lookups for the entry can fail.var tag = new Tag { Name = "a" }; var map = new Dictionary<Key, string> { [new Key(1, tag)] = "x" }; tag.Name = "b"; // changes state inside the stored key map.ContainsKey(new Key(1, tag)); // false unless the "a" and "b" key hashes collide map.ContainsKey(new Key(1, new Tag { Name = "a" })); // false: Equals now sees "b" public record Tag { public string Name { get; set; } = ""; } public readonly record struct Key(int Id, Tag Label);
readonlyprevents reassigning the struct’s own fields but does not freeze the objects they reference. A reference member compared by reference, such as aList<T>under the default comparer, stays stable when its contents change, because its identity does not. That holds only while the effective comparer also compares it by reference. A supplied comparer that compares the list’s elements makes the key depend on mutable state again. A record struct key is therefore safe when every component that the effective equality and hash observe is immutable or compared by reference.Microsoft: records · Microsoft: value types · Microsoft: Dictionary<TKey,TValue> · Microsoft: Object.GetHashCode
When is
StringBuildera better choice than string concatenation?
StringBuilderis the better choice when text is built through many appends, especially in a loop or another frequently executed path where allocations matter. For a small, fixed expression, interpolation or concatenation is clearer.The difference comes from string immutability:
- Repeated concatenation: a
stringcannot change, so eachresult += partallocates a new string holding the accumulated text plus the new part. Each loop step leaves behind an intermediate string, and the growing text is copied again every time.StringBuilder: each append writes into a mutable buffer, andToString()creates the final string once. The buffer reallocates only when it runs out of capacity, so an initial capacity close to the expected final size avoids that growth.No fixed number of appends marks the switch point. The signal for a builder is repeated construction in code where measured allocation cost matters.
Why can
ReferenceEquals(a, b)be false whena == bis true for strings?They test different things. For strings,
==compares contents: it returns true when both strings hold the same characters in the same order, using an ordinal, case-sensitive comparison.ReferenceEqualsreturns true only when both references point to the same object.Equal text can therefore live in two separate objects. Strings built at run time are not interned automatically, so two strings built from the same pieces are normally distinct instances:
==returns true whileReferenceEqualsreturns false.Interning can make identity coincide through two different mechanisms:
- Explicit
String.Intern: it returns the intern pool’s reference for a string with that value, adding the argument to the pool if no equal string is there yet. Equal strings passed throughString.Interntherefore always come back as the same reference.- Automatic literal interning: identical string literals typically share one pooled instance, so
ReferenceEqualsoften returns true for them. A compilation can requestNoStringInterning, and runtime behavior may vary, so literal identity is not a dependable contract.Because identity depends on how each string was produced,
ReferenceEqualsis not a test of content. Content comparisons belong to==orstring.Equals, with an explicitStringComparisonstating the rule the data needs: ordinal comparison (OrdinalorOrdinalIgnoreCase) for protocol keys and identifiers, and culture-aware comparison for natural-language text shown to users.Sources: Microsoft Learn: Object.ReferenceEquals · Microsoft Learn: String equality operator · Microsoft Learn: String.Intern · Microsoft Learn: Best practices for comparing strings
Why might changing a struct inside
foreachnot update the collection, and how can it be fixed?In an ordinary (non-
ref)foreach, each struct element reaches the loop body as a copy held in a read-only iteration variable. Changes to the copy’s own fields never reach the stored element.
- Assigning the iteration variable, or one of its fields, is rejected by the compiler.
- Calling a mutating method on it compiles, but the method changes the copy’s fields rather than the stored element’s, so the update is silently lost.
The copy is shallow. When a field of the struct refers to an object, such as a
List<T>, the copy and the stored element refer to the same object. A change made to that object through the copy, such as adding an item, therefore remains visible through the collection. Only changes to the struct’s own field values are lost.The stored elements can be updated in two ways:
- Indexed write-back: a loop by index reads
items[i]into a local, changes the local and assigns it back toitems[i], replacing the whole element value.refiteration: when the collection exposes its elements by reference, asSpan<T>does,foreach (ref var item in span)makesiteman alias for the stored element, so changes apply in place.Mutable structs make this mistake easy. Immutable structs, updated by creating a new value and assigning it back, keep the copy boundary visible.
Sources: Microsoft Learn: Iteration statements · Microsoft Learn: Structure types · Microsoft Learn: Value types
Where does boxing commonly happen, and how can it be reduced?
Boxing happens wherever a value type is converted to
objector to an interface it implements. The runtime allocates an object on the managed heap and copies the value into it. A nullable value type is boxed according to its contents: a populatedint?boxes its underlyingintvalue, and an empty one converts to anullreference without allocating.Common places:
object-typed variables, collections and parameters:object o = 42;, adding anintto aList<object>or anArrayList, or passing a struct to a parameter of typeobject.- Interface-typed variables and parameters: assigning a struct to an
IComparablevariable, or passing it where an interface type is declared.Each box is a separate heap allocation that the garbage collector must later reclaim. The box also holds a copy, so later changes to the original value do not affect it.
Ways to reduce boxing:
- Generic collections and APIs: a generic collection such as
List<int>is specialized forint, so adding an element stores the value without converting it toobject.- Constrained generics instead of interface-typed parameters: a method such as
Max<T>(T a, T b) where T : IComparable<T>can callCompareToon a struct argument without converting it to the interface type, provided the struct itself implements that member.These changes matter mainly in frequently executed code. They justify extra API complexity where profiling shows boxing allocations are significant.
Sources: Microsoft Learn: Boxing and unboxing · Microsoft Learn: Nullable value types · Microsoft Learn: Generic classes and methods · Microsoft Learn: OpCodes.Constrained
When should a type be a
struct,class, orrecord class?The choice follows the semantics the type needs. A
structfits a small value that is copied field by field, aclassfits an object with its own identity, and arecord classfits a reference type whose contents define its equality. Copying, mutation and measured allocation costs then confirm or adjust that semantic choice.
Aspect structclassrecord classAssignment and passing Copies every field, so each variable holds its own field values. The copy is shallow: a reference-type field copies only the reference, so both copies can still point to one object. Copies a reference, so several variables can share and observe one object. Copies a reference, as a class does. Default equality Equalsis true when both have the same type and the same stored values.Equality holds only when both references point to the same object, even if every field matches. Compiler-generated Equalsand==compare the type and member values. The generatedGetHashCodecompares nothing; it returns hash codes compatible with that equality, so equal records hash equally.Mutation A change to a copy’s own fields affects only that copy, so such changes are easy to lose. A change to an object reached through a reference field is visible through every copy that shares it. A change is visible through every reference to the object. Positional properties are init-only by default, and a withexpression creates a modified shallow copy whose reference-type members still point to the same objects.Allocation and cost Stored inline in the variable, field or array element that holds it, with no allocation of its own. Large values make every copy expensive, and boxing allocates. One heap allocation per instance. One heap allocation per instance, as with a class. Inheritance Cannot derive from another struct or class or serve as a base type, though it can implement interfaces. Supports class inheritance. Can inherit only from another record. Typical fit Small data-centric values with little behavior, such as a coordinate. Entities with identity, a lifecycle or polymorphic behavior. Data defined by its contents, such as messages or DTOs. Sources: Microsoft Learn: Structure types · Microsoft Learn: Value types · Microsoft Learn: Classes (C# language specification) · Microsoft Learn: Records
How do managed and unmanaged code differ, and why does interop require careful lifetime management?
Managed code runs under the CLR and relies on its services, such as garbage collection of managed objects, exception handling and the runtime type system. Unmanaged (native) code runs outside those services. It follows a native platform ABI and its own ownership rules, such as explicit freeing or reference counting.
Interop needs careful lifetime management because the GC manages only one side of the boundary:
- The GC tracks managed objects only. It cannot tell when a native handle or buffer is no longer needed.
- A native resource is released only when code calls its native cleanup. Disposal does this deterministically when ownership ends. A finalizer on the managed wrapper can do it later, as a fallback, if disposal was missed.
SafeHandleis the usual managed owner for a native handle. It releases the handle when disposed, or during finalization if disposal never happened.If no code clearly owns a native resource, nothing may release it, and it leaks. Interop declarations must also match the native ABI, such as calling convention and data layout. That requirement affects whether a call works correctly, not how long a resource lives.
Microsoft: Native interoperability ABI support · Microsoft: SafeHandle class
How does .NET turn IL into machine code, and when do JIT, ReadyToRun and Native AOT each fit?
Assemblies normally contain IL, which the processor cannot run directly. By default the runtime’s JIT compiler turns each method into machine code when it is first needed. ReadyToRun moves part of that work to publish time while keeping the JIT. Native AOT moves all of it to publish time and ships no JIT.
Aspect JIT ReadyToRun Native AOT When machine code is produced At run time, as methods are first used. Tiered compilation can later recompile hot methods with more optimization. At publish time for many methods. The JIT still compiles methods without usable precompiled code and can re-optimize hot ones. Entirely at publish time. The app runs without a JIT. Startup Early calls pay for compilation. Less compilation work at startup. No JIT work at startup. Runtime flexibility Dynamic loading and runtime code generation work. Same as JIT, because the IL is kept. Restricted: no runtime code generation, and code must work with trimming. Main cost Compilation time on first use. Larger assemblies built for a specific target. Compatibility constraints and platform-specific output. JIT is the flexible default, especially when an app loads code dynamically or generates code at run time. ReadyToRun reduces startup compilation without giving up the JIT. Native AOT fits startup- and footprint-sensitive deployments whose code and dependencies work with AOT and trimming. ReadyToRun trades larger assemblies for less startup compilation, while Native AOT trades runtime flexibility for startup and footprint benefits, so the choice rests on measured startup and footprint needs and on the runtime features the app requires. Native AOT is not a universal upgrade.
Microsoft: .NET application publishing overview · Microsoft: Native AOT deployment
How does the .NET garbage collector decide what managed memory it can reclaim?
The GC can reclaim any managed object that is no longer reachable from a root. Reachability alone decides: an object that is still reachable stays, even if the program will never use it again.
- Start from roots. Roots are references the runtime knows about directly. They include static fields, local variables and CPU registers of running methods while the JIT reports them as live, and GC handles.
- Mark reachable objects. The GC follows reference fields from the roots, then from every object it reaches. Each object found this way is live.
- Treat everything else as reclaimable. An object that was not reached is eligible for reclamation. Cycles do not keep objects alive: two objects that refer only to each other are collectible once nothing reachable points to either.
Qualifications:
- Generations: a collection reclaims objects only in the generations it collects. A young-generation collection also inspects the older-heap ranges that CoreCLR’s write barriers recorded in a card table as possibly pointing to young objects. It does not scan the whole older generation, and it does not reclaim older objects. An unreachable generation 2 object is therefore reclaimed only by a collection that includes generation 2.
- Finalization: an unreachable object whose finalizer is still registered survives until its finalizer has run. A later collection normally reclaims it.
- Timing: eligibility does not mean immediate reclamation or memory returned to the operating system.
The reachability decision covers managed memory only. Native handles and buffers need explicit cleanup, typically through
IDisposableandusing.Microsoft: Fundamentals of garbage collection · dotnet/runtime: CoreCLR GC write barriers
How do generations make .NET garbage collection more efficient?
Generations let the GC collect the small, recently allocated part of the heap often and the large, long-lived part rarely. This pays off because most objects become unreachable soon after allocation. A collection limited to new objects reclaims much of the garbage while examining little of the heap.
- Generation 0: most new objects start here. It is collected most often, and a collection typically finds many of its objects already unreachable.
- Generation 1: objects that survived a generation 0 collection. It buffers between short-lived and long-lived objects. A generation 1 collection also collects generation 0.
- Generation 2: long-lived objects, such as application-lifetime services and caches. A generation 2 collection is a full collection. It includes every generation and must consider every reachable object, so its work grows with the whole live heap. Its actual cost depends on the workload, heap size and GC mode.
Survivors of a generation 0 or 1 collection usually move up one generation, although pinning and collector heuristics can keep an object in its current generation. Objects that survive a generation 2 collection stay in generation 2. Large objects, normally 85,000 bytes or more, go to the Large Object Heap, which is collected together with generation 2.
The design has two consequences:
- A young collection does not walk the whole older generation, so references from old objects to young ones must be recorded separately. The runtime records them with write barriers and a card table, and the young collection inspects only the recorded older-heap ranges.
- An object that lives just long enough to be promoted is then reclaimed only by a less frequent collection that includes its new generation.
Microsoft: Fundamentals of garbage collection · dotnet/runtime: CoreCLR GC write barriers
How does the .NET GC find old-to-young references during a young-generation collection?
The GC relies on write barriers that maintain a card table. When code stores a reference into a heap object, the write barrier can mark the area of the older heap that now points to young objects. A young-generation collection scans those marked areas, in addition to the roots, instead of walking the entire older generation.
This is needed because a young object can be reachable only through a field of an older object. Roots alone would miss that reference, and the collector would reclaim a live object. Scanning every older object on each young collection would make it nearly as expensive as a full collection.
In CoreCLR the process is:
- The JIT inserts a write barrier around managed reference stores into heap objects.
- When a store may create a reference from an older generation to a younger one, the barrier marks the matching card-table entry. Each entry covers a small range of heap memory.
- A generation 0 or 1 collection scans the objects in dirty card ranges, follows the young references found there, and keeps those young objects alive.
The cost is a little extra work on reference writes, in exchange for young collections that do not walk the whole older heap. Which stores the barrier marks, how much memory each card covers and the exact scanning work are CoreCLR implementation details. They depend on runtime version and collection type and are not a .NET API contract.
dotnet/runtime: CoreCLR GC write barriers · Microsoft: Fundamentals of garbage collection
Why can process memory stay high after a generation 2 collection?
A generation 2 collection frees space only from unreachable managed objects, and freed space is not necessarily returned to the operating system. Memory can stay high for several independent reasons:
- Live retention: objects still reachable from roots, such as caches or static fields, survive every collection. The memory stays until the reference keeping the graph alive is removed.
- Fragmentation: pinned objects, and areas that are swept rather than compacted, can leave free gaps between survivors. The Large Object Heap in ordinary collections is one such area. The heap keeps that space, but new allocations may not fit into it.
- Retained capacity: the GC can keep memory committed after a collection so later allocations can reuse it. Committed heap size can therefore exceed the size of live objects.
- Memory outside the GC heap: native allocations and other non-GC memory in the process are not reclaimed by a managed collection.
Live object bytes, committed GC memory and process working set are different measurements, so a drop in one need not appear in another. Runtime counters and memory traces separate these causes before code or GC settings are changed. A large live heap points to retention. Much free or committed space relative to live objects points to fragmentation or retained capacity.
Can a .NET application leak memory even though it has garbage collection?
Yes. The garbage collector reclaims a managed object only when the object is no longer reachable, and it cannot tell that a reachable object is no longer needed. Resources outside the managed heap can also leak.
- Managed retention: an object stays alive while a chain of references from a GC root, such as a static field or a local variable on a thread’s stack, still reaches it. An unbounded cache held in a static field keeps its entries reachable. So does a long-lived publisher whose event still holds a subscriber’s handler. The GC is working correctly here: it never reclaims data the program can still reach.
- Unmanaged resources: native memory and operating-system resources, such as files or network connections, must be released explicitly, and the GC does not know how to release them. If neither the owner’s
Disposecall nor a finalizable wrapper such asSafeHandlereleases them, they stay allocated even after the managed wrapper object is collected.Rising memory alone does not prove a leak, because heap growth, caches and allocators can keep memory high for legitimate reasons. A leak is established by the reference path that keeps unwanted objects alive, or by a native allocation that is never released.
Microsoft: Fundamentals of garbage collection · Microsoft: Implement a Dispose method · Microsoft: Debug a memory leak in .NET
How can a suspected memory leak in a running .NET application be diagnosed?
Diagnosis first confirms a sustained growth trend, then follows the kind of memory that is growing until evidence explains why it is retained. For managed memory, that evidence is the reference path from a GC root to objects that are no longer needed.
- Confirm a trend: runtime counters, for example from
dotnet-counters, are compared across comparable workload cycles and at comparable points relative to collections. A single high reading or growth during warm-up is not yet evidence of a leak.- Locate the growth: counters indicate where to look, not the cause. Heap-size readings depend on the measurement. On .NET 9 and later, the runtime’s GC heap-size metric,
dotnet.gc.last_collection.heap.size, includes fragmentation. For apps on .NET 8 and earlier,dotnet-countersfalls back to legacy EventCounters, whosegc-heap-sizecounter is based onGC.GetTotalMemoryand excludes fragmentation. GC committed memory also includes capacity ready for future allocations. Whichever measurement is used, a rising heap metric is a lead toward managed retention rather than proof of it. Process memory that rises without matching growth in GC committed memory is a lead toward memory outside the managed heap, such as native allocations or handles.- Managed growth, compare heap contents: dumps taken at equivalent workload points, for example with
dotnet-dump collect, show which types grow in count or total size. Indotnet-dump analyze,dumpheap -statsummarizes the heap by type.- Managed growth, find the retaining path: for a representative instance of a growing type,
gcroot <address>lists the chain of references from a GC root, such as a static field, to that object. The path names the holder, such as a cache without eviction or an event subscription that is never removed. The fix bounds or ends that ownership.- Native or handle growth: this branch needs evidence from its own layer, such as a native allocation or an operating-system handle that is never released.
gcrootexplains only managed reachability, so it cannot establish this cause.The type that uses the most memory is only a lead until a retaining path explains it. A retaining path proves that an object is reachable. The retention is a leak only when the object is no longer useful.
Microsoft: Debug a memory leak in .NET · Microsoft: .NET runtime metrics · Microsoft: Well-known EventCounters in .NET · Microsoft: GC.GetTotalMemory · Microsoft: dotnet-counters · Microsoft: Fundamentals of garbage collection
Why is
usingneeded when .NET already has garbage collection?The garbage collector frees managed memory when allocations or low system memory trigger a collection, not when code has finished with an object. Many objects also hold resources other than managed memory, such as files, network connections or database connections. Those resources are limited, and the GC does not know how to release them. At best, a finalizer releases them at an unpredictable time later, so a resource can stay open long after its last use or be exhausted.
usingmakes the release deterministic. The compiler wraps the scope intry/finallyand callsDisposeon theIDisposableobject in thefinallyblock. The resource is therefore released as soon as control leaves the scope, whether normally or through an exception.await usingdoes the same forIAsyncDisposableby awaitingDisposeAsync.using (var stream = File.OpenRead(path)) { // reads from stream } // stream.Dispose() has run here, even if reading threwDisposal releases the resource, not the object’s managed memory. The GC still reclaims that memory later, once the object is unreachable.
Microsoft: using statement · Microsoft: Fundamentals of garbage collection · Microsoft: Implement a Dispose method
How do
IDisposableand a finalizer differ?
Disposeis deterministic cleanup that the owning code calls when it has finished with a resource. A finalizer is an optional fallback that the garbage collector runs at an unpredictable time after it finds the object unreachable, so that cleanup can still happen ifDisposewas never called.IDisposableitself requires only theDisposemethod, not a finalizer.
Aspect IDisposable.DisposeFinalizer Trigger Owning code calls it, directly or through using.The GC queues it to run after finding the object unreachable. Timing Runs at a known point, so scarce resources are released promptly. Runs at an unpredictable time after the object becomes unreachable. .NET 5 and later do not run finalizers when the application exits, so it may never run. What it can safely clean up Managed IDisposableobjects the type owns, plus its unmanaged resources.Only unmanaged resources the object owns directly. Other managed objects may already have been finalized, because finalization order is undefined. Cost An ordinary method call. Each finalizable object adds an entry that the GC must process in its finalization queue. When a type defines its own finalizer, the two work together.
Disposeperforms the cleanup and callsGC.SuppressFinalize, so the finalizer runs only when disposal was missed. Most types can avoid an own-type finalizer altogether. A type whose native handle is wrapped in aSafeHandleneeds no finalizer of its own, becauseSafeHandlealready provides finalization for that handle. A custom finalizer is justified only for an unmanaged resource that the type owns directly and that no safe wrapper can hold.Microsoft: Implement a Dispose method · Microsoft: Finalizers · Microsoft: SafeHandle
How does the .NET dispose pattern work?
The pattern routes both cleanup paths through one method,
Dispose(bool disposing). Its argument tells it whether owning code calledDisposeor a finalizer is running.
- Explicit path: owning code calls the public, non-virtual
Dispose(), which callsDispose(true)and thenGC.SuppressFinalize(this). Withdisposingset totrue, the method releases both the managedIDisposableobjects the type owns and its unmanaged resources. Suppressing finalization tells the GC that cleanup is already done, so it does not run the finalizer.- Finalizer path: if
Disposewas never called and the type has a finalizer, the finalizer callsDispose(false). That call releases only unmanaged resources the object owns directly. It does not touch other managed objects, because finalization order is undefined and they may already have been finalized.- Repeated calls:
Dispose(bool)records that disposal has happened and returns early on later calls. CallingDisposemore than once is therefore harmless and never releases the same resource twice.- Inheritance: in an unsealed type,
Dispose(bool)isprotected virtual. A derived class overrides it to release its own resources and then callsbase.Dispose(disposing).public class NativeBuffer : IDisposable { private IntPtr _buffer = Marshal.AllocHGlobal(1024); private bool _disposed; public void Dispose() { Dispose(true); GC.SuppressFinalize(this); } protected virtual void Dispose(bool disposing) { if (_disposed) return; if (disposing) { // Owned managed IDisposable fields are disposed here. } Marshal.FreeHGlobal(_buffer); _buffer = IntPtr.Zero; _disposed = true; } ~NativeBuffer() => Dispose(false); }This type needs a finalizer only because it holds a raw native pointer directly. When a
SafeHandleowns the native resource, the type implementsDispose()andDispose(bool)without a finalizer, and theSafeHandlesupplies the finalization fallback.Microsoft: Implement a Dispose method · Microsoft: Finalizers
What does the CLR do when an application starts, and why does startup behavior matter?
A native host first locates and initializes a compatible runtime and asks it to run the app. Most of the remaining work then happens on demand rather than in fixed phases. Assemblies load as code needs them. Each method runs from usable ReadyToRun precompiled code or is JIT-compiled when first needed. Static type initialization runs no later than the first use that requires it, and its exact timing depends on whether the class has an explicit static constructor. This work can add cold-start latency, so early requests can be slower than steady state. How much depends on the application, its deployment and which code paths run first.
- Host: for an app running on CoreCLR, the app’s executable or the
dotnetcommand acts as a native host. It reads the runtime configuration, selects a compatible runtime, resolves the paths of the app’s dependencies, initializes the CLR and requests that it run the app.- Assembly loading: the CLR loads the entry assembly in order to run it. Referenced assemblies load as needed, not necessarily before the entry point runs. Their exact load timing is unspecified and can vary with the runtime version and optimizations.
- Code generation: a method needs native code before it runs, including the entry point. If its assembly contains usable ReadyToRun code for the method, that code runs. Otherwise the JIT compiler compiles the method’s IL. Some methods still need the JIT in a ReadyToRun app, and tiered compilation can later replace frequently called code with more optimized code.
- Type initialization: in C#, a class with an explicit static constructor is initialized when its first instance is created or a static member is first referenced. Without one, static field initializers run at an implementation-dependent time before the first use of a static field. Expensive static setup adds its cost to whichever operation reaches that boundary first.
These costs land on the first work that crosses each boundary. They can appear in the first requests after a deployment or scale-out, or later when a new code path first runs. ReadyToRun reduces runtime JIT work, including for code first used after startup, but produces larger binaries. Native AOT compiles the app ahead of time and has no runtime JIT, at the cost of restrictions such as no runtime code generation. Traces show which cost dominates for a particular application.
dotnet/runtime: Host components · Microsoft: Managed assembly loading algorithm · Microsoft: Managed execution process · Microsoft: ReadyToRun compilation · Microsoft: Native AOT deployment · C# specification: Classes
What does Big O notation describe?
Big O notation describes an upper bound on how an algorithm’s cost, usually running time or extra memory, grows as a function of input size
n. It states how an approach scales, not how long it takes on a particular machine.Formally,
f(n) = O(g(n))means positive constantscandn₀exist such thatf(n) ≤ c · g(n)for everyn ≥ n₀. Beyond some input size, the cost grows no faster than a fixed multiple ofg(n).
- Upper bound, not necessarily tight: a loose bound is still true, so a linear-time algorithm is also
O(n²). A bound that holds from both above and below is tight and is writtenΘ.- Defined input size: a bound is meaningful only once
nis stated. For string processing,ncan count strings or total characters, and the two can give different bounds.
Why does Big O notation drop constant factors and lower-order terms?
They cannot change an expression’s growth class, so dropping them loses nothing that Big O expresses. Its definition only requires some fixed constant
cand starting sizen₀such thatf(n) ≤ c · g(n)for everyn ≥ n₀. A constant factor can be folded intoc. A lower-order term grows more slowly than the dominant term. Beyond some size, it is therefore at most a fixed multiple of that term and can be folded intocas well.For example, when
n ≥ 1, each term ofn² + 100n + 500is at most a multiple ofn²:
n² + 100n + 500 ≤ n² + 100n² + 500n² = 601n²So
c = 601andn₀ = 1satisfy the definition, and the expression isO(n²), the same class asn²or5n².The class describes eventual growth only. The discarded parts still affect actual cost: for every
nup to 104,100n + 500is larger thann²itself.
What is the difference between average-case and amortized complexity?
Average-case complexity is the expected cost when inputs or random choices follow a stated probability distribution. Amortized complexity is a guaranteed bound on the total cost of a sequence of operations, spread across those operations. Amortized analysis uses no probability, so it is not a kind of average-case analysis.
Aspect Average case Amortized What is averaged The cost of an operation over possible inputs or random choices, weighted by their probability. The total cost of a sequence of operations, divided by the number of operations. Assumption A stated distribution. If real inputs do not follow it, the bound does not describe them. No distribution. The bound holds for every sequence the analysis covers. What can still be slow A particular unlucky or adversarial input can cost much more than expected. A single operation can be expensive, but expensive steps are rare enough that the sequence total stays within the bound. Example A hash lookup is constant time on average only under suitable hashing and collision assumptions. Dynamic-array append is amortized O(1). A resize copies every element inO(n), but growing capacity by a constant factor, such as doubling, keeps the total fornappends atO(n).NIST: Average-case complexity · MIT 6.006 Spring 2020: Lecture 2
When can constant factors outweigh asymptotic complexity?
Constant factors can outweigh asymptotic complexity when inputs are small or bounded.
Big O describes how cost grows with input size and deliberately ignores constant factors. An
O(n log n)bound only says the cost stays below some constant timesn log noncenis large enough. An algorithm with a better growth rate but more overhead can therefore be slower below some crossover size. Typical overhead includes setup work, extra bookkeeping and poorer memory access patterns. Above the crossover, the better growth rate eventually dominates.Hybrid sorting is a concrete case. The .NET 10 documentation describes
Array.Sortas introsort, which sorts partitions of 16 or fewer elements with insertion sort. Insertion sort grows quadratically, but its low overhead suits such small partitions. The value 16 is that implementation’s threshold, not a general rule.The crossover depends on the implementation, hardware and data. It is therefore established by measuring the actual workload rather than inferred from Big O alone.
NIST: Big-O notation · Microsoft: Array.Sort (.NET 10) · dotnet/runtime ArraySortHelper source
How can the dominant operation reveal whether a performance problem needs a different data structure or algorithm?
The dominant operation is the step that accounts for most of the work, and it shows where the cost lies. If a data structure that performs that operation cheaply removes most of the work, the representation was the problem. If the operations the task requires remain too expensive even with a suitable structure, the algorithm has to change.
- Repeated membership checks: checking membership in a
List<T>scans the list. Checking each item of a batch against a list of similar size inside a loop therefore costs O(n²).HashSet<T>.Containstakes expected O(1) time with a good hash function, so the same batch costs expected O(n) while the surrounding logic stays unchanged.- Ordered iteration or range queries: when repeated sorting or range scanning dominates,
SortedSet<T>keeps elements ordered as they are added and removed, andGetViewBetweenreturns a view of a range. The set positions an element when it is inserted, so changing the fields that determine its order in place is unsupported. Instead, the element is removed, changed and then added again.- Algorithm change: sometimes the remaining cost comes from how many operations the approach performs rather than how expensive each one is. No data structure removes that cost, so the algorithm itself has to do less work.
Microsoft: HashSet.Contains (.NET 10) · Microsoft: SortedSet class (.NET 10)
How does backtracking work, and how does it relate to brute-force search?
Backtracking is a form of brute-force search that builds candidates one choice at a time. It abandons a partial candidate as soon as that candidate is known to be unsuitable.
The search process:
- Start with an empty partial candidate.
- Choose one available option and add it to the partial candidate.
- Recursively explore the remaining choices from that partial candidate.
- If the candidate is complete and valid, record it. If a sound check proves the partial candidate cannot lead to a valid solution, it is a dead end and its exploration stops.
- Return to the previous decision point, undo the choice and try the next option. When no options remain, return to the level above.
For example, when placing queens row by row, a placement that already attacks another queen is abandoned without filling the remaining rows.
Approach How it searches Consequence Naive complete-candidate enumeration It generates every complete candidate and then checks each one. Invalid candidates are fully built before they are rejected. Backtracking It extends a partial candidate one choice at a time and can check it along the way. Rejecting a partial candidate skips all of its continuations. Pruning depends on having a check that can reject partial candidates. Without such a check, backtracking still visits every complete candidate. Even with pruning, the number of branches can grow exponentially in the worst case, so backtracking does not guarantee a fast search.
CS106B: Exhaustive Search and Recursive Backtracking (Stanford)
In backtracking with shared mutable state, why undo each choice before trying the next?
Undoing restores the parent state, so each sibling choice starts from the same prefix instead of inheriting the previous branch’s changes.
Pattern: choose → explore (recurse) → un-choose.
Example: permutations of A, B, C built in one shared list.
- At the root the list is
[ ]. Choose A, giving[A].- Explore below A. The inner levels add and remove B and C, record
[A, B, C]and[A, C, B], and return with the list at[A].- With undo: remove A, giving
[ ]. Choose B, giving[B]. The B branch now explores permutations that start with B.- Without undo: A stays in the list. Choosing B gives
[A, B].The B branch should start from the root’s empty prefix, but it starts from the A branch’s leftover state. It therefore explores the wrong part of the search space. If A’s ‘used’ flag also stays set, the search can record
[A, B, C]again and never produce[B, A, C]. If only the list is wrong, it can record an invalid result such as[A, B, A]. Later sibling branches can inherit even more leftover state.Related changes, such as ‘used’ flags, must also be undone. A retained solution must be copied before later undo steps.
Alternative: each branch can instead receive its own copied or immutable state. Then no explicit undo is needed. Copying state adds a per-branch cost; immutable representations may share structure.
CS106B: Exhaustive Search and Recursive Backtracking (Stanford) · .NET ImmutableStack implementation
What changes between memoization and tabulation if the recurrence is the same?
Memoization and tabulation evaluate the same recurrence and store one result per state. They differ in how the evaluation order is determined and which states get computed.
- Memoization (top-down): starts from the requested state, recursively follows its dependencies and caches each result the first time it is computed.
- Tabulation (bottom-up): loops fill the stored results in an order where every dependency is already available.
Aspect Memoization (top-down) Tabulation (bottom-up) Evaluation order Recursion from the target discovers dependencies as it goes, so no order has to be written down. The dependencies must still be acyclic. The loops must visit states in an order that solves every dependency before any state that reads it. States computed Only states reachable from the target are computed, so unneeded states are skipped. Every state in the planned table is computed, including states the target never reads. Control cost Each call costs a function call and a cache check. Recursion depth follows the longest dependency chain, so very deep chains can overflow the call stack. Plain loops avoid recursive calls and their stack-depth limit. When both compute the same set of states with equivalent work per state, their asymptotic time is identical: the number of states times the work per state.
Coin change with a
30¢target shows the two orders. Memoization starts atbest(30)and recurses into the smaller remaining amounts it needs. Tabulation fillsbest[0]throughbest[30]in increasing order, which works because each amount reads only smaller amounts.
What is memoization, and when does it improve performance?
Memoization caches a function’s results by argument. The first call with a given set of arguments computes the result and stores it. Later calls with the same arguments return the stored result instead of computing it again.
It improves performance only when the same arguments recur. Naive recursive Fibonacci is the classic case:
fib(n)callsfib(n - 1)andfib(n - 2), and both branches keep recomputing the same smaller values. The number of calls therefore grows exponentially, although onlyn + 1distinct arguments exist. With memoization each argument is computed once, and the time becomes linear inn. In general, the time drops to roughly the number of distinct argument sets times the work per call.In a recursive function, the recursive calls must also go through the cache. Calls to the raw, uncached function bypass it and bring the repeated work back.
- No repeats, no gain: if every call has distinct arguments, as with the unique subarrays in merge sort, the cache never hits and only adds lookup and storage overhead.
- Memory: a long-lived cache keeps one entry per distinct argument set unless it is bounded. An evicted entry is recomputed when it is requested again.
Applied to a recurrence with overlapping subproblems, memoization is the top-down form of dynamic programming.
What is dynamic programming, and what does a typical DP solution need to define?
Dynamic programming solves a problem through a recurrence over smaller subproblems. It stores each subproblem’s result so later steps reuse it instead of recomputing it. In finite, one-pass DP, the dependencies between subproblems are acyclic, so each distinct subproblem is solved only once.
A typical DP solution defines:
- State (subproblem): what each subproblem means, such as
best(a), the fewest coins that make amounta. The state must capture everything the remaining answer depends on, so every path that reaches the same state needs the same answer.- Stored result: the value kept for each state, such as a minimum, a count or a yes/no answer.
- Base cases: states answered directly, such as
best(0) = 0.- Recurrence: how a state’s result is computed from the results of the states it depends on, such as
best(a) = 1 + min(best(a - c))over usable coinsc.- Evaluation order: for a finite set of states with acyclic dependencies, an order that solves every dependency before any state that reads it. Memoized recursion follows that order on demand, and tabulation writes it out as loops.
In this one-pass form, the running time is roughly the number of distinct states times the work to compute one state from its stored dependencies.
If the dependencies form a cycle, no such order exists, and a single recursive or tabulated pass cannot solve the problem. Such problems need another method that can revisit states until their values converge, such as a graph shortest-path algorithm or repeated updates.
Storing results pays off when different paths reach the same state (overlapping subproblems). Optimal substructure means an optimum can be built from optimal answers to subproblems. It matters only when the goal is to choose an optimum. Counting and decision problems use the same machinery without optimizing.
When is memoizing a function call correct?
Memoizing a call is correct when returning the stored result is indistinguishable, to callers, from running the function again. Two conditions make that true:
- The key determines the result. The cache key includes every input that can change the result. If an input is missing, calls that differ only in that input share one entry. A later call then receives the earlier call’s answer even when its own answer would differ.
- Skipping the call loses nothing required. A cache hit does not execute the function, so the function must not perform an effect that every call needs, such as sending a message or recording each request.
A pure function’s result depends only on its arguments, and the function has no observable side effects. Purity therefore guarantees the second condition. It also narrows the first condition to the arguments, but the key must still represent every argument that affects the result. A pure two-argument function cached by its first argument alone can return wrong results. Purity with a complete key is a simple sufficient condition, not a necessary one. A function with effects that no caller requires can also qualify.
When the result also depends on mutable external data, such as a price table, entries computed from the old data can become stale once that data changes. Either the data’s version becomes part of the key, or affected entries are invalidated when the data changes.
Why must a memoization key include every part of a subproblem’s state?
A cache hit returns whatever is stored under the key, without checking which subproblem produced it. If two different subproblems produce the same key, the later one receives the earlier one’s answer. Whenever their true answers differ, the result is silently wrong rather than reported as an error. Some colliding subproblems happen to have the same answer, but an incomplete key removes any guarantee that a hit is correct.
Example: 0/1 knapsack.
best(i, capacity)is the highest value obtainable from itemsionward withcapacityweight remaining, so its answer depends on both values.
- One branch reaches item 3 with 10 units of capacity left, computes the best value of the remaining items and stores it under key
3.- Another branch reaches item 3 with only 2 units left.
- The lookup for key
3hits and returns the result for 10 units. That value can include items that do not fit in 2 units, so the second branch can receive an answer that is too high.Keying the cache on
(i, capacity)keeps those subproblems separate. The memo key is the DP state, so it must include everything that affects the remaining answer. History that does not affect it, such as which earlier items were chosen, can be left out. Including such history would still be correct, but it would create more distinct keys and fewer cache hits.
What does ACID mean for a database transaction, and where do its guarantees stop?
ACID names four guarantees a database gives a transaction, a group of reads and writes that ends in one commit decision. They hold only inside that database and only as strongly as its configured isolation and durability settings allow.
- Atomicity: the transaction has one outcome. Either all of its writes commit or none do, so a crash or error cannot leave a partial commit.
- Consistency: a correct transaction moves the data from one valid state to another. Database constraints such as foreign keys, uniqueness and
CHECKenforce the rules the database can express. Application logic must enforce the rest.- Isolation: concurrent transactions are limited in what they can observe of each other and in which combined results can commit. The configured isolation level sets how strict this is.
- Durability: once the database acknowledges a commit, the changes survive the failures its durability configuration covers, such as a crash.
The guarantees stop at these boundaries:
- Configuration: weaker settings narrow the guarantees. For example, PostgreSQL asynchronous commit can acknowledge a commit before it is durably logged, so a crash can lose the most recently committed transactions.
- Retries: atomicity does not make an operation safe to repeat. If a client times out after the server has committed and then retries, the work is applied twice. Making the operation idempotent prevents this, for example with a unique request ID.
- Other databases and services: a local commit cannot include writes elsewhere. Work that spans services needs a separate protocol, such as two-phase commit or a saga of local transactions.
Microsoft: ACID properties · PostgreSQL: Asynchronous commit
What is write skew, and why can snapshot isolation allow it?
Write skew is a concurrency anomaly in which two transactions read the same data, each makes a decision that is valid for what it read, and each writes a different row. Both commit, but the combined result breaks an invariant that neither transaction broke alone.
Example: a rule requires at least one doctor on call, and Alice and Bob are both on call.
Step Transaction A Transaction B Read Sees Alice and Bob on call Sees the same state Decide Bob remains, so Alice may leave Alice remains, so Bob may leave Write Sets Alice off call Sets Bob off call Commit Succeeds Succeeds No doctor is left on call.
Snapshot isolation gives each transaction a stable snapshot. Each transaction checks the rule against a snapshot that does not contain the other’s change. Because they update different rows, there is no concurrent update of the same row to reject, so both commit. The problem is a read-write dependency: each transaction writes data that the other read when deciding.
Preventing write skew requires protecting the whole invariant: a constraint where it can be expressed, locks covering every row or the predicate the decision read, or Serializable isolation. PostgreSQL’s Serializable level detects this kind of dependency and aborts one transaction with a serialization failure. The application then retries the whole transaction.
How should concurrency protection be chosen for a read-modify-write transaction?
Concurrency protection is chosen from the invariant the decision depends on, not from the row being written. The mechanism must protect every input that informed the decision: the rows read, any predicate such as ‘at least one doctor on call’, and rows whose absence mattered.
Mechanisms that can cover the whole decision:
- Database constraint: when the invariant can be expressed as a constraint, the database rejects any commit that would violate it, whatever the interleaving.
- Serializable isolation: the database commits only results that some one-at-a-time order of the transactions could have produced. It may abort a transaction to enforce this. The whole transaction, including its reads and decisions, must then be retried. Retrying only the final write would reuse stale reasoning.
- Locks: explicit locks protect the decision only when they cover every relevant row or the whole predicate.
- Application-specific optimistic check: valid only when it detects a change to any decision input, not just to the updated row.
Two choices that look sufficient but are not:
- Repeatable Read keeps repeated reads or the transaction snapshot stable, depending on the engine. It can still allow write skew: two transactions read the same rows, each updates a different row, and together they break the invariant.
- A row-version token on the updated entity detects a concurrent change to that row only. It misses rows inserted or deleted under the predicate and changes to other rows that were read.
PostgreSQL 18: Transaction isolation · Microsoft: EF Core concurrency conflicts
When can a larger connection pool make performance worse?
A larger pool makes performance worse once the database, rather than the pool, is the bottleneck. Until that point, more connections let more work run concurrently. After the database reaches its CPU, I/O or lock-contention limit, additional sessions add contention and queueing inside the server instead of throughput. Latency rises, and every open session still consumes client and server resources.
The possible session count is also larger than one setting suggests:
- The limit is per pool. A setting such as SqlClient’s
Max Pool Sizecaps one pool, not the whole application or fleet.- Pools fragment. Providers keep separate pools for different connection strings. Depending on the provider, they also separate pools by identity or other connection state. Per-tenant or per-user configurations can multiply connections even when each pool looks small.
- Instances multiply pools. Each process owns its own pools. Fifty instances with a maximum of 100 connections each can open 5,000 sessions, even if the database can usefully run only a small fraction of them at once.
Pool size is therefore a fleet-wide budget. It starts from measured database capacity, reserves headroom for administration and background work, and divides the rest across instances. Reading pool wait time alongside database CPU, I/O and active sessions shows which side is saturated. Waits while the database is healthy point to leaks, long checkouts or an undersized pool. Waits on a saturated database are not fixed by a larger pool.
SQL Server connection pooling · SqlClient connection options
With client-side connection pooling, what happens when application code opens and disposes a database connection?
Opening borrows a physical connection from the pool, and disposing returns it to the pool instead of closing it. The application works with a short-lived logical connection, while the provider keeps the expensive physical connection open for reuse. Establishing a physical connection can require a socket, a security handshake, authentication and a server session, depending on the provider and connection configuration. Without pooling, every open creates a new physical connection and repeats that setup. The cost belongs to establishing the connection, not to each query sent over a connection that is already open.
- Open: the provider selects the pool that matches the connection configuration. It takes an idle physical connection, or creates a new one if the pool is below its maximum size.
- Exhausted pool: if every connection is checked out and the pool is at its maximum, the open call waits for another caller to return one. If none returns before the connection timeout, the open fails.
- Close or dispose: the logical connection ends. The provider resets reusable session state and places the physical connection back in the pool for the next caller.
- Retirement: idle and maximum-lifetime settings let the provider close extra or aging physical connections later.
Failure consequence: a connection that is never closed or disposed may stay checked out. The usable pool also shrinks through leaks, long transactions and slow unrelated work while a connection is open, such as waiting for an HTTP response. Open calls then time out. Pooling therefore works best when a connection is opened just before database work and disposed as soon as that work ends, usually through
usingorawait using.
What determines whether a service should use relational or non-relational storage?
Three needs determine the choice: the invariants the data must keep, the scope of the transactions that protect them, and the queries the service must answer. These needs are judged against a concrete engine’s consistency, query, recovery and operational limits.
- Relational storage is the usual default when correctness depends on constraints, joins, transactions across several records, or query combinations that keep changing. The database enforces those rules close to the data, and a new query can usually be expressed without reshaping storage.
- A specialized non-relational store fits when one access pattern clearly dominates and the engine’s consistency, query and recovery limits are acceptable for that workload. Typical patterns are lookup by key, reading and replacing a whole document, relationship traversal and full-text relevance.
- A combination keeps a relational source of truth and adds one specialized read path, such as a search index or cache, when that path solves a measured need. Each added store brings synchronization and recovery work that must have an owner.
How should a slow database-backed request be diagnosed before scaling the database?
Diagnosis measures where the slow request spends its time and fixes the smallest demonstrated cause. It moves to read offloading or larger infrastructure only when a measured limit remains.
- Time breakdown: the request’s latency is split into pool wait, query execution, lock wait, network, serialization and downstream calls. The same interval is correlated with database CPU, memory, I/O, connection count, rows scanned and replication lag.
- Actual query and plan: the real SQL with representative parameters goes through the engine’s plan tooling, such as PostgreSQL
EXPLAIN (ANALYZE, BUFFERS). Estimated and actual rows, scans, joins, spills and I/O show where the work goes.- Smallest demonstrated fix: typical fixes return fewer rows or columns, remove an N+1 path, add or correct an index, shorten a transaction, or change the query or schema shape. The same trace and load are then rerun to confirm the effect.
- Pool check: connection waits while the database is healthy point to leaks, long checkouts or fleet-wide pool sizing. A larger pool does not repair saturated storage or lock contention.
- Read offloading: a cache, or a read model refreshed asynchronously from the source, fits only when repeated reads tolerate a declared staleness window. A materialized view that the database updates as part of each base-table write, such as a SQL Server indexed view, stays current but makes those writes more expensive. Read replicas fit when reads dominate and replica lag is acceptable.
- Capacity changes: the node is scaled after the query path is sound. Partitioning or sharding is justified only when one node’s measured capacity, data size or recovery boundary is still the limit.
Jumping straight to caches, replicas or shards can improve one graph while adding stale reads or duplicated writes that hide the original defect.
PostgreSQL: Using EXPLAIN · SQL Server: Create indexed views
Why might Elasticsearch map one logical value as both
textandkeyword?Full-text search and exact-value operations need different index forms, and one field type serves only one of them well. A multi-field indexes the same source value both ways, so each query can use the form that matches its intent.
textpasses the value through an analyzer that splits and normalizes it into terms. A standard analyzer may indexRunning Shoesasrunningandshoes, so a search forshoesmatches. The index holds separate normalized terms instead of the whole original value, sotextis a poor basis for exact filters, sorting and aggregations.keywordindexes the whole value unchanged. A term filter, a sort or an aggregation such as a facet count works on the exact valueRunning Shoes. A full-text search forshoesdoes not match it.{ "properties": { "name": { "type": "text", "fields": { "raw": { "type": "keyword" } } } } }With this mapping, full-text queries target
name, while exact filters, sorting and aggregations targetname.raw.
Why can a newly indexed Elasticsearch document be missing from search results for a short time?
Elasticsearch search is near real-time. A document becomes searchable only after a refresh opens the segment that contains it, so an acknowledged write can be stored before it appears in search results.
- Index: the document enters an in-memory indexing buffer and the transaction log on its shard. Acknowledgement depends on the primary and replica write settings, not on search visibility.
- Refresh: on a periodic interval, or when explicitly requested, buffered documents become a new immutable Lucene segment that is opened for search.
- Merge: background merges combine small segments and discard obsolete document versions.
A shorter refresh interval reduces the delay but creates more small segments and more merge work. That work consumes CPU, disk bandwidth and temporary disk space. Forcing a refresh after every write is therefore a costly default and fits only bounded workflows that truly need immediate visibility.
Elasticsearch near-real-time search · Elasticsearch refresh parameter · Elasticsearch translog settings
When is a non-relational store a poor fit for a service?
A non-relational store is a poor fit when the service’s correctness depends on guarantees that the chosen engine cannot provide within the required scope. Those guarantees are constraints, transactions across related records, or support for a changing set of queries.
- Invariants move into application code. If the database cannot enforce a rule across records, the application must enforce it, coordinate concurrent writes and repair copies that drift apart. For example, a transfer between two account records needs both balance changes to commit or fail together. An engine whose transactions are limited to one document or partition provides that atomicity only when both accounts lie inside one such scope. Under a partition-scoped transaction, two separate account documents in the same partition can still commit together. When the accounts fall outside every atomic scope the engine supports, such as in different partitions, the application must coordinate the transfer itself.
- New queries can require remodeling. Data in such stores is often shaped around known reads. A query the model was not designed for may need another index, a duplicated record or a different layout.
- The label decides little. “NoSQL” names no single consistency or transaction model. Some engines provide strong reads or transactions within a bounded scope, while others offer eventual or tunable consistency. The fit therefore depends on the concrete engine and the scope of each operation.
Microsoft: Understand data store models · Azure Cosmos DB: Database transactions and optimistic concurrency
What lifetime should an EF Core
DbContexthave, and why?An EF Core
DbContextshould live for one unit of work and serve one operation at a time. It is a session that tracks entities and coordinatesSaveChangesfor that unit of work. It is also not thread-safe, and EF Core does not support parallel operations on one instance.
- Web requests: in ASP.NET Core,
AddDbContextregisters the context as scoped by default, so each request usually gets its own instance. One request can still contain several separate units of work.- Background or parallel work: each task needs its own fresh context, typically created through
IDbContextFactory<TContext>, rather than a shared request instance.- Singletons: a singleton that captures a scoped context keeps one instance alive across requests, so concurrent requests can end up sharing it. This is the captive-dependency failure.
- Context pooling:
AddDbContextPoolreuses reset context instances to lower setup cost. Per-request state, such as a tenant identifier, must therefore be re-established safely on each use.EF Core DbContext lifetime, configuration, and initialization · EF Core advanced performance topics and context pooling
How does EF Core change tracking work, and when is a no-tracking query appropriate?
EF Core change tracking records the entities that a
DbContextloads, adds or attaches. It then compares their current values with remembered original values to decide whatSaveChangesmust write. A no-tracking query fits results that are only read and will not be updated through that context.With the default snapshot strategy, tracking works in four steps:
- Track: a tracking query, the default for entity queries, attaches each returned entity to the context and keeps a snapshot of its original values. If an entity with the same key is already tracked, EF Core returns that existing instance instead of a new object. This is identity resolution.
- Modify: application code changes ordinary properties. Under this strategy, assigning a property does not notify EF Core.
- Detect: before
SaveChangeswrites, automatic change detection compares current values with the snapshot and marks changed entities and properties as modified.- Save:
SaveChangessends the inserts, updates and deletes that the tracked entity states require.
.AsNoTracking()skips setting up this tracking information, which usually makes read-only queries cheaper. In exchange,SaveChangesneither detects nor saves changes to the returned objects. A plain no-tracking query also skips identity resolution, so one row returned twice becomes two separate objects. Later use of such a detached object for an update requires attaching it, stating which entity or properties are modified and handling concurrency. The context never recorded the object’s original values.EF Core: Tracking vs. no-tracking queries · EF Core: Change detection
How does EF Core detect that another writer changed a row between reading it and saving an update?
EF Core detects the conflict through optimistic concurrency. The original value of a configured concurrency token is added to the
WHEREclause of theUPDATEorDELETE. A command that then affects zero rows raisesDbUpdateConcurrencyException. This detection does not require a database lock to span the read, the application’s work and the save. Whether any lock is held during that interval depends separately on the enclosing transaction and its isolation level.
- Read: EF Core loads the row and remembers the token’s original value. In SQL Server, a
rowversioncolumn is a common token because the database changes it on every update.- Save: the generated command changes the row only if its key and token still equal the values that were read.
- Conflict: if another writer updated the row in between, the token differs and the predicate matches nothing. EF Core treats the zero-row result as a conflict and throws
DbUpdateConcurrencyException.-- Simplified shape of the generated command UPDATE Orders SET Status = @newStatus WHERE Id = @id AND Version = @originalVersion;The application then resolves the conflict. It reloads and merges values, retries the operation from fresh state, or reports the conflict.
Limit: the check covers only the rows that the command updates or deletes. It does not detect a concurrent change to other rows that a decision relied on, such as a newly inserted overlapping booking. Protecting such a predicate needs a stronger mechanism, such as Serializable isolation.
What is the N+1 query problem, and how can it be detected?
The N+1 query problem occurs when one query loads N parent rows and code then triggers a separate query for each parent’s related data. The result is N + 1 round trips instead of one or two. In EF Core it commonly comes from lazy loading: accessing a navigation inside a loop can issue a query per item.
var orders = await db.Orders.ToListAsync(); // 1 query foreach (var order in orders) Console.WriteLine(order.Customer.Name); // with lazy loading: can query once per orderEach extra query adds a round trip, so the cost grows with the number of parents. A page that seems fast with little test data can degrade as production data grows.
Detection:
- Command count per operation: counting the database commands that one request or test issues reveals N+1 when the count grows with the number of parent rows.
- Logged SQL: the log shows the same statement repeated many times, with only the key parameter changing.
- Traces: one request contains many short, sequential database spans.
Fixes reduce the round trips. Options include a projection that selects only the related columns the result needs, an
Includethat eager-loads the navigation, or a deliberate second query that loads the related rows for all parents at once. The right choice depends on the result shape and consistency needs. No single collection-size threshold decides it.
When is raw SQL a better boundary than an ORM query?
Raw SQL is a better boundary when a query needs something the ORM cannot express, such as a provider-specific database feature. It is also a better boundary when the SQL the ORM generates has a measured performance problem that cannot be fixed predictably through the ORM. Raw SQL is not automatically faster. The reason to use it is a missing capability or a measured plan problem.
Moving a query into SQL text brings its own obligations:
- Data values: values are passed as database parameters, never concatenated into the SQL text. In EF Core,
FromSqlturns interpolated values into parameters (EF Core 7.0 and later; earlier versions useFromSqlInterpolated). AFromSqlRawstring built by concatenating values reintroduces SQL injection risk.- Identifiers and SQL fragments: databases do not accept column names, table names or other parts of the SQL structure as parameters. Any dynamic piece is checked against an allow-list of known values before it enters the SQL text.
- Provider dependence: the SQL is written for one database engine, so its tests run against that same engine.
Why can eager-loading sibling collections be expensive?
Loading two collections of the same parent in one SQL query joins both of them to the parent. Each parent row is then repeated for every combination of its matching children, so the row count multiplies instead of adding up. EF Core’s documentation calls this a cartesian explosion.
For example,
Include(b => b.Posts).Include(b => b.Contributors)for one blog with 10 posts and 10 contributors returns 10 × 10 = 100 rows for that blog. Each row repeats the blog’s columns together with one post and one contributor. The database, the network and EF Core all process the extra rows.Two mitigations trade this cost for others:
- Split query (
AsSplitQuery()): EF Core loads the blogs and each collection with separate SQL commands, returning about 1 + 10 + 10 rows. The cost is extra round trips. The commands also run at different moments, so an update between them can produce a combined result that never existed at a single point in time. Running them in a serializable or snapshot transaction gives one consistent view, at the cost of that isolation level’s own overhead.- Projection: selecting only the needed columns makes each duplicated row smaller. It does not reduce the number of rows the join produces.
What problem does normalization solve in a relational schema?
Normalization prevents insertion, update and deletion anomalies. These arise when one fact is stored redundantly in rows that exist for other reasons. Normalization decomposes tables according to their dependencies, so each fact is stored once, with the key that actually determines it.
For example, an
Orderstable that is the only place storing customer addresses repeats a customer’s address on every order row. It has all three anomalies:
- Update anomaly: changing the address means updating every order row for that customer. Missing one row leaves two contradictory addresses.
- Insertion anomaly: a new customer’s address cannot be recorded until the customer places an order.
- Deletion anomaly: deleting a customer’s last order also deletes the only record of the address.
Storing the address in a
Customerstable keyed by customer ID, with each order referring to that ID, removes all three. The goal is not to remove every repeated value: the customer ID repeated on each order is a reference, not a duplicated fact. The address an order was shipped to is a different, historical fact, and it belongs on the order.A valid decomposition is lossless: joining the smaller tables reproduces exactly the original rows, without losing any or inventing new ones. The constraints the business needs must also remain enforceable after the split.
Codd: Further Normalization of the Data Base Relational Model
When does a relational schema need 4NF or 5NF rather than stopping at BCNF?
A schema needs 4NF or 5NF when redundancy remains that no functional dependency explains. 3NF and BCNF are built on functional dependencies, where one set of attributes determines a single value of another attribute. Fourth and fifth normal form handle two other kinds of dependency.
Aspect 4NF 5NF (project-join normal form) Dependency removed A multivalued dependency: one value determines a set of values that is independent of the table’s other attributes. A join dependency: the full fact is exactly the join of three or more smaller projections, and this is not implied by the candidate keys. Example A {Restaurant, PizzaType, DeliveryArea}table whereRestaurant ↠ PizzaTypeandRestaurant ↠ DeliveryAreaare independent. Every pizza type repeats for every area. Adding one type needs one row per area, and a missing row falsely says the pizza is unavailable there.A {Supplier, Product, Customer}table under the rule that a supplier supplies a product to a customer exactly when the supplier offers the product, serves the customer, and the customer buys the product.Decomposition {Restaurant, PizzaType}and{Restaurant, DeliveryArea}.SupplierProduct,SupplierCustomerandProductCustomer.Both decompositions are valid only when the dependency really holds in the domain. If pizza availability differs by delivery area, the two sets are not independent, and the three-column table is the correct fact. If a supplier offers a product only to some of its customers, joining the three pairwise tables produces combinations that never existed, so the decomposition loses information. The design follows from the dependencies that actually hold, not from a target normal-form number.
Codd: Further Normalization of the Data Base Relational Model · Fagin: Multivalued Dependencies and a New Normal Form for Relational Databases · Fagin: Project-Join Normal Form
What conditions justify denormalizing a table, and what risks does it introduce?
Denormalization is justified when three conditions hold:
- A specific read path has a measured latency or resource problem.
- Cheaper fixes, such as an index or a corrected query plan, do not meet its target.
- Keeping a stored copy up to date costs less than recomputing the result on every read.
High read volume alone does not justify it.
The risk is that the same fact now lives in two places, and every change must reach both:
- Drift: a write path that updates the source but misses the copy leaves them silently inconsistent.
- Staleness: a copy updated asynchronously or on a schedule serves old data between updates. A PostgreSQL materialized view, for example, changes only when
REFRESH MATERIALIZED VIEWruns.- Write cost and contention: each source change also updates the copy. A stored
TotalOrderAmountonCustomersadds an update to every order write, and concurrent orders for the same customer contend on that one row.A safe design therefore names four things:
- the authoritative source;
- the acceptable freshness;
- how the copy is updated, either in the same transaction or by an asynchronous projection;
- a reconciliation check that detects and repairs drift.
How do 2NF and 3NF differ?
2NF removes facts that depend on only part of a composite key. 3NF also removes facts that depend on another non-key attribute instead of on a key. Every relation in 3NF is also in 2NF.
A non-prime attribute is one that is not part of any candidate key.
Aspect 2NF 3NF Problem removed Partial dependency: a non-prime attribute depends on only part of a composite candidate key. Transitive dependency: a non-prime attribute depends on another non-key attribute, which in turn depends on the key. Formal condition The relation is in 1NF, and every non-prime attribute depends on the whole of every candidate key. For every nontrivial functional dependency X → A,Xis a superkey orAis prime.When it can be violated For a table already in 1NF, only when a candidate key has several attributes; such a table whose candidate keys are all single attributes is automatically in 2NF. Also when the key is a single attribute. Example violation Key {Make, Model}:Discountdepends only onMake, so the same discount repeats on every model row for that make.Key Model:Model → StoreandStore → Phone, so each store’s phone number repeats on every model row for that store.Fix A separate {Make, Discount}table holds each make’s discount.A separate {Store, Phone}table holds each store’s phone number.Each fix is a lossless decomposition: joining the new tables on the shared attribute reproduces the original rows exactly. Whether every original dependency can still be enforced within a single table is a separate design concern.
Codd: Further Normalization of the Data Base Relational Model
In what logical order are the clauses of a SQL
SELECTquery evaluated, and why does that order matter?Logically, a
SELECTquery is evaluated from its input outward: rows are built, filtered, grouped, filtered again as groups, projected into output columns, then sorted and limited. Each step works on the output of the previous steps, so a clause can use only what earlier steps produced.
FROMand joins build the input rows.WHEREremoves rows that do not satisfy its condition.GROUP BYforms groups and computes aggregates.HAVINGremoves groups.SELECTcomputes the output columns and their aliases.DISTINCTremoves duplicate output rows.ORDER BYsorts the output.LIMITandOFFSETkeep a slice of the sorted rows.Several consequences follow from this order:
WHEREcannot test an aggregate of its own query block, because that block’s groups do not exist yet. A subquery insideWHEREcan still compute its own aggregate for the comparison.WHEREandHAVINGcannot refer to an output alias defined inSELECT, because that alias is computed later.ORDER BYruns afterSELECT, so it can.LIMITwithoutORDER BYreturns an unpredictable subset of rows, because no order has been defined.Alias rules vary by dialect. PostgreSQL also accepts an output column name in
GROUP BY, while SQL Server requires the original expression there, so portable SQL repeats the expression. The order is logical: the optimizer can execute the query differently, as long as the result is the same.
What is the difference between
WHEREandHAVING?
WHEREfilters individual rows before they are grouped.HAVINGfilters the groups thatGROUP BYproduces, so its condition can test aggregate values such asCOUNT(*)orSUM(amount).SELECT department, COUNT(*) AS headcount FROM employees WHERE hire_date >= DATE '2024-01-01' -- keeps recent hires only GROUP BY department HAVING COUNT(*) > 5; -- keeps departments with more than five recent hires
Aspect WHEREHAVINGWhat it filters Input rows of the query, before grouping. Groups, after grouping and aggregation. Aggregates in the condition Cannot use an aggregate of its own query, because no groups exist yet. A subquery inside WHEREcan still compute its own aggregate.Can use aggregates; the condition is evaluated once per group. Effect on later work Rows it removes never reach grouping or aggregation. Groups are built and aggregated before it removes any. A condition on a grouping column, such as
department = 'Sales', gives the same result in either clause. Written inWHERE, it expresses the cheaper logical order, because rows are removed before any grouping work.
In a
LEFT JOIN, why does putting a condition on the right table inWHEREinstead ofONchange the result?A condition in
ONdecides which right-side rows match. Left rows without a match are still kept, withNULLin the right-side columns. A condition inWHEREruns after the join and keeps only the rows for which it is true. An ordinary comparison such aso.total >= 50is not true for thoseNULL-extended rows, soWHEREremoves them, and the query behaves like an inner join for that condition.In this example,
customersholds Ada and Lin. Ada has orders with totals 40 and 70, and Lin has none.-- Condition in ON: every customer is kept SELECT c.name, o.total FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id AND o.total >= 50; -- Ada | 70 -- Lin | NULL -- Condition in WHERE: applied after the join SELECT c.name, o.total FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id WHERE o.total >= 50; -- Ada | 70In the second query, the join first produces a Lin row with a
NULLtotal.NULL >= 50evaluates to unknown rather than true, soWHEREremoves Lin.A right-side condition that is true for
NULLbehaves differently. With the sameON o.customer_id = c.idjoin,WHERE o.customer_id IS NULLis true only for theNULL-extended row, so it keeps Lin and returns only customers without orders.The
ONform answers ‘all customers, with their large orders if any’. TheWHEREform answers ‘customers who have a large order’. The correct placement depends on which of those questions the query is meant to answer.
In SQL Server, how do stored procedures and user-defined functions differ?
A stored procedure is a database operation called as its own statement; it can change data and return several kinds of results. A T-SQL user-defined function (UDF) computes a value for use inside a query, and SQL Server forbids it from changing database state.
Aspect Stored procedure T-SQL user-defined function How it is called As a statement with EXEC. It cannot be used inside aSELECTexpression.Inside a query: a scalar UDF in an expression such as a SELECTlist orWHEREcondition, a table-valued UDF inFROM.What it returns Any number of result sets, output parameters and an integer return status. One scalar value or one table. Changing data Can insert, update and delete data and call other procedures. Cannot modify database state; only table variables declared inside the function can change. Other restrictions Can use dynamic SQL, temporary tables and TRY...CATCH.Cannot use dynamic SQL, temporary tables or TRY...CATCH, and cannot call ordinary stored procedures.Scalar UDFs carry a performance risk. Without inlining, SQL Server invokes a T-SQL scalar UDF once per qualifying row and does not run the calling query in parallel. In SQL Server 2019 (15.x) and later, at database compatibility level 150 or higher, the optimizer can inline an eligible scalar UDF by turning its body into part of the calling query’s plan. Inlining applies only when the function and the calling query meet the documented eligibility requirements.
SQL Server: CREATE PROCEDURE · SQL Server: Create user-defined functions · SQL Server: Scalar UDF inlining
In SQL Server, when is a temp table a better fit than a CTE?
A temp table is a better fit when an intermediate result is expensive to compute and is needed more than once. It also fits when later steps need indexes or statistics on that result to be planned well. A CTE is a better fit for structuring a single statement, because it adds no storage or cleanup.
Aspect CTE Temp table Scope A named result for the one statement that immediately follows it. A real table that later statements can query until it is dropped or its session or creating procedure ends. Materialization SQL Server does not promise to store the result. If the outer query references the CTE several times, its query can run again for each reference. Rows are computed once and stored, so later reads reuse them. Optimization Optimized together with the statement that uses it. Can be indexed, and SQL Server keeps statistics on it that later queries’ plans can use. Inspection The intermediate result cannot be examined on its own. The stored rows can be queried directly, which helps debugging. Cost No separate write or cleanup. Writing the rows, maintaining any indexes and dropping the table add work. The rows are a copy, so later changes to the source tables do not appear in them. Neither option is faster by default. A temp table pays off when reuse, indexing or better row estimates save more work than writing the rows costs.
How do SQL Server's transaction isolation levels differ in the read anomalies they allow?
SQL Server provides five isolation levels. From
READ UNCOMMITTEDtoSERIALIZABLE, each lock-based level prevents more read anomalies at the cost of more blocking.SNAPSHOTprevents the same read anomalies asSERIALIZABLEby reading row versions instead of holding read locks.The anomalies are:
- Dirty read: reading data another transaction has changed but not committed.
- Nonrepeatable read: reading the same row twice in one transaction and getting different committed values.
- Phantom read: re-running a range query in one transaction and getting a different set of rows because another transaction inserted or deleted rows.
Level How reads behave Read anomalies still possible Main cost READ UNCOMMITTEDReads take no shared data locks and do not wait for writers’ exclusive data locks. Dirty, nonrepeatable and phantom reads. Results can include changes that are later rolled back. READ COMMITTED(default)Reads see only committed data. With locking, a read waits for writers and releases its shared lock soon after reading. Nonrepeatable and phantom reads. Readers and writers block each other. REPEATABLE READShared locks on rows read are held until the transaction ends. Phantom reads. Longer-held locks increase blocking and deadlocks. SERIALIZABLERange locks also stop other transactions from inserting into ranges already read. None of the three. The most blocking and deadlock risk. SNAPSHOTReads see committed data as of the start of the transaction, using row versions instead of shared data locks, so reads do not block writers’ data changes. None of the three. Row versions add overhead. An update to a row that another transaction changed after the snapshot began fails with an update conflict. The level must be enabled with the ALLOW_SNAPSHOT_ISOLATIONdatabase option.The
READ_COMMITTED_SNAPSHOTdatabase option changes howREAD COMMITTEDreads behave, but it is not a separate isolation level.
What changes about
READ COMMITTEDreads when SQL Server's Read Committed Snapshot Isolation (RCSI) is enabled?With RCSI enabled, a
READ COMMITTEDread no longer takes shared locks. It reads row versions instead, and each statement sees the data that was committed when that statement started. For data locks, readers therefore stop blocking writers, and writers stop blocking readers.
- How it is enabled: RCSI is the
READ_COMMITTED_SNAPSHOTdatabase option. Sessions that useREAD COMMITTED, the default level, get the new behavior without code changes.- Statement-level, not transaction-level: each statement takes a fresh view. Two statements in the same transaction can still see different committed data, so nonrepeatable and phantom reads remain possible.
SNAPSHOTisolation, by contrast, keeps one view for the whole transaction.- Writes still lock: data modifications still take locks, so writers still block other writers.
- Cost: maintaining row versions adds storage and I/O overhead.
A read no longer waits for an in-flight write. It can therefore return the last committed value of a row that another transaction is about to change. A read-then-write sequence that relied on that waiting needs explicit concurrency protection.
In SQL Server, why is the
NOLOCKhint unsafe as a general performance fix?
NOLOCKmakes the read of that table behave likeREAD UNCOMMITTED. For data, the read takes no shared locks and is not blocked by other transactions’ exclusive locks. It avoids waiting on concurrent writers only by giving up the guarantee that the result is correct. The hint covers only data locks: the query still takes schema-stability locks, so a concurrent schema change can still block it.While data changes concurrently, a
NOLOCKread can return:
- Uncommitted data: values written by a transaction that later rolls back, so the result contains data that was never committed.
- Missing or duplicate rows: the scan takes no shared data locks, so rows that concurrent changes move while it runs can be skipped or read twice. A page split is one way rows move.
These anomalies need not raise an error. A scan can sometimes fail when data moves under it, but it can also finish normally with a wrong total, a missing record or a double-counted row, and nothing signals the problem.
It is also not a general speed-up. A query that is slow because of its own work, such as a large scan or a missing index, does the same work under
NOLOCK. When the real problem is readers waiting on writers’ data locks, row versioning throughREAD_COMMITTED_SNAPSHOTorSNAPSHOTremoves that waiting while still returning committed data.SQL Server: Transaction locking and row versioning guide · SQL Server: Table hints
What evidence justifies sharding a database?
Sharding is justified by measured evidence that one database primary cannot handle the workload’s write throughput or storage, after cheaper options have failed to remove that limit. A hard requirement, such as isolating tenants or keeping data in particular locations, can justify it earlier.
For scale-driven sharding, the relevant ceiling is write throughput or storage, because each cheaper option addresses a different limit:
- Query and index repair: removes wasted work, which may be the actual bottleneck.
- Vertical scaling: adds CPU, memory or I/O to the same server.
- Read replicas: add read capacity, but in a single-primary setup every write still goes to one server.
- Caching: serves repeated reads without reaching the database.
- Partitioning inside one database: splits a table into partitions that stay within one database and one transaction domain, so it does not spread work across independent servers.
The bar is high because sharding adds lasting costs:
- Queries that need data from several shards must fan out and merge results.
- Transactions and constraints that span shards need extra coordination.
- Data must be moved when shards are rebalanced.
What makes a shard key viable for a workload?
A shard key is viable when four conditions hold:
- The workload’s common operations include it.
- It spreads both data and traffic evenly.
- Data that must change atomically shares the same key value.
- Its values rarely change.
Property Why it matters Failure when it is missing Present in common operations The router can send each operation to the one shard that owns the key. Common queries without the key must fan out to every shard and merge the results. Even spread of data and load Each shard gets a similar share of storage and requests. Hot shards: skewed values, sequential ranges or one very large tenant overload a single shard even when total cluster capacity is sufficient. Keeps atomic changes together Rows that must change in one transaction live on one shard, so a local transaction is enough. Those changes need cross-shard coordination or a multi-step workflow. Stable values Rows avoid moves caused by a change to their own key value. Ownership can still move when the key-to-shard mapping or shard topology changes, such as during rebalancing. A changed key value can map to a different shard. When it does, the row must migrate across shards instead of being updated in place. For example,
tenant_idcan fit a multi-tenant service well: most requests carry it, a tenant’s data usually changes together, and it does not change. It still fails if one tenant outgrows a single shard.
How do lookup, range and hash sharding strategies differ?
All three strategies map a shard key to the shard that owns the data. They differ in what decides that mapping, which in turn decides load balance, range-query efficiency and how hard rebalancing is.
Aspect Lookup (directory) Range Hash How a key is mapped A stored map records which shard owns each key, tenant or virtual shard. Contiguous key ranges belong to shards, such as customer IDs 1–9999on one shard.A hash of the key determines the shard, so no map has to be stored. Load balance Placement is explicit, so large tenants can be placed deliberately. Can be uneven. Sequential keys, such as timestamps or increasing IDs, send new writes to the newest range and create a hotspot. Usually spreads data and load evenly, although one very hot key still lands on a single shard. Range queries No inherent locality; it depends on how the map groups keys. Efficient, because neighboring keys sit on the same shard. Neighboring keys scatter, so a range query fans out to many shards. Rebalancing Data is moved and its map entries are updated. With keys mapped to virtual shards, a virtual shard can move between physical servers without changing application code. Moving or splitting a range moves its data. Rebalancing may not fix load that concentrates on adjacent keys. Difficult. With a simple hash(key) % N, changing the number of shards changes the owner of most keys.Routing overhead A map lookup on every routed request. A comparison of the key against stored range boundaries. Computing the hash.
What is the practical difference between a Layer 4 and a Layer 7 load balancer?
A Layer 4 load balancer chooses a backend once per transport connection, using connection details such as IP addresses, ports and protocol. A Layer 7 load balancer terminates and parses the application protocol, usually HTTP, so it can route each request by its content.
Aspect Layer 4 Layer 7 What it can see Connection details only. It does not parse HTTP, so a request’s host, path and headers are invisible to it. The parsed request, including its host, path and headers. Unit of balancing The connection. Every request sent on that connection reaches the same backend for the connection’s lifetime. The individual request. Requests from one client connection can reach different backends. Routing rules Limited to connection details, so path-based or header-based routing is not possible. Content-based rules, such as sending /api/*to one group of backends and other paths to another.Cost Forwards traffic without parsing the application protocol, so it keeps little protocol state. Terminating and parsing HTTP adds processing, protocol state and more ways to fail. The unit of balancing matters for multiplexed protocols such as HTTP/2 and gRPC, where one long-lived connection carries many requests. An L4 balancer sends all of those requests to one backend. An L7 balancer that terminates HTTP/2 can spread them across backends.
Request awareness does not make retries safe. When an L7 proxy is configured to retry failed requests, a retry is safe only if the application tolerates the same request being replayed.
AWS: How Elastic Load Balancing works · AWS: Application Load Balancers · RFC 9110: Idempotent methods
How does the OSI model map onto the TCP/IP stack?
In the common four-layer TCP/IP view, OSI layers 5–7 are grouped as application, layer 4 maps to transport, layer 3 to Internet, and layers 1–2 to link. This is a teaching and diagnostic mapping, not a strict protocol-by-protocol equivalence: real protocols can span or blur those responsibilities. RFC 1122: Internet architecture · ISO/IEC 7498-1: OSI reference model
What do a MAC address, an IP address and a port each identify?
A MAC address identifies a network interface on its local link, such as an Ethernet or Wi-Fi network. An IP address identifies a host’s interface across interconnected networks, and routers forward packets by it. A port identifies an endpoint, such as a listening application, inside one host for a transport protocol such as TCP.
The three work together when a packet travels to a server:
- The sender addresses the IP packet to the server’s IP address and the TCP segment inside it to the server’s port.
- On each hop, the packet travels inside a link-layer frame for that link. On an Ethernet or Wi-Fi link, the frame is addressed to the MAC address of the next device on that link, usually a router.
- Each router reads the destination IP address, chooses the next link and places the packet in a new frame for that link. When the next link also uses MAC addressing, the new frame carries new MAC addresses.
- On the destination host, TCP uses the destination port to deliver the data to the right socket.
A MAC address therefore matters only within one link, while the IP address guides the packet across networks. A port number is meaningful only within one host’s, or one network namespace’s, port space for that transport protocol.
Which values identify a TCP connection?
A TCP connection is identified by four values: the local IP address and port, and the remote IP address and port. Each IP address and port pair is a socket, and the pair of sockets distinguishes one connection from every other TCP connection in the host, or network namespace, that holds it. The combination is often called the 4-tuple.
Because the remote address and port are part of the identity, one local socket can take part in many connections. A server listening on port 443 can hold many connections at once, because each differs by client IP address or client port. For the same reason, one client can open several connections to the same server port by using different local ports.
How does HTTP/2 differ from HTTP/1.1?
HTTP/2 keeps HTTP’s semantics but changes how messages travel on a connection. It replaces HTTP/1.1’s text messages with binary frames, carries many concurrent exchanges as streams on one connection and compresses header fields.
Aspect HTTP/1.1 HTTP/2 Message format Text messages sent one after another on the connection. Binary frames, each tagged with a stream ID, so frames from different exchanges can be interleaved. Concurrency Without pipelining, one request at a time per connection. Pipelining allows several outstanding requests, but responses must return in request order, so clients commonly open several connections to deliver responses independently. Many concurrent streams on one connection, and responses need not follow request order, so a client normally needs only one connection per origin. Header fields Sent as text in full with every message, so repeated fields such as cookies are resent each time. Compressed with HPACK, so repeated fields cost fewer bytes. Semantics Defined by HTTP’s methods, status codes and fields. Unchanged, so applications and APIs keep the same meaning for every request and response. Starting the protocol Needs no negotiation and is the fallback when HTTP/2 is not agreed. Negotiated: over TLS, both sides agree on h2through ALPN during the handshake.Both versions run over TCP, so packet loss on an HTTP/2 connection can still delay all of its streams until TCP recovers.
What does HTTP/2 multiplexing improve over HTTP/1.1, and what head-of-line blocking remains?
HTTP/2 multiplexing removes HTTP/1.1’s response-order head-of-line blocking: a slow response no longer has to finish before later responses on the same connection can be delivered. TCP-level head-of-line blocking remains, because all streams still share one ordered TCP byte stream.
- What improves: an HTTP/1.1 connection returns responses in request order. Without pipelining, the next request waits for the previous response. With pipelining, several requests can be outstanding, but a slow first response still delays the responses behind it. HTTP/2 splits requests and responses into frames tagged with a stream ID and interleaves frames from many streams on one connection, so responses no longer have to arrive in request order.
- What remains: TCP delivers bytes to the receiving application strictly in order. When one TCP segment is lost, bytes that arrive after it are held back until the retransmission fills the gap, even if they belong to other streams. A lost packet can therefore stall every stream on the connection until TCP recovers.
HTTP/2 removes this ordering dependency, not every interaction between streams. Streams still share connection-level flow control, and each block of header fields must be sent as a contiguous run of frames, so one stream can still slow another.
RFC 9113: HTTP/2 · RFC 9112: HTTP/1.1 pipelining · RFC 9293: TCP
When can HTTP/1.1 be necessary or perform better than HTTP/2?
HTTP/1.1 is necessary when some hop on the path cannot negotiate HTTP/2. It can perform better on a lossy path, because with several HTTP/1.1 connections the delay caused by a lost packet stays on one connection. It is not generally faster, so the choice rests on measuring both versions on the real path.
- Compatibility: HTTP/2 is used on a hop only when both ends agree to it, for example through ALPN during the TLS handshake. If the client, server or intermediary at either end of a hop supports only HTTP/1.1, that hop uses HTTP/1.1.
- Loss isolation: HTTP/2 normally carries all requests to an origin on one TCP connection, so one lost packet can delay every stream until TCP retransmits it. With several HTTP/1.1 connections, the same loss delays only the requests on the affected connection.
- Cost of extra connections: pipelining lets one HTTP/1.1 connection carry several outstanding requests, but responses must return in request order. Delivering responses independently of each other therefore takes several connections. Each needs its own TCP setup, plus TLS setup when encrypted, and its own state on clients, servers and intermediaries. Connections that share a path also compete for its capacity, and opening more of them can add to congestion.
What does HTTP idempotency guarantee about retrying a request?
An idempotent HTTP method guarantees that sending the same request several times has the same intended effect on the server as sending it once. A client can therefore resend such a request after a communication failure, for example when the connection closes before the response arrives, even though the first attempt may already have succeeded.
PUT,DELETEand the safe methods, such asGETandHEAD, are idempotent.POSTis not.The guarantee is narrower than identical behavior:
- Responses can differ. A retried
DELETEleaves the resource deleted, but the second response can report that the resource no longer exists.- Only the requested effect is covered. The server can still log, audit or record history separately for each copy of the request.
- Method semantics do not make a particular retry harmless. The request body must be replayable, the retry must fit within the caller’s deadline and credentials must still be valid. A request that carries a single-use value, such as a one-time token, cannot simply be replayed.
For a non-idempotent method such as
POST, an automatic retry is appropriate only when the client knows the operation is idempotent for that resource or can detect that the first attempt was never applied.
What is WebSocket, and how is a WebSocket connection opened?
WebSocket is a protocol for full-duplex, message-based communication over one long-lived connection: once the connection is open, either side can send a message at any time. It starts with an HTTP handshake, which lets it use the usual web ports and work with HTTP proxies and other intermediaries.
The HTTP/1.1 opening handshake defined by RFC 6455 works as follows:
- The client sends an HTTP
GETrequest withUpgrade: websocket,Connection: Upgradeand a randomSec-WebSocket-Key.- A server that accepts replies
101 Switching Protocolswith aSec-WebSocket-Acceptvalue computed from that key. The value shows that the server understood the request as a WebSocket handshake rather than ordinary HTTP.- From then on, the same TCP connection carries WebSocket frames instead of HTTP messages. Data frames carry text or binary messages, and control frames carry ping, pong and the closing handshake.
ws://runs over plain TCP andwss://runs over TLS.The protocol provides framing and two-way delivery. It does not define request–response correlation, reconnection or resuming after a disconnect, so applications that need them build them on top.
What conditions make Server-Sent Events a better fit than WebSockets?
Server-Sent Events (SSE) fit better when updates flow mainly from server to browser, client actions can travel as ordinary HTTP requests, and built-in reconnection with a resume cursor is useful. WebSockets fit better when both sides need to send messages over the same long-lived connection, or when messages are binary.
Aspect Server-Sent Events WebSockets Direction One-way: the server sends an ordered stream of events in one long-lived HTTP response. Client actions use separate HTTP requests. Full duplex: either side can send messages at any time on one connection. Message format UTF-8 text events, so binary data has to be encoded as text. Text or binary messages. Reconnection The browser’s EventSourcereconnects automatically. If the stream has given it a nonempty event ID, the reconnection request sends that ID in aLast-Event-IDheader.Not part of the protocol. The application implements reconnection, usually with backoff. Resuming after a disconnect Works only if the server tags events with IDs, keeps recent events and replays those after Last-Event-ID. Clients still have to tolerate duplicates or gaps.The application defines its own resume tokens, replay and acknowledgements. Connection setup An ordinary HTTP request and response. Intermediaries can drop a quiet stream, so servers often send periodic comment lines to keep it active. Opens with an HTTP handshake, then exchanges WebSocket frames instead of HTTP messages. Neither protocol guarantees a particular latency. Delivery time still depends on the server, the network and any intermediaries.
What factors determine whether a system should use a monolith, a modular monolith, or microservices?
The choice depends on whether real boundaries repeatedly need independent deployment, scaling or ownership, and whether the system can carry the cost of distribution. Without that pressure, one deployment is usually the better fit. A modular monolith adds clear internal boundaries without the cost of a distributed system.
Aspect Monolith Modular monolith Microservices Deployment One deployable unit, so every change ships with the whole application. Still one deployable unit, so all modules release together. Each service deploys on its own, so one service can change without redeploying the others. Internal boundaries Code can reach across areas directly, so boundaries rely on discipline. Module boundaries are explicit and enforced, so modules use each other only through public contracts. Boundaries are service APIs or messages over the network. Calls, transactions and data In-process calls over a shared data layer; one local transaction can cover a whole operation. The same in-process calls and local transactions as a monolith. Each service owns its data. Calls cross the network and can fail or slow down. No single local transaction spans services, so consistency between them is usually eventual. Scaling The whole application scales as one unit. The whole application scales as one unit. Each service scales according to its own load. Operational cost Lowest: one build, deployment and runtime to monitor. Close to a monolith, plus the effort of keeping boundaries enforced. Highest: many deployments, network failure handling, contract versioning and monitoring across services. Microservices fit when stable boundaries repeatedly need independent releases, independent scaling or separate team ownership, and the organization can operate the resulting distributed system. System size or expected growth alone does not justify the split.
Microservices architecture style (Azure Architecture Center)
When is simple layering enough for a small CRUD service, and when is Clean Architecture worth it?
Simple layering is usually enough when the service mainly validates input and reads or writes records, because there are few business rules to protect. Clean Architecture is worth its extra cost when substantial business rules must stay independent of the database and other infrastructure, and must be testable without them.
The decision is about responsibilities and dependency direction:
- Simple layering: the API calls business logic, and business logic calls data access directly. Business code refers to persistence code, so it is coupled to that code. When the logic is thin, this coupling costs little and the design stays direct.
- Clean Architecture: business rules and use cases declare the interfaces they need, such as an order repository. Infrastructure implements those interfaces, and dependency injection connects them at runtime. Business code never refers to database code, but each boundary adds abstractions and configuration.
A layered design can also invert selected dependencies where a rule needs isolation. Expected future growth does not justify either choice by itself. The question is whether real business rules make isolation and independent testing worth the extra abstraction.
Common web application architectures (Microsoft Learn) · The Clean Architecture (Robert C. Martin)
What is Clean Architecture?
Clean Architecture structures an application so its business rules sit at the center, independent of frameworks, UI and databases. Its purpose is core logic that can be understood, tested and changed independently of those details.
- Entities: general business rules, such as an
Orderthat rejects zero quantities.- Use cases: application-specific rules that coordinate entities, such as
PlaceOrder.- Adapters: translate between use cases and the outside world. Examples are controllers, presenters and repository implementations.
- Frameworks and drivers (infrastructure): outer details such as ASP.NET Core, EF Core and the database.
Dependency rule: inward means toward the business rules. Any source-code dependency that crosses one of these boundaries must point inward, so inner code never names an outer detail. Code at the same level may still refer to its peers.
Example:
PlaceOrderdeclares and callsIOrderRepository.SqlOrderRepository, a database adapter, implements that interface, and dependency injection supplies it at runtime. Control flows outward to the database, but the source dependency points inward. A fake repository therefore allowsPlaceOrderto be tested without a database.The circles describe dependency direction. They do not require a particular number of projects or folders.
Robert C. Martin: The Clean Architecture · Microsoft: Common web application architectures
What is the difference between traditional layered and Onion/Clean Architecture?
In traditional layering, source dependencies usually follow the call chain downward, so business logic depends on data access. In Onion and Clean Architecture, dependencies point toward the business rules. The business and application code defines the interfaces it needs, and infrastructure implements them.
In the table, A → B means that code in A refers to types in B.
Aspect Traditional layered Onion/Clean Typical source dependencies UI → business logic → data access. Business logic compiles against the data layer. UI → core, and infrastructure → core. The business and application core refers to nothing outside itself. How business rules reach storage Business logic calls classes or interfaces that the data-access layer owns. The core declares the interface it needs, such as IOrderRepository. Infrastructure implements it, and dependency injection supplies it at runtime. Testing business rules Isolating the rules is harder because they refer to data-access types. The rules can be tested with simple fakes of core-owned interfaces, without a database. Changing infrastructure Changes to data-access types can ripple into business code. Changes stay behind the core’s interfaces unless the core’s needs change. Layered designs can also use dependency inversion. For example, business logic can own a repository interface that the data layer implements. Onion and Clean make that inward direction a defining rule rather than an optional technique. Onion and Clean share this rule but name their layers differently, and neither requires a particular number of projects.
Common web application architectures (Microsoft Learn) · The Clean Architecture (Robert C. Martin)
What is plug-in architecture?
Plug-in architecture lets a host application gain behavior from separately built extensions through a shared contract.
It has three parts:
- Host: the main application. It decides where extensions can fit, then finds, loads, creates and calls them.
- Shared contract: an interface or abstract type that both sides reference, such as
ICommandwith aNameand anExecutemethod.- Extensions (plug-ins): separately built implementations of the contract.
For example, an image editor can expose a filter contract and load sharpening or blur filters as extensions. The host understands how to apply a filter; each extension supplies the actual transformation.
This makes supported behavior extensible, but introduces contract-compatibility and loading costs. Extensions remain limited by the host’s extension points, and loading them in-process does not provide a security sandbox.
How does a plug-in add behavior through an existing host extension point?
A plug-in adds behavior by implementing a contract that the host already defines. The host finds that implementation, creates it and calls it through the contract, without knowing the plug-in’s concrete type.
Example: the host’s shared contract is
IExporter { string Format { get; } void Export(Document doc, Stream output); }.
- Implement the contract:
PdfExporter : IExporterreports- Build and provide it: the extension’s author compiles it separately and places it where the host looks, such as a plugins folder.
- Discover: the host loads the plug-in and finds types that implement
IExporter.- Create: the host instantiates
PdfExporter, either directly or through DI.- Invoke: when the user chooses PDF, the host calls
Export(doc, stream)the same way it calls any other exporter.Limit: a plug-in can only do what its extension point expresses.
IExportercannot add collaborative editing. That works only if the host already offers a suitable extension point for it. Otherwise, a suitable extension point must be added or changed first.
When is plug-in architecture the wrong choice?
It is usually a poor fit when one team owns the features and ships them with the host: dynamic discovery, loading, and extension-contract compatibility add work without enabling independent delivery. Microsoft: common web application architectures
What is the key difference between MVC and MVVM?
In MVC, a controller receives each user action, works with the model, and then selects a view and supplies the data it renders. In MVVM, the view binds to properties and commands exposed by a view-model. The view-model changes its own state and raises change notifications, and the bindings refresh the view. The view-model never references the view.
Aspect MVC MVVM Handling input The controller receives the request, calls application behavior and chooses the response. The path from input to output is explicit in the action code. The view forwards user actions to view-model commands through bindings. The same view stays on screen and reflects the new state; navigation is handled separately. Updating the view The controller passes data to the selected view, which renders it for that response. Bound controls refresh when the view-model raises a change notification, such as PropertyChangedfromINotifyPropertyChanged. A missed notification leaves the screen showing stale data.Testing Controller logic can be unit-tested without rendering a view. View-model state and commands can be unit-tested without creating any UI controls. Typical fit Request/response web applications, such as ASP.NET Core MVC. Long-lived client screens on platforms with data binding, such as .NET MAUI. The cost of MVVM is its binding infrastructure: observable properties, commands and change notifications must be implemented and kept correct.
Microsoft: Overview of ASP.NET Core MVC · Microsoft: Model-View-ViewModel (.NET MAUI)
Why is two-phase commit (2PC) problematic in microservices?
Two-phase commit is a blocking protocol. Every participant that has prepared must wait for one coordinator’s decision while still holding its locks, so a slow or failed coordinator or a network delay can stall other services. It also works only when every participant supports the same transaction protocol, which couples independently failing services to one shared coordinator.
How the blocking arises:
- Prepare: the coordinator asks each participant to make its tentative work durable and vote. A participant that votes yes keeps the locks that work needs and promises to follow the final decision.
- Decide: the coordinator records commit only if every participant voted yes. Any no vote produces abort.
- Complete: each participant commits or rolls back when the decision reaches it, and only then releases its locks.
Consequences for microservices:
- Locks held across network delays: between steps 1 and 3, a coordinator crash or network delay leaves prepared data locked. Requests that need those rows wait or time out, so one component’s failure reduces the availability and throughput of others.
- Shared protocol requirement: each database, broker or service in the transaction must take part in the same prepare-and-commit protocol under one coordinator. A participant that cannot join that protocol cannot share the atomic commit.
This is why microservice workflows more often commit locally in each service and coordinate the remaining steps through messages, as in a saga.
How does the Outbox pattern support at-least-once event delivery?
The Outbox pattern makes an event as durable as the state change it describes, then publishes it with retries. After the change commits, the event can be delayed or published more than once, but it is not lost while the relay keeps retrying.
- Write atomically: the service saves the domain change and an outbox record for the event in one local database transaction. Either both commit or neither does.
- Relay: a separate process reads committed outbox records and publishes them to the broker.
- Retry: if publishing fails, the record stays pending and the relay tries again.
- Record progress: after the broker accepts the event, the relay marks the record as published.
Without an outbox, the service writes to the database and to the broker as two separate operations. A failure between them can leave one done and the other missing, for example a committed order whose event was never published.
Duplicates: if the relay crashes after the broker accepts the event but before step 4, it publishes the event again. Consumers therefore need idempotent handling or deduplication, for example by recording the IDs of events they have already processed.
The atomicity covers only the local store, whose transaction must include both records. No single transaction spans the database and the broker. In Azure Cosmos DB, for example, the two writes go into one transactional batch, which must target a single logical partition.
Microsoft: Transactional Outbox pattern with Azure Cosmos DB
What is the difference between at-most-once, at-least-once and exactly-once message delivery?
The three guarantees describe what a failure can do to a message within a stated boundary, such as a consumer reading from a broker. At-most-once never redelivers, so a message can be lost but never repeated. At-least-once redelivers until progress is recorded, so no message is lost but one can be processed more than once. Exactly-once means each message’s result is committed once, with neither loss nor duplication. It holds only inside a boundary that commits the result together with the consumer’s progress. Delivery attempts can still repeat after a failure, but only one durable outcome survives.
Guarantee Consumer sequence After a crash At-most-once The consumer records its progress, by acknowledging or saving its position, and then processes the message. A crash after recording progress loses the message, because it is not delivered again. At-least-once The consumer processes the message, then records its progress. A crash before progress is recorded causes redelivery, so the work can happen twice. Exactly-once The result and the consumer’s progress commit atomically. Examples are output topics and consumer offsets committed in one Kafka transaction and read with read_committedisolation, or a database write committed in the same transaction as the message’s offset or ID.Either both commit or neither does. An aborted attempt leaves no visible result and is retried. After a restart, the consumer resumes from the committed position or recognizes an already recorded ID, so the result is not applied twice. The guarantee covers only what that atomic commit includes. Kafka transactions make consuming and producing between Kafka topics exactly-once only when the output and the consumer offsets commit in one transaction and downstream consumers are configured with
read_committed. The defaultread_uncommittedsetting can expose records from aborted transactions. A database write, email or HTTP call outside the transaction is not covered. Such a side effect needs the external system’s cooperation, typically an idempotency key or the message position recorded atomically with the local state change. End-to-end exactly-once across arbitrary external systems is therefore generally not available. Retry limits that move a message to a dead-letter queue, and retention or TTL expiry, are separate limits outside these guarantees.
How do Kafka, RabbitMQ classic or quorum queues, and RabbitMQ Streams differ in workload fit, ordering and operational factors?
RabbitMQ classic or quorum queues fit work items that are processed and then removed. Kafka fits high-throughput event streams that are retained so many independent consumers can read and replay them, when its partitioned log and ecosystem are central to the design. RabbitMQ Streams provide similar log-style replay and fan-out for systems that already run RabbitMQ. None of the three orders messages globally: order holds only within one queue, partition or stream partition. Operating cost and the team’s experience with each platform also shape the choice.
Aspect RabbitMQ classic/quorum queues Kafka RabbitMQ Streams After consumption An acknowledged message leaves the queue, so the queue holds only pending work. Quorum queues replicate their contents across nodes for data safety. Events stay in a partitioned log for the retention period. Each consumer group tracks its own position in that log. Messages stay in a replicated append-only log. Consumers read them without removing them. Replay Not built in. Reprocessing needs republishing or a copy kept elsewhere. A consumer can move its offset back within retention and process events again. A consumer can start reading from an offset or a timestamp. Ordering boundary One queue. Redelivery and competing consumers can still change processing order. One partition, not the whole topic. One stream. A super stream preserves order only within each partition. Operations Health is watched through queue depth, unacknowledged messages, redelivery and node health. Partitions, replication, consumer rebalances, consumer lag and retention all need planning and monitoring. Streams are a RabbitMQ feature, so a team that already operates RabbitMQ gains log-style consumption without running a second broker platform. Typical workload Latency-sensitive commands and background jobs such as GenerateInvoice, with acknowledgements, retries and flexible routing through exchanges.High-throughput events such as OrderPlaced, consumed independently by billing, analytics and rebuildable projections.Large fan-out, replay or large backlogs in a system that already operates RabbitMQ. Apache Kafka: Introduction · RabbitMQ: Quorum Queues · RabbitMQ: Streams · RabbitMQ: Super Streams
How can a message-based system keep one entity's events in order?
Most brokers order messages only within one partition or queue, not across a whole topic. One entity’s events therefore stay in order only under three conditions: they all travel through the same partition, they are processed one at a time for that entity, and the consumer applies them by a per-entity sequence rather than by arrival.
- Route by a stable key: the producer sends every event for the entity with the same key, such as
OrderId. In Kafka, events with the same key go to the same partition, and consumers read each partition in the order it was written.- Process each key sequentially: concurrent handlers for one partition, or competing consumers sharing one RabbitMQ queue, can finish an entity’s events in a different order than they arrived.
- Apply by version or sequence number: retries, redelivery and prefetch can still deliver an event twice or late. The consumer stores the last version it applied for the entity and compares each arriving event with it:
- At or below the applied version: a duplicate whose change is already in place, so it is acknowledged and skipped.
- The next version: applied, and the stored version advances.
- Further ahead: an earlier change has not arrived yet. Applying this event now would build on missing state, and discarding the earlier one when it arrives would lose its change. The consumer holds or retries the later event, or recovers the gap from the source, until the missing event is applied.
The rule differs when each event carries the entity’s complete state rather than a change. A newer snapshot then supersedes every older one, so it can be applied across a gap, and a late older snapshot can be discarded without losing anything.
Adding partitions increases parallelism but creates no order between partitions. RabbitMQ super streams follow the same rule and preserve order only within each partition.
Apache Kafka: Introduction · RabbitMQ: Super Streams · Microsoft: Event Sourcing pattern
What is the difference between horizontal and vertical scaling?
Vertical scaling (scaling up) gives one instance more resources, such as a larger VM. Horizontal scaling (scaling out) adds instances and divides the work among them.
The direction changes several practical consequences:
Aspect Vertical scaling Horizontal scaling How capacity grows A larger machine runs the same instance, so work does not have to be divided. More instances share the load, which works only when work can be divided and routed among them. Upper limit The largest available instance size caps growth. Shared dependencies, such as the database, become the limit rather than one machine. Disruption while scaling Resizing often makes the instance temporarily unavailable, so it is automated less often. Instances can be added or removed while the application keeps running, which suits autoscaling. Effect of an instance failure Adds no redundancy. A failed instance takes all of its capacity with it. Remaining instances can keep serving while the failed one is replaced. State Scaling one instance up does not by itself require moving in-process state. Shared state must move outside interchangeable instances, or stateful work needs explicit partition ownership. Microsoft: Autoscaling guidance · The Twelve-Factor App: Concurrency · The Twelve-Factor App: Processes
What architectural prerequisites must be met before horizontal scaling works?
Work must be divisible across instances, and routing must reach an instance able to own or process it. Shared durable state must live outside interchangeable workers, or stateful work needs explicit partition ownership and routing. Stateless processes are a simpler case because any replica can take the next unit; they do not mean the application has no persistent state. The Twelve-Factor App: Processes · The Twelve-Factor App: Concurrency
Why can horizontal scaling fail even with many instances?
Adding instances helps only when the instances themselves are the bottleneck and work actually spreads across them. Otherwise extra replicas add load or cost without adding useful capacity.
- Downstream saturation: each replica adds its own database connections and outbound calls. Twenty replicas with a pool of 100 connections each can open up to 2,000 connections, so the database, a connection limit or an external API quota saturates instead.
- Instance-local state: sessions, files or locks held in one process are missing when the next request reaches another replica, so requests fail or behave inconsistently.
- Uneven distribution: a hot partition key or sticky routing can overload one instance while fleet averages look healthy. More replicas do not relieve the instance that owns the hot work.
- Scaling lag: an autoscaler acts on observed metrics, and new instances need startup and readiness time before they serve traffic. Capacity therefore arrives after demand has risen unless spare capacity or an earlier signal covers the delay.
When throughput barely improves after the replica count doubles, the bottleneck is elsewhere.
The Twelve-Factor App: Processes · Microsoft: Autoscaling guidance · Kubernetes: Horizontal Pod Autoscaling
Why does Command-Query Separation (CQS) make code easier to reason about?
CQS makes each operation’s contract show whether it can change state. A query returns information without observable side effects; a command changes state and traditionally returns no result. Finding where the code changes state therefore only requires following command calls.
- Queries: calling one causes no state change, so a query call can be added, repeated or moved, for example while debugging, without introducing a mutation.
- Commands: every intended mutation goes through a command, so the paths that change state are easy to find, and retry and transaction decisions attach to those calls.
CQS rules out state changes caused by queries, not changes in what queries observe. A side-effect-free query can return a different result when it reads the current time or data that a command or another process has changed. Its position relative to commands therefore still matters, and caching its result still needs a freshness decision.
Repeating or caching a query avoids unintended state changes only when the query actually honors the contract. A query that secretly writes, such as one that updates a last-accessed timestamp, changes state on every call, and a method name alone cannot guarantee the contract.
When is it pragmatic to violate Command-Query Separation (CQS)?
Violating CQS is pragmatic when reading and changing state form one coherent or atomic operation, or when a separate query would only add a round trip without adding clarity.
- Atomic read-and-remove:
Stack<T>.Pop()removes the top item and returns it in one call, a long-accepted exception to strict CQS. In concurrent code, separate check and remove calls would let another thread take the item in between, soConcurrentStack<T>.TryPop(out item)does both as one atomic operation.- Create that returns its identifier: an insert can return the database-generated ID, because the caller needs it and a follow-up query would only cost another round trip.
The combined operation should still make its state change obvious and return only what the caller needs, such as an identifier or version rather than the full entity. A returned result does not show whether retrying the command is safe, so retry behavior needs its own explicit contract.
.NET: ConcurrentStack
.TryPop · Martin Fowler: Command Query Separation
What is the difference between Command-Query Separation (CQS) and CQRS?
CQS is a rule for individual operations: each method either changes state (a command) or returns data without side effects (a query). CQRS, Command Query Responsibility Segregation, applies the same distinction at the architecture level by giving the write path and the read path separate models and handlers.
Aspect CQS CQRS Scope One method or operation at a time, usually within a single class. A subsystem’s whole write path and read path. What is separated Command and query signatures; both kinds of method can work on one shared model. The models themselves: a write model for changes and a read model shaped for queries. Data storage Not addressed; a single shared store is normal. One store can serve both sides; separate read and write stores are common but optional. Cost Low; mainly a discipline for method contracts. High; two models and their handlers must be built and kept consistent. The two are independent: a CQRS system does not guarantee that every method follows strict CQS, and following CQS does not require CQRS.
What is dependency injection, and what problem does it solve?
Dependency injection (DI) means a class receives the objects it depends on from outside, usually through its constructor, instead of creating them itself. It is a form of Inversion of Control: the class declares what it needs, and the application decides which implementations to supply.
Without DI, a class that calls
new SmtpEmailSender()is tied to that implementation. Replacing it requires editing the class, the class must also know how to build the sender’s own dependencies, and a unit test cannot substitute a fake.With DI:
- The class depends on an abstraction, such as
IEmailSender, and lists it as a constructor parameter.- At the composition root, typically
Program.cs, the application chooses the implementation, either by constructing the object graph directly or by registering it in a container.- When a container is used, it builds the class’s dependencies recursively and manages their lifetime and disposal.
The result is explicit dependencies, implementations that can change without editing their consumers, and tests that pass collaborators directly. DI does not require a container; a container automates wiring and lifetime management.
What do the
Transient,Scoped, andSingletonservice lifetimes mean in .NET dependency injection, and what is a typical use for each?The lifetime controls how often the container creates a service and which consumers share an instance:
Transientcreates one per resolution,Scopedshares one within a scope, andSingletonshares one across the whole application.
Lifetime Instances and sharing Disposal of container-created instances Typical use TransientA new instance every time the service is resolved, even twice within the same scope. When the scope that resolved it ends. Lightweight stateless services, such as a mapper or formatter. ScopedOne instance per scope, shared by everything resolved in that scope. ASP.NET Core creates a scope for each HTTP request. When the scope ends, such as at the end of the request. Per-request or unit-of-work state, such as an EF Core DbContext, whichAddDbContextregisters as scoped by default.SingletonOne instance shared by every scope and every concurrent caller until the root provider is disposed. When the root provider is disposed at application shutdown. Services that are safe for concurrent use, such as a clock abstraction or a thread-safe, size-bounded cache. The uses are typical fits, not guarantees from the container: a singleton is safe only if its implementation is thread-safe.
Lifetimes also interact when one service holds another:
- Singleton holding a scoped service: a lifetime violation, because the scoped instance would outlive its scope.
- Longer-lived service holding a transient: valid, but the captured instance lives as long as the service that holds it. It must suit that effective lifetime: thread-safe if a singleton’s concurrent callers use it, and acceptable to dispose only when that longer lifetime ends.
What is a captive dependency, and how can it be fixed?
A captive dependency occurs when a longer-lived service holds a shorter-lived one, so the shorter-lived instance stays in use beyond its intended lifetime. The typical case is a singleton, such as a hosted service, that receives a scoped
DbContextthrough its constructor.Consequences:
- Request- or operation-specific state carries over into later, unrelated work.
- A scoped service that is not designed for concurrent use, such as
DbContext, can be shared by concurrent operations and expose stale tracked state.- Cleanup moves from the end of the scope to the end of the longer lifetime.
Fix: the long-lived service depends on
IServiceScopeFactoryinstead of the scoped service. For each unit of work, it creates a scope, resolves the scoped service from that scope, uses it and disposes the scope. A worker that processes independent items creates a new scope per item rather than one scope for its whole lifetime.await using var scope = scopeFactory.CreateAsyncScope(); var db = scope.ServiceProvider.GetRequiredService<AppDbContext>(); await ProcessItemAsync(db, item, cancellationToken);Detection: when the host is built in the Development environment, scope validation rejects a scoped service injected into a singleton or resolved from the root provider. It does not catch every lifetime mistake. A transient captured by a singleton, for example, also lives as long as the singleton, and its transient registration does not make it thread-safe when concurrent callers share it.
.NET: use scoped services within a BackgroundService · .NET DI guidelines · .NET service lifetimes · .NET dependency injection overview
Why is Service Locator considered an anti-pattern in business code?
Service Locator hides a class’s dependencies. Instead of declaring them in its constructor, the class asks a container such as
IServiceProviderfor them at runtime withGetService<T>()orGetRequiredService<T>(). In business code, this causes three problems:
- Hidden requirements: the constructor no longer shows what the class needs, so its dependencies are discovered only by reading the implementation.
- Late failures: a missing registration fails when that code path runs, not when the object graph is built or validated.
- Harder tests: a unit test must configure a container or fake provider instead of passing collaborators to the constructor.
public sealed class CheckoutService(IServiceProvider provider) { public Task ProcessAsync() => provider.GetRequiredService<IOrderRepository>().SaveChangesAsync(); }Runtime lookup still has a narrow place in infrastructure: factories that choose an implementation at runtime, middleware or framework activation code, and background services that create explicit scopes. Application and domain services declare their collaborators in their constructors instead.
What evidence justifies introducing a pattern?
A pattern is justified when the code already shows a design pressure that the pattern handles more cheaply than the direct design. That pressure takes one of three forms:
- Recurring variation: the same point in the code repeatedly needs different behavior. A known structure for substituting or composing that behavior then pays off.
- A responsibility boundary: one concern needs to be separated from another, and the pattern makes that boundary explicit.
- A failure mode: the direct design keeps producing the same defect or cost, and the pattern makes it cheaper to handle.
The decision is a cost comparison. Every pattern adds indirection, such as extra types and interfaces, and its control flow is less direct. When the pattern answers a problem the code already has, it makes the variation or boundary easier to see. When the problem is only anticipated, the indirection is speculative. It hides simple control flow without solving anything yet.
Example: dependency injection behind an interface earns its place when implementations are actually substituted or composed. One concrete dependency with no variation is usually clearer as a direct call.
How do the Repository and Unit of Work patterns differ?
Repository shapes how domain code reaches persisted aggregates, and Unit of Work controls when the changes from one business operation are committed. They solve different problems and usually work together.
Aspect Repository Unit of Work Responsibility Presents stored aggregates as a domain-oriented, collection-like interface, such as IOrderRepositorywithFindAsync,AddandRemove.Tracks the changes made during one business operation and commits them through one transaction boundary. What it keeps out of domain code Query details such as SQL, EF includes and open-ended IQueryable<T>stay in infrastructure.Individual inserts, updates and deletes are gathered and written together instead of one by one. Effect of a call AddorRemoveonly changes the set of pending objects; nothing is written yet.One save writes the pending changes from every repository that shares the unit of work. EF Core counterpart DbSet<T>already provides collection-like access.DbContextalready tracks changes untilSaveChangesAsynccommits them.Flow: a handler creates an
Order, adds it throughIOrderRepository, and then callsSaveChangesAsyncon the unit of work. Several repository operations in the same handler therefore commit through one save, inside one transaction when the provider supports it.EF Core: DbContext lifetime, configuration, and initialization · EF Core: Using transactions
Why does EF Core's
DbContextalready implement the Unit of Work pattern?
DbContextalready performs the two jobs that define a Unit of Work. It records the changes made during one business operation, and it commits them together.
- Tracking: the context’s change tracker records entities returned by tracking queries and entities added, modified or removed through the context. Results of no-tracking queries do not enter the tracker. Tracked changes stay pending in memory until a save.
- Commit:
SaveChangesorSaveChangesAsyncwrites all pending changes. When the database provider supports transactions, EF Core applies them in one transaction by default. If any change fails, the whole save is rolled back.Repositories that receive the same context instance therefore share one unit of work. Each repository can call
AddorRemove, and one save commits all of their changes. AnIUnitOfWorkinterface implemented by the context only exposes that save; the mechanism is still EF Core’s.Boundaries:
- Each context instance has its own change tracker. Separate contexts are separate units of work by default, and each one’s save commits on its own.
- With relational providers, separate contexts can join one explicit transaction by sharing the same
DbConnectionandDbTransaction. Their tracking stays separate, but their saves commit or roll back together with that transaction.- A
DbContextis designed for one short-lived unit of work and is not thread-safe. A singleton registration would share one long-lived tracker across concurrent requests, which is unsafe.AddDbContextregisters the context as scoped by default, which gives one instance per request in ASP.NET Core.EF Core: Tracking vs. No-Tracking Queries · EF Core: DbContext lifetime, configuration, and initialization · EF Core: Using transactions
When is a generic
IRepository<T>an anti-pattern?A generic
IRepository<T>is an anti-pattern when it serves as the domain-facing contract and its operation surface does either of two things: it lets callers bypass aggregate invariants, or it exposes operations that do not fit the aggregate. Generic typing alone is not the defect. The defect is what the contract lets callers do.The harmful case shows up in two ways:
- Bypassed invariants: the contract lets callers add, update or delete objects directly instead of going through the aggregate root, whose methods enforce the rules. For example, suppose
Orderforbids changing line items after confirmation. AnIRepository<OrderLine>lets code delete a line from a confirmed order without that check.- Unsuitable operations: the contract gives an aggregate methods such as unrestricted update or delete even when the domain has no such operation.
When every exposed operation fits and changes still go through the aggregate root, an interface is not harmful merely for being generic. A generic interface constrained to aggregate roots, for example, can be acceptable if each operation it offers is valid for those roots. An aggregate-specific interface such as
IOrderRepositoryexposes only operations that make sense forOrder. Its infrastructure implementation can still reuse a generic base class internally, as long as the generic methods do not widen the domain-facing contract.Microsoft: Design a DDD-oriented microservice · Microsoft: Design the infrastructure persistence layer
When is a repository layer over EF Core worth adding instead of injecting
DbContextdirectly?A repository and unit-of-work layer over EF Core is worth adding when it protects a real domain boundary. Otherwise injecting
DbContextdirectly is usually clearer. EF Core already provides collection-like access throughDbSet<T>, andDbContextchange tracking withSaveChangesAsyncalready acts as a unit of work.Reasons that justify the extra interfaces:
- Domain isolation: application and domain code express aggregate operations, such as loading an order with the state its rules need, instead of EF queries.
- Controlled access: aggregates are loaded only through known shapes that preserve their invariants and ownership rules.
- Multiple implementations: the same domain contract genuinely has more than one persistence implementation.
Reasons that do not justify it on their own:
- Mirroring EF Core: interfaces that wrap every
DbSet<T>with matching CRUD methods only rename EF Core and add indirection.- Testability alone: a fake repository cannot reproduce database behavior such as query translation and constraints. That behavior is better checked against the real provider.
Injecting
DbContextkeeps code simple and keeps all EF Core features available, but couples application code to EF Core. Repository interfaces give domain-oriented access and an explicit commit boundary at the cost of extra indirection. For a small CRUD service with no domain rules to isolate, directDbContextis the clearer design.
What is CQRS?
CQRS (Command Query Responsibility Segregation) handles state changes and reads through separate models. Commands go through a write model that enforces business rules and changes state. Queries go through a read model that returns data shaped for a screen or API and changes nothing.
- Commands express a business intent, such as
PlaceOrder. The write model validates each command against its invariants before changing state.- Queries, such as
GetOrderSummary, only return data. Their model can be shaped for the request, for example one flat row per order summary, so the query avoids joins.The separation lies in the models and code paths, not necessarily in storage:
- Shared data: both models can work against the same database through separate logic. No copy has to be kept in sync.
- Separate read store: a denormalized copy must be updated from write-side changes. Updating it within the same request reduces staleness, but it ties the read side’s latency and failures to the write. A failure between the two updates can still leave the copy out of date unless both commit atomically. Updating it asynchronously from events removes that coupling, but reads can be stale until the projection catches up. Events then need reliable publication, such as an outbox, and an idempotent projector.
Message brokers, separate databases and Event Sourcing are optional choices, not part of the definition.
When is CQRS useful, and when does it add unnecessary complexity?
CQRS separates command-side changes from query-side reads so each model can match different rules, shapes, or scaling needs. It is useful when a shared model has become a poor fit for both sides. If simple CRUD already handles both, separate handlers or projections add synchronization and operational cost without enough benefit. Separate databases, messaging, and event sourcing are optional; asynchronous projections can be stale. Microsoft: CQRS pattern
What is a bounded context in Domain-Driven Design?
A bounded context is the explicit boundary within which one domain model and its language apply. Inside it, each term has one consistent meaning. Another context can use the same word for a different concept with its own model.
For example, “Customer” in Sales means a prospect with contact details and deal history, while “Customer” in Billing means an account with payment terms and a credit limit. A single shared
Customertype would collect optional fields and rules that belong to neither context.
- Ownership: each context owns its model. Team organization, code bases and database schemas often mark the boundary. Contexts can still share a deployment or a physical database if table ownership keeps one model from leaking into the other.
- Integration: contexts communicate through an explicit contract or translation, such as Sales publishing
OrderPlacedand Billing creating an invoice from it, instead of using each other’s internal types.A bounded context is a model boundary, not necessarily a separate deployment.
Eric Evans: Domain-Driven Design Reference (Bounded Context)
What is an aggregate in Domain-Driven Design?
An aggregate is a cluster of related domain objects treated as one consistency boundary. Outside code holds references only to its root entity and changes the cluster only through that root. The root enforces the invariants that must hold across the whole cluster.
For example, an
Orderroot owns its order lines. Adding a line goes throughOrder.AddLine, which rejects a non-positive quantity before any state changes, and other code cannot edit the line collection directly.
- Transaction scope: the aggregate boundary also scopes transactions. A transaction usually changes one aggregate, so consistency and locking stay local. Changing several in one transaction can still be correct when an immediate invariant spans them and the storage commits them atomically. If that happens often, the boundary is probably wrong.
- Sizing: the boundary belongs around facts that must agree at commit time. An oversized aggregate loads and locks unrelated state. An undersized one turns an immediate invariant into coordination between transactions.
- Later consequences: effects that can follow after the commit, such as updating another aggregate, are handled asynchronously, often through domain events.
- Persistence: repositories load and save whole aggregates through their roots.
Eric Evans: Domain-Driven Design Reference (Aggregates) · Microsoft: Domain events: design and implementation
When is full DDD the wrong choice?
Full DDD is the wrong choice when a domain has few business rules, such as a CRUD service that mainly validates and stores records. Its modeling and coordination cost then protects little, and a simple transaction script or data-centric model is easier to build and maintain.
- Tactical patterns such as aggregates, value objects and repositories earn their cost where invariants must hold inside a clear consistency boundary. Without such rules, they add types and indirection without protecting anything.
- Strategic patterns such as a ubiquitous language and bounded contexts can still help on their own when the same term means different things in different parts of the business, even if the code inside each context stays simple.
- Effort can differ by subdomain: the core domain that differentiates the business may justify rich modeling, while supporting or generic subdomains stay simple or are bought or adopted.
The deciding factor is the amount of real domain behavior, not project size.
What is a design pattern?
A design pattern is a named, reusable solution to a recurring design problem. It describes the problem, how a set of objects collaborates to address it, and the consequences of using it. It is not ready-made code or a fixed class diagram.
- Name: a shared word for the design. Saying Strategy communicates that an algorithm varies behind a stable contract.
- Problem: the pressure the pattern responds to, such as an unknown set of dependents that must be notified of state changes (Observer).
- Solution: the roles and their collaboration, adapted to each codebase rather than copied literally.
- Consequences: what the pattern buys, such as replaceable behavior, and what it costs, such as extra types and indirection.
The best-known catalog is the Gang of Four book, which describes 23 object-oriented patterns grouped as creational, structural and behavioral. A pattern is justified only when its problem actually exists in the code; otherwise its indirection is pure cost.
GoF: Design Patterns: Elements of Reusable Object-Oriented Software · Design Patterns overview · Choose a pattern by intent and cost
How do the GoF categories (creational, structural, behavioral) relate to choosing a design pattern?
The three categories group the GoF patterns by the kind of problem they address. They narrow where to look, but the concrete design pressure and each pattern’s intent decide which pattern fits.
Category Problem it addresses Examples Creational Where construction decisions live: which concrete class is created, how a complex object is assembled, or how instances are copied or limited. Factory Method, Abstract Factory, Builder, Prototype, Singleton Structural How objects and types are composed into larger structures, and how interfaces are adapted or wrapped. Adapter, Composite, Decorator, Facade, Proxy Behavioral How responsibilities and communication are distributed among objects. Strategy, Observer, Command, State, Template Method A category alone cannot decide, because patterns with a similar structure serve different intents. State and Strategy both delegate behavior to another object, but they differ by intent rather than by who selects the delegate: State lets behavior follow the context’s current state, with transitions defined by the context or by the state objects, while Strategy makes an interchangeable algorithm a policy that a client usually supplies. The useful starting point is the pressure in the code, such as an algorithm that must vary or an interface that does not match. The pattern whose intent addresses that pressure is the candidate.
GoF: Design Patterns: Elements of Reusable Object-Oriented Software · Creational patterns at a glance · Choose a pattern by intent and cost
When does using a design pattern become an anti-pattern?
A pattern becomes an anti-pattern when the indirection it adds costs more than the variation or responsibility boundary it protects. The pattern name alone never pays for the extra complexity.
Every pattern adds cost:
- more types or delegates to read and maintain
- an extra hop between a caller and the behavior
- execution order, lifetime or error handling that becomes harder to follow
Typical signs that the cost is not repaid:
- No real variation: an Abstract Factory with only one product family, or a Strategy replacing a small conditional that is stable and clearer.
- Trivial construction: a Builder around values that a readable constructor or options record already handles.
- No boundary to protect: an Adapter between two contracts that the same team controls and could simply align.
- Flexibility nobody uses: a Command wrapper where nothing is delayed, logged, retried or undone, or an Observer with one required dependent that could be called directly.
When the variation that justified a pattern disappears, collapsing the abstraction back into direct code removes its cost.
GoF: Design Patterns: Elements of Reusable Object-Oriented Software · Choose a pattern by intent and cost
How do Adapter, Decorator, and Proxy differ despite their similar wrapper shape?
An object Adapter, a Decorator and a Proxy all wrap another object, but for different reasons. An Adapter translates an incompatible contract into the one a client expects. A Decorator keeps the same interface and adds behavior. A Proxy keeps the same interface and controls access to the real object. A class Adapter adapts through inheritance instead of wrapping an object, so the wrapper comparison describes the object form.
Aspect Object Adapter Decorator Proxy Interface toward the client A different interface from the wrapped object’s: the one the client expects. The same interface as the wrapped object. The same interface as the real object. Responsibility Translating calls and data between two incompatible contracts at one boundary. Adding a responsibility around the call. Several decorators can be stacked at runtime, and their order affects behavior. Deciding whether, when or how a call reaches the real object. Effect on a call It is converted into the wrapped object’s operations, and results are converted back. It normally reaches the wrapped object, with extra behavior before or after. It may be forwarded, deferred or refused according to the access rule. Because their class shapes look alike, the wrapper’s responsibility, not its structure, identifies the pattern.
GoF: Design Patterns: Elements of Reusable Object-Oriented Software · GoF: Adapter (InformIT excerpt) · Choose a pattern by intent and cost
How does Information Expert reduce coupling, and when should another principle override it?
Information Expert assigns a responsibility to the object that already holds the information needed to perform it. This reduces coupling because callers no longer reach into that object’s data to apply its rule themselves. They depend only on the result, and the rule lives in one place instead of being duplicated by each caller.
Example:
Orderowns its line items, soOrder.Totalcalculates the total. AnOrderServicethat summedorder.LineItemsitself would depend on the internal structure ofOrder, and other callers could repeat the same calculation.Information Expert identifies a strong candidate, not an automatic winner. Another principle should prevail when keeping the behavior on the data holder would:
- Pull infrastructure into a domain object: behavior that needs I/O, such as persistence or sending email, belongs in a separate collaborator. Pure Fabrication justifies such a non-domain object, for example a mapper or a gateway.
- Coordinate several aggregates: a policy that needs information beyond one object’s boundary fits a separate collaborator that coordinates them.
- Add a dependency across a module or layer boundary: Low Coupling favors keeping that dependency out of the object, even though the object has the data.
- Give the object unrelated responsibilities: High Cohesion favors keeping its responsibilities closely related.
Behavior that depends only on the object’s own state and invariants stays with that object.
Craig Larman: Applying UML and Patterns, Information Expert (sample chapter) · GRASP principles · Information Expert limits · GRASP tradeoffs
How does GRASP differ from SOLID?
GRASP and SOLID start from different questions. GRASP, described by Craig Larman, asks which object should own a responsibility: knowing, creating, coordinating or varying a behavior. SOLID asks whether classes, interfaces and their dependencies are shaped so that change stays contained.
Aspect GRASP SOLID Starting question Which object should be responsible for this behavior? Are this class’s responsibilities, interfaces, inheritance and dependency direction designed so that change stays contained? Focus Assigning responsibilities among collaborating objects while designing how they work together. Properties of individual classes and interfaces and the direction of dependencies between them. Content Nine heuristics: Information Expert, Creator, Controller, Low Coupling, High Cohesion, Polymorphism, Pure Fabrication, Indirection, Protected Variations. Five principles: Single Responsibility, Open/Closed, Liskov Substitution, Interface Segregation, Dependency Inversion. Typical decision Ordercreates its line items and calculates its own total, because it holds that data.A class that changes for several unrelated reasons is split, or high-level code depends on an abstraction rather than a concrete detail. The two overlap in their goals: low coupling, high cohesion and protection from expected variation. For example, Polymorphism and Protected Variations in GRASP pursue the same goal as Open/Closed in SOLID. They complement each other: GRASP helps decide where behavior goes, and SOLID helps check the shape of the resulting classes and dependencies.
Craig Larman: Applying UML and Patterns (sample chapter) · Robert C. Martin: SOLID course notes · GRASP principles · GRASP tradeoffs · SOLID principles
How does ASP.NET Core middleware differ from a classical Chain of Responsibility?
Both send a request through ordered handlers that can stop it early. In the classical pattern, linked handler objects pass the request along until one of them takes responsibility for it. ASP.NET Core middleware is a pipeline of delegates composed once at startup, in which each component normally does its part and calls the next, wrapping everything after it.
Aspect Classical Chain of Responsibility ASP.NET Core middleware Handlers and links Handler objects that each keep a reference to their successor. Request delegates; each middleware receives the rest of the pipeline as a nextdelegate.Building the chain Links are set between objects, and an implementation may allow them to change later. Use,MapandRunregistrations are composed into one request delegate when the app starts. The order is then fixed, although branches can still choose a path for each request.Role of each handler Usually either handles the request or forwards it, so typically one handler does the work. Usually every component does some work and calls next. Not callingnextshort-circuits the rest, and aRuncomponent always ends the pipeline.Work after the next step Not a focus of the pattern’s intent. Code after await next(context)runs as the response returns, so components see the response in reverse registration order.
What is the Command pattern?
The Command pattern turns a request into an object that holds the action to perform and the input it needs. An invoker triggers the command, typically through an
Executemethod, without knowing which object does the work or how.
- Command: an object such as
PlaceOrderCommandthat carries the request data and knows which receiver to call.- Receiver: the object that actually performs the work, such as an
OrderService.- Invoker: the code that calls
Execute, such as a button, a queue or a scheduler.- Client: creates the command and passes it to the invoker.
A restaurant ticket works the same way: it records the order, waits in the kitchen queue and tells the chef what to cook, so the waiter never cooks and the chef never takes orders.
Because the request becomes an object, it can be queued, logged, retried or delayed, and a command that records enough state can also undo or compensate for its effect. Wrapping a request in an object does not make its side effects atomic or durable. An in-memory undo history disappears with the process, and steps with external effects still need durable state, idempotency keys and compensation. A plain delegate such as
Actionis enough when naming, serialization and undo are not needed.GoF: Design Patterns · Command: solution and limits · Command: delegates and .NET examples
How does MediatR implement the Command pattern, and what does it add?
MediatR represents a command as a request object that carries the operation’s data, and
Senddispatches it to the one handler registered for that request type. It splits classic Command into parts: the request holds the data, the handler holds the execution logic that a classic command would put inExecute, and the mediator acts as the invoker that finds the right handler.What it adds:
- Decoupled callers: a controller or other sender depends only on the mediator and the request type, not on the handler or the services the handler uses.
- Pipeline behaviors: shared policies such as validation or logging wrap every handler call without being repeated in each handler.
The cost is indirection: following a call from sender to handler requires knowing the registrations. A direct service call stays clearer for one small, fixed interaction. MediatR is one way to realize a command-style design, not a requirement of the Command pattern.
How does EF Core use LINQ expression trees as an Interpreter?
EF Core treats a query’s lambda as a sentence in a small language. The lambda arrives as an expression tree, and EF Core walks its nodes and translates them into a query that its database provider can run, instead of running the lambda as compiled code. With a relational provider, that query is SQL.
Queryable.Whereaccepts anExpression<Func<T, bool>>, so the compiler builds a tree that describes the predicate.Enumerable.Whereaccepts aFunc<T, bool>, which can only be executed.- When the query runs, EF Core visits the supported nodes, such as member access, comparisons and
&&, and the provider produces an equivalent query in its own language, which is SQL for a relational database.- Captured variables, such as
minTotalino => o.Total > minTotal, can become query parameters instead of literals, which are SQL parameters with a relational provider.- A node with no translation, such as a call to an ordinary C# method that the provider does not map, makes EF Core 3.0 and later throw when it appears in a filter or another server-evaluated part of the query. Only the top-level projection can be partly evaluated on the client.
The tree plays the role of the Interpreter pattern’s abstract syntax tree. In the classic pattern each node evaluates itself; EF Core instead translates the tree into the provider’s query language, such as SQL, which the database then executes.
C# expression trees · EF Core: client vs. server evaluation · EF Core: how queries work
How does the Interpreter pattern differ from the Strategy pattern when rules are chosen at runtime?
Both let behavior vary at runtime, but they vary different things. Strategy chooses which already-written implementation runs. Interpreter represents the rule itself as data, such as an expression tree, and an evaluator walks that data to produce the result. Choosing an implementation at runtime is therefore not enough to make a design an Interpreter.
Aspect Strategy Interpreter What varies The implementation selected, such as a flat-rate or a weight-based shipping calculator. The rule’s structure, built from expression nodes such as And(TierEquals(Silver), TotalGreaterThan(200)).New behavior Selecting or configuring an existing implementation needs no new code; behavior that none of them provides needs a new implementation. Can often be composed from existing expression types without new classes, within what the grammar can express. Where the logic lives In each strategy’s code, which the caller simply invokes. Split across node types that an evaluator applies to a context, such as an order and its customer. Main costs Variation is limited to the implementations that exist, so uncovered behavior means new code to write, test and ship. Running a strategy is ordinary compiled code. A grammar to design and maintain, slower tree evaluation and harder debugging. Rules from untrusted users also need a restricted grammar with allowlisted fields and operations. Composing rules from data avoids new code only when the rule representation can be stored and loaded safely. Rule objects built in startup code still change only with a deployment.
Interpreter: solution · Interpreter: pitfalls · Interpreter: tradeoffs
What should determine whether an API returns
IEnumerable<T>,IReadOnlyList<T>, orIAsyncEnumerable<T>?The return type should state what consumers can do with the result: enumerate it synchronously, count and index it, or await while items arrive. None of the three, by itself, promises that the data is buffered, repeatable or a snapshot.
Return type Promises to consumers Does not promise Fits when IEnumerable<T>Synchronous, forward-only enumeration. A count, indexing or materialized data. It may be lazy, so enumerating it again can repeat the work, such as a query, and see different results. Consumers only iterate, and lazy or streamed production is acceptable. IReadOnlyList<T>Countand access by index, with no mutation members on the interface.A snapshot or immutability; a producer that keeps the underlying list can still change it. Consumers need the size or random access, usually for results already in memory. IAsyncEnumerable<T>Asynchronous streaming: await foreachcan wait between items, for example while the next page is fetched.Buffering or repeatability. Producing items requires I/O, and consumers should process items as they arrive rather than wait for all of them. Buffering, repeatability and snapshot semantics form a separate contract; when they matter, the API documentation has to state them.
What does the compiler generate for a
yield returnmethod?The compiler rewrites an iterator method into a generated state-machine class that implements the returned iterator interfaces, such as
IEnumerable<T>andIEnumerator<T>. The body is split at eachyield return, and the object records where execution stopped and keeps the values that must survive between steps.
- Calling the method only creates the state-machine object. None of the body runs yet.
- The first
MoveNext()runs the body from the start to the firstyield return, stores the yielded value asCurrent, saves its position and returnstrue.- Each later
MoveNext()resumes just after the saved position and runs to the nextyield return.- Reaching the end of the body or a
yield breakmakesMoveNext()returnfalse.Locals and parameters used across a
yield returnare kept in the generated object, so their values survive betweenMoveNext()calls. Because the body runs only during enumeration, its work and exceptions, including argument checks, happen when the sequence is consumed rather than when the method is called. The generated type and field names are compiler implementation details.
What is the Mediator pattern, and how does MediatR's request dispatch differ from it?
The Mediator pattern moves the rules for how a group of objects interact into one mediator object. The participants, called colleagues, notify the mediator instead of calling each other, and the mediator decides which other colleagues react. A many-to-many web of dependencies becomes hub-and-spoke coordination, so a colleague can be changed or added without the others knowing about it. Air traffic control is the usual analogy: pilots coordinate through the tower rather than with every nearby aircraft.
MediatR follows the same decoupling direction but solves a narrower problem:
Aspect Classic Mediator MediatR SendWhat it connects A set of peer colleagues that react to one another. One sender and the single handler registered for a request type. Where interaction rules live In the mediator, which knows the colleagues and coordinates them. In the handler. MediatR only locates and invokes it, with pipeline behaviors around the call. What stops depending on what Colleagues stop referring to each other. The sender stops referring to the handler and the services it uses. A MediatR handler that calls several services coordinates them itself; MediatR does not coordinate peers on its behalf.
What are the signs that a mediator is adding indirection without reducing coupling?
A mediator adds only indirection when it routes calls without providing a useful shared boundary: no common dispatch for several callers, no cross-cutting policy and no real coordination between participants. Typical signs:
- Callers still know the participants: senders still reference concrete handlers or the services behind them, so the mediator removed no dependency.
- Coordination is still duplicated: several callers repeat the same sequence around their mediator calls instead of one place owning it.
- Fixed two-party calls go through it: a request has one caller, one handler and no shared behavior, so the route only makes the execution path harder to find.
- The mediator becomes a god object: one central mediator or handler accumulates unrelated rules instead of dispatching to focused handlers.
Moving a workflow into a handler can decouple the caller from the services it uses, but the workflow itself is no simpler. Direct calls fit simple, stable interactions. The indirection pays off when several transports share an operation or when pipeline behaviors apply shared policies.
How do MediatR pipeline behaviors implement the Chain of Responsibility pattern?
Pipeline behaviors form a chain of wrappers that ends at the request handler. Each
IPipelineBehavior<TRequest, TResponse>receives the request and anextdelegate for the rest of the chain, and decides whether to pass the request on.
Sendnests the behaviors registered for the request type around the handler.- The outermost behavior runs its code before
next, such as logging or validating the request.- Calling
nextinvokes the following behavior, and the innermostnextinvokes the handler.- As calls return, each behavior can run code after
next, such as recording elapsed time or adjusting the response.- A behavior can short-circuit by returning a response or throwing without calling
next, as a caching behavior does when it returns a stored result. Later behaviors and the handler then never run.Registration order determines the nesting, so it changes what happens. A logging behavior outside a validation behavior records rejected requests; inside it, rejected requests never reach the logger. Order that matters for correctness should be explicit and covered by tests rather than left to incidental registration order.
What is the Memento pattern?
The Memento pattern captures an object’s state as a snapshot so the object can be restored to it later, without exposing the object’s internals to the code that keeps the history.
- Originator: the object whose state is saved, such as a shopping cart. It creates mementos and restores itself from them, so it is the only code that interprets the saved state.
- Memento: the snapshot itself, ideally immutable and opaque to everything except the originator.
- Caretaker: stores mementos, for example in an undo stack, without inspecting or changing them.
Undo follows from these roles: before a change, the originator produces a memento and the caretaker keeps it; to undo, the caretaker hands the memento back and the originator restores its state from it.
The snapshot must be a real copy. If it shares mutable objects with live state, later edits also change the saved snapshot, so mutable nested objects need deep copies.
In C#, opacity is often only a convention: a public memento record can still be read by the caretaker. A stricter design exposes an opaque interface or nests the memento type inside the originator.
How can memento history be kept from growing without limit?
Memento history is bounded by a retention limit: a maximum number of snapshots, a memory budget, or both. When a new snapshot would exceed the limit, the oldest one is discarded.
Each snapshot copies the originator’s state, so an unbounded history grows with every recorded change. The right limit depends on what the history is for:
- Undo in an editor: a rolling window keeps the most recent steps. Steps older than the window can no longer be undone.
- Large snapshots: a memory budget caps total size, because a snapshot count alone does not control memory when individual snapshots can be large.
- Recovery after a crash or abandoned session: often only the latest checkpoint is needed, so earlier snapshots can be dropped.
Storing a small inverse change instead of a full copy makes each entry smaller, but it does not bound the history by itself.
A full audit trail is a different requirement. It needs durable records of events or changes, as in Event Sourcing, rather than an unlimited in-memory stack of object copies.
When is Memento overkill compared to simpler approaches?
Memento is overkill when each change is small and isolated compared with the whole state. Recording how to reverse that one change, as Command-style undo does, is then cheaper and simpler than copying the entire state before every step.
Aspect Inverse change (Command-style undo) Memento snapshot What each history entry stores Only what reverses one action, such as a removed item and its former position. A copy of the originator’s earlier state. Cost per entry Grows with the size of the change, so small edits to a large object stay cheap. Grows with the size of the state, so every step pays for a full copy even when little changed. Main correctness risk The entry must keep enough context to reverse the change safely; a missing detail produces a wrong reversal. The copy must not share mutable objects with live state; otherwise later edits also change the snapshot. Best fit Isolated, reversible edits. One action that changes several interdependent values, or a need to restore a complete earlier state. A Memento earns its cost when one action touches several related values. Restoring the snapshot resets them together, so no separate inverse logic is needed for each value.
What is the Observer pattern?
The Observer pattern defines a one-to-many relationship: a publisher keeps a list of independently registered subscribers and notifies all of them when its state changes. The publisher depends only on a notification contract, not on the subscribers’ concrete types, so a subscriber can be added or removed without changing the publisher.
- Publisher (subject): owns the changing state, keeps the subscription list and sends the notifications.
- Subscribers (observers): register through the contract and decide how to react.
- Notification contract: a callback signature or interface, such as an order’s
StatusChangedevent or anIOrderStatusObserverinterface.C# events provide this directly:
+=subscribes a handler,-=removes it, and raising the event invokes the handlers currently subscribed. Event handlers are invoked synchronously, so asynchronous server-side subscribers fit better behind aTask-returning interface that keeps their completion and failures observable..NET events · Observer implementation · Events and observable APIs in .NET
How should event subscriptions be managed to avoid retaining subscribers after their lifetime ends?
Each subscription needs an owner that removes it when the subscriber’s lifetime ends, usually the component that added it. Weak events are a fallback when a longer-lived publisher has no clear owner to unsubscribe.
Why retention happens: subscribing stores a delegate in the publisher, and that delegate usually references the subscriber. While the publisher stays reachable, so does the subscriber. This becomes a leak when the publisher outlives the subscriber, such as a closed view whose handler is still attached to a surviving view model.
- Explicit ownership: the component that subscribes with
+=unsubscribes with-=when its lifetime ends, such as when a view closes or the subscriber is disposed.- Keeping the handler: unsubscribing needs the same delegate that was added, so a lambda handler has to be stored if it will be removed later.
- Weak events: the publisher’s reference no longer keeps the subscriber alive, so no unsubscribe owner is required. The extra machinery is worthwhile only when ownership cannot be aligned with the subscriber’s lifetime.
When an owner exists, explicit unsubscription is simpler and more predictable.
What is the State pattern?
The State pattern moves state-dependent behavior into separate state objects that share one interface. A context object holds its current state and delegates each operation to it, so replacing the state object changes how the same operation behaves.
For example,
Order.Ship()delegates to the order’s current state:
PendingStaterejects the call because the order is unpaid.PaidStatetransitions the order toShippedState.ShippedStaterejects the call because the order has already shipped.In this example, each state decides which transitions it allows. That ownership is a design choice: either the context or a concrete state may decide which state comes next and when. In both forms, callers request domain operations such as
Pay()orShip()instead of setting a status directly.The pattern replaces the same
switchon a status repeated across many operations. A new state becomes one new class, plus edits wherever transitions to or from it are defined. The cost is many small classes, and a new operation must be implemented in every state. A small machine with few state-dependent rules is often clearer as an enum with one transition function.Gamma et al.: Design Patterns, State chapter · MSDN Magazine: Solidify your C# application architecture with design patterns · State pattern implementation · State tradeoffs
How does a compiler-generated async state machine relate to the State pattern?
Both let stored state decide what happens next, but the async state machine is a compiler implementation technique, not the GoF State pattern.
Aspect Compiler-generated async state machine GoF State pattern What the state means Where the method paused, meaning which incomplete awaitit is waiting on.The object’s current mode in the domain, such as PendingorPaid.How state is stored A generated state field, plus fields for the locals that must survive the await.The context holds a reference to a state object that implements a shared interface. How behavior changes When the awaited operation completes, its continuation calls MoveNext(), which branches on the state field and resumes after thatawait.The context delegates each operation to its current state object; replacing that object changes behavior. Who designs it The compiler generates it from an asyncmethod.Developers design the state classes and the transitions between them. The async machine therefore shows state-driven control flow built from generated fields and branching, not from interchangeable state objects.
C#: async scenarios · Stephen Toub: How Async/Await Really Works in C# · Familiar state machines · State pattern implementation
What is the Strategy pattern?
Strategy puts interchangeable algorithms for one step of a workflow behind a common contract. The code that runs the workflow calls the contract without knowing which algorithm it is using, so algorithms can be added or replaced independently of that workflow.
- Context: owns the workflow and delegates the variable step. For example, a shipping service calls
IShippingCostStrategy.Calculate(order)without containing any pricing rules.- Strategy contract: the interface, or a delegate for a single operation, that every algorithm implements.
- Concrete strategies: the algorithms themselves, such as flat-rate, weight-based or zone-based pricing.
- Selection: a client, configuration or selection policy decides which strategy the context uses.
Problem it solves: without Strategy, one conditional chain holds every variant. Adding same-day delivery then means editing the method that already carries all the other pricing rules, with a risk of disturbing them. With Strategy, a new variant is a new implementation plus its registration, and existing strategies stay unchanged.
Separating the algorithms does not choose between them. When several could apply, the selection rule remains its own design decision.
In .NET,
IComparer<T>is a familiar example: it supplies the ordering that sorting methods use.Strategy: Problem · Strategy: Solution · Microsoft:
IComparer<T>
What determines whether a strategy should use an interface or a
Func<T, TResult>delegate?The shape of the collaboration decides, not whether the strategy holds state. A
Func<T, TResult>fits when the strategy is a single operation whose parameters and captured variables supply everything it needs. An interface is clearer when the strategy has several related members, its own dependencies, or a lifetime that must be managed.
- Delegate: any lambda or method group with a matching signature can be passed without declaring a type. LINQ predicates and key selectors are strategies of this kind.
- Interface: a named contract keeps related members together. Each implementation can declare constructor dependencies, which a DI container supplies with a chosen lifetime. For example, a shipping-cost strategy that exposes
Priority,AppliesToandCalculateand needs anIZoneCalculatoris clearer as an interface than as three separate delegates.State alone does not decide the choice. A lambda can capture outer variables, and a captured variable is not collected while the delegate that references it is still reachable. A delegate can therefore carry state too.
Microsoft: Lambda capture of outer variables · Microsoft: .NET dependency injection
How should strategy selection work when several strategies can handle the same request?
Overlaps need an explicit rule owned by the domain: a unique key, a documented precedence, or deliberate composition. The order in which a DI container returns registrations is not such a rule.
- Explicit key: the request carries an identifier, such as the chosen shipping method, that maps to exactly one strategy. Overlaps cannot occur, and an unknown key fails visibly.
- Documented precedence: each strategy reports whether it applies and declares a priority. The highest-priority applicable strategy wins. A tie at the top priority is a configuration error, and it is detected rather than resolved arbitrarily.
- Composition: when combining is the intended behavior, such as applying several eligible discounts, every matching strategy runs under a defined combination rule.
In .NET’s built-in container, resolving a single service returns the last registration, and resolving
IEnumerable<T>returns all registrations in registration order. That order is deterministic, but it reflects how the application was wired, not a business decision. Reordering registrations would silently change which strategy wins, so any intended order belongs in explicit data such as a priority.
What should determine whether algorithm variation uses Template Method or Strategy?
The decision depends on who owns the algorithm and when the variation must be chosen. Template Method fits when a base class owns a stable sequence and subclasses vary a few steps. Strategy fits when the varying behavior is a separate collaborator that must be selected or replaced independently, including at runtime.
Aspect Template Method Strategy Who owns the algorithm The base class runs the sequence and calls hooks that subclasses override. The context calls a composed object or delegate that supplies the varying behavior. When a variant is chosen When a subclass is instantiated, so the choice is fixed for that object. When a strategy is passed in or selected, so it can change at runtime. Coupling Subclasses depend on the base class’s hook order and shared state, so a base-class change can affect every subtype. Inheritance also uses the class’s single base-class slot. Variation stays outside the class hierarchy, so the base-class slot remains free. Cost Smallest design when the base genuinely owns most of the workflow. Adds a separate abstraction and, with several strategies, logic to select among them. Template Method fits poorly when behavior must change at runtime or when inheritance exists only to reuse a few lines.
What is the "Hollywood Principle," and how does Template Method apply it?
The Hollywood Principle means the framework or base class owns the control flow and calls supplied extension points. Template Method applies it by fixing the algorithm’s sequence while subclasses implement selected hooks; subclasses follow that protocol instead of driving the whole operation. Template Method implementation
What is double dispatch, and why does the Visitor pattern need it?
Double dispatch chooses the code to run from the runtime types of two objects, here the element and the visitor. Visitor needs it because an ordinary virtual call dispatches on only one runtime type, and ordinary C# overload resolution picks a
Visitoverload at compile time from the argument’s static type. Through anICartItemreference, the concrete element type is not available for that choice.Visitor recovers it in two dispatches:
- The caller invokes
item.Accept(visitor). Virtual dispatch onitemruns the concrete element’sAccept, such asPhysicalProduct.Accept.- Inside that method,
thishas the static typePhysicalProduct, sovisitor.Visit(this)binds to theVisit(PhysicalProduct)overload at compile time.- That call is virtual on the visitor, so it runs the concrete visitor’s implementation, such as
TaxCalculatorVisitor.Visit(PhysicalProduct).The first dispatch fixes the element type and the second fixes the operation. New operations can therefore be added as new visitor classes without changing the element classes.
When does EF Core use
ExpressionVisitor, and what does it do?EF Core uses expression visitors when it translates a LINQ query for execution. A series of visitor passes walks the query’s expression tree, reshapes it into a form the provider can handle, and translates the supported parts into the provider’s query, such as SQL.
- Traversal:
ExpressionVisitorprovides node-specific visit methods, so a pass can handle method calls, member access or constants differently.- Rewriting: a visit method can return a replacement node, so each pass hands a reshaped tree to the next. No single visitor simply emits SQL node by node.
- Phases: preprocessing passes reshape the query, and translation passes then convert it into the provider’s representation.
Each pass is a separate operation over the same tree structure, which is the kind of variation Visitor is designed for.
A translation error does not by itself show that a visit method or overload is missing. It can occur when the active provider lacks a server-side translation for that expression in that query context, but the error alone does not establish a single cause.
Visitor framework examples · EF Core: client vs. server evaluation · EF Core source: QueryTranslationPreprocessor
When is pattern matching a better fit than Visitor?
Pattern matching is a better fit when the type set is small or when there are only a few short operations. A
switchexpression keeps the type dispatch visible in one function, withoutAcceptmethods or a visitor interface. A closed type set does not decide the choice by itself, because Visitor also depends on a stable element set. Visitor earns its ceremony when element types are stable, new operations are added often, and those operations benefit from carrying state across elements.
Aspect Pattern matching Visitor Adding an operation Another function with a switchover the types.Another visitor class; element classes do not change. Adding an element type A new case in each relevant switch.A new Visitmethod on the visitor interface, implemented in every visitor.Detecting a missed case Over an open hierarchy with a discard _arm, the compiler cannot report a forgotten type; the value reaches the discard, which may throw at run time.Adding the method to the visitor interface makes every visitor that lacks it fail to compile. State across elements Awkward; a switchholds no state between elements, so an accumulator lives in surrounding code that the arms or the caller update.Natural; a visitor accumulates results in its own fields during traversal. Readability Dispatch is explicit in one place. Double dispatch through Acceptis harder to follow.
What should happen when an Abstract Factory gains a new product type such as
IFraudDetector?It depends on whether every family must provide the new product. If so, the product is added to the abstract factory interface and implemented in every concrete factory. If the capability varies independently of the family, it belongs in a separate collaborator instead of the factory.
- Required in every family: the factory interface gains a method such as
CreateFraudDetector(), and each provider’s factory returns its own implementation. A concrete factory without that method fails to compile, so no family is left incomplete. This is the pattern’s known cost: each new product type changes the interface and every family.- Independent of the family: when fraud detection does not have to match the payment provider, it is injected on its own rather than forced into every family’s factory.
A default no-op implementation avoids the compile errors but hides a missing required behavior. For a safety check such as fraud detection, payments would then proceed unchecked without any error.
Splitting the factory interface is justified only when consumers use genuinely separate families; arbitrary splitting hides the fact that the products must vary together.
When is Abstract Factory overkill compared to a simpler approach?
Abstract Factory is overkill when only one product type varies, when a single provider is fixed, or when the products can be chosen independently. Injecting each product directly is simpler in those cases.
The pattern adds a factory interface and one concrete factory per family. That layer pays off only when several product types must vary together. Choosing a payment provider, for example, must also choose its matching processor, receipt generator and refund handler, and the factory makes that choice once. Without such a coordinated compatibility requirement, the extra indirection buys nothing.
The deciding pressure is coordinated variation, not a particular number of providers or classes.
How does Abstract Factory relate to the DI container in modern .NET?
They solve different problems and are often combined. A DI container is a general object factory that builds object graphs from registrations. An Abstract Factory is a typed creation boundary: choosing one concrete factory chooses a whole family of related products.
- What the container provides: each service is resolved from its own registration. Registrations can switch several services together, but
IServiceProviderdoes not check that the resolved implementations belong to the same provider, so a Stripe processor can sit next to a PayPal receipt generator by mistake.- What the factory adds: consumers depend on one interface, such as
IPaymentProviderFactory, and each concrete factory creates all of its family’s products. The compatibility decision lives in one place instead of being spread across registrations.- How they combine: the container registers and injects the chosen concrete factory, for example with
AddSingleton<IPaymentProviderFactory>(...). One factory registered at startup fixes one family for the whole application; choosing a family per request needs a registry or strategy that selects among factories.The factory centralizes compatibility but does not prove it. Its shared product interfaces cannot show that the returned implementations come from one provider; provider-specific types can move more of that check into the type system.
.NET dependency injection · Abstract Factory framework examples · Abstract Factory pitfalls
When is a Builder worth using instead of a constructor?
A Builder is worth its extra type when constructing the object is a process rather than one call. This holds when inputs arrive through several steps, often with many of them optional, or when the finished object needs validation or derived values that depend on the complete set of inputs. A constructor, or an object initializer with
requiredmembers, stays clearer when the values fit in one readable call and the constructor or initializer can enforce the same invariants.What a Builder adds over a long constructor:
- Named steps instead of positional arguments: calls such as
ShipTo(address)andWithGiftWrap(message)replace a parameter list whose call site reads liketrue, null, false. A new option becomes another named method instead of another placeholder argument for every caller.- Cross-field validation in one place:
Build()sees the whole configuration. A rule such as requiring a shipping address and at least one item is therefore checked once, before the object exists.- Derived values and defaults:
Build()computes values such as the order total and applies defaults. One example is using the shipping address for billing when no billing address was given. Callers no longer have to do this themselves.The costs are an extra type that mirrors the product’s API and, for any required input that is checked only in
Build(), runtime rather than compile-time enforcement of its presence, whichrequiredmembers provide. Mandatory inputs taken through the builder’s own constructor, such as the customer innew OrderBuilder(customer), must still be passed when the builder is created. A builder around independent property assignments only duplicates an object initializer.Builder tradeoffs · Builder implementation · Microsoft: required modifier (C# reference)
Why does
WebApplicationBuilderuse a builder instead of a constructor with parameters?An ASP.NET Core application is configured by many independent setup steps, not by one fixed set of arguments.
WebApplicationBuildercollects that setup in stages, andBuild()creates theWebApplicationonly after everything has been contributed.How the stages work:
WebApplication.CreateBuilder(args)creates the builder with preconfigured defaults, such as the standard configuration sources.- Setup code adds to the builder’s properties:
- configuration through
builder.Configuration- service registrations through
builder.Services, including registrations contributed by library extension methods- host and web-server settings through
builder.Hostandbuilder.WebHostbuilder.Build()creates theWebApplicationfrom the accumulated configuration and registrations.A constructor with parameters would require callers to assemble the complete set of services and settings before a single call. That set is open-ended, because the application and its libraries both contribute to it. The builder therefore gives every contributor a shared place to add its part before anything is assembled. Staging also lets later setup steps read earlier results, such as configuration values that decide which services to register. A constructor is still used at the end: in ASP.NET Core 10,
Build()builds the underlying host and passes it to theWebApplicationconstructor.Microsoft: WebApplicationBuilder class · Microsoft: Configuration in ASP.NET Core · Microsoft: .NET dependency injection · ASP.NET Core 10: WebApplicationBuilder source
How do Factory Method and Abstract Factory differ?
Both hide which concrete class is created, but they vary different things. Factory Method lets a subclass decide one product by overriding a creation hook, which inherited creator code such as a shared workflow often calls. Abstract Factory is a separate object, given to its client, that creates a whole family of related products that must work together.
Aspect Factory Method Abstract Factory What varies One product returned by an overridable creation hook, often used inside a shared workflow, such as the sender a notification workflow uses. A family of compatible products, such as a whole Stripe or PayPal provider family that is swapped together. Mechanism Inheritance: the creator declares an overridable creation hook such as CreateSender(), and concrete creators override it to choose the product. This hook is what defines the pattern.Composition: the client holds a factory object with one creation method per product and calls it without knowing which family it is. Where creation sits In the creator’s overridable hook. The same creator often also runs code that uses the product, but it need not. In a dedicated factory, separate from the code that uses the products. Adding a variant A new creator subclass overrides the hook to return a new product, leaving any creator code that uses it unchanged. Creator and product hierarchies grow in parallel only when each concrete product gets its own creator subclass. A new factory implementation supplies a whole new family. Adding a new kind of product to the family changes the factory interface and every family implementation. What it protects Creator code that uses the product, when it exists, stays unchanged while one creation decision varies. Products obtained from one factory belong to the same family, as long as each factory creates only matching products. When neither a subclass-overridable creation hook nor a product family is needed, a standalone factory function is usually smaller than both.
GoF: Design Patterns: Elements of Reusable Object-Oriented Software · GoF: Abstract Factory (InformIT excerpt) · Factory Method solution · Abstract Factory solution · Creational patterns at a glance
When does Factory Method become the wrong choice?
Factory Method is the wrong choice when no creator needs an overridable creation hook through which subclasses decide the product. That hook defines the pattern. A shared base workflow that uses the product is a common reason the hook pays off, but it is not required. When nothing needs a subclass creation decision, the creator hierarchy is ceremony.
Simpler or better-fitting options:
- Implementation chosen once at startup: a DI registration of the product interface, such as
INotificationSendermapped toEmailNotificationSender, selects the implementation without a creator subclass per variant.- Plain construction: a static factory method or a constructor is smaller when creating the object needs no subclass decision. Many APIs called factory methods, such as
Task.FromResult<T>(), are static functions rather than the GoF pattern.- Several related products must vary together: Abstract Factory keeps the compatible family behind one factory object instead of separate creation hooks.
When each variant gets its own creator subclass, as in the notification example, the cost being avoided is a parallel hierarchy: each new variant adds both a product class and a matching creator class, even when the creator contributes nothing beyond picking the product.
Related .NET factory APIs · Factory Method solution · Microsoft: .NET dependency injection · GoF: Factory Method pattern
How does Factory Method support the Open/Closed Principle?
Factory Method lets a new product variant arrive as new code, in the form of a creator subclass, while the creator’s existing workflow stays unchanged. That is the Open/Closed Principle at one variation point: the workflow is open to new products but closed to modification.
How a new variant is added:
- The base creator runs its workflow and gets the product only through the overridable factory method. In the notification example this is
CreateSender(), which returns the product interfaceINotificationSender.- A new channel, such as Slack, adds a
SlackNotificationSenderand a creator subclass whose override returns it.- The composition root registers the new creator. The shared workflow, the existing creators and the code that calls the workflow are not edited.
The protection covers only the anticipated variation point. A variant that needs a different product contract, or a change to the shared workflow, still requires editing existing code. The composition root that selects the creator also changes when the new creator is put into use.
Robert C. Martin: Design Principles and Design Patterns · Robert C. Martin: SOLID course notes · Factory Method solution
When is Prototype useful instead of constructing an object directly?
Prototype is useful when a new object should start as a copy of an existing instance rather than be built from scratch. Two situations justify it:
- A configured template exists: many objects differ from a base in only a few values. Product variants, for example, share the base product’s name, description, category and shipping rules. Copying the template and overriding the differences avoids manual copy code that lists every member, which silently produces incomplete objects when a newly added member is missed.
- The caller does not know the concrete runtime type: code that holds an object through a base type cannot call the right constructor, but the object knows its own type and can copy itself through its own copy method.
Direct construction is clearer when the type and the values are already known and no template carries useful state.
A prototype still needs a defined copy contract:
- which members stay shared
- which are duplicated
- which get fresh values, such as a new identifier
In C#,
record with { }makes a shallow copy with selected members replaced, and an explicit copy constructor states deeper copy rules.Prototype solution · Prototype pitfalls · Microsoft: Records (C# reference)
What separates a shallow copy from a deep copy, and which nested objects must a deep copy duplicate?
A shallow copy creates a new outer object, but its reference-type fields still point to the same nested objects as the original. A deep copy also duplicates nested objects, so changing them through the copy does not affect the original. A practical deep copy duplicates the mutable nested state that must evolve independently, not everything reachable.
Aspect Shallow copy Deep copy Outer object A new instance. A new instance. Value-type fields Copied bit by bit. Copied. Reference-type fields Only the reference is copied, so both objects share the same nested instance. Members covered by the copy policy get new instances. Adding an item to a nested List<T>The change is visible through both objects. Only the object that was changed sees it. .NET mechanisms Object.MemberwiseClone()andrecord with { }.A copy constructor or copy method that duplicates chosen members. Where the boundary goes:
- Duplicated: mutable state that the copy owns and changes independently, such as an order’s line items and its shipping address.
- Shared: immutable objects, such as immutable collections, because neither copy can change them.
- Shared deliberately: objects that are meant to be common, such as the
Customeran order refers to.- Left out or reset: runtime-only state such as event subscriptions, locks and caches. A cloned entity usually also needs a new identifier.
A member typed as
IReadOnlyList<T>only limits the exposed API. The underlying list may still be mutable and shared.Microsoft: Object.MemberwiseClone · Microsoft: Records (C# reference) · Prototype pitfalls
How does a DI singleton differ from the classical Singleton pattern?
A classical Singleton enforces a single instance inside the class itself and exposes it through a global static access point. A DI singleton is an ordinary class that the container creates once per root provider and injects into consumers, so its single instance is a container lifetime rule rather than a property of the type.
Aspect Classical Singleton DI singleton Who enforces one instance The type itself, through a private constructor and a static instance field, often initialized with Lazy<T>.The container: after AddSingleton<T>(), the root provider creates the instance on first request and returns the same one for later resolutions.How consumers get it Through a static access point such as AppConfig.Instance, so the dependency is hidden inside method bodies.Through constructor injection, so the dependency is visible in the consumer’s constructor. How far the single instance extends Every caller of the static access point shares it. One root provider and registration. A second provider, such as one created by calling BuildServiceProvideragain, or direct construction withnewproduces another instance.Testing and replacement Consumers cannot be handed a substitute, and tests share the static state. Tests pass a fake or substitute to the consumer’s constructor. Creation and disposal Controlled by the type. Controlled by the container, which disposes instances it created when the provider is disposed. In both forms, one instance serves concurrent callers, so the instance’s own state must be thread-safe.
Microsoft: Service lifetimes · Microsoft: .NET dependency injection · Classical Singleton problem · Singleton lifetime in .NET
What must be true before choosing singleton lifetime for mutable state?
Singleton lifetime suits mutable state only when four conditions hold: sharing is intended, concurrent access is safe, retained memory is bounded, and no scoped state is captured. One instance serves every caller of its provider until the provider is disposed, usually at application shutdown, so each condition guards against a specific failure.
- Sharing is intended: every request and user resolved from that provider sees the same state. If the data really belongs to one request or one user, it leaks between callers.
- Concurrent access is safe: one instance serves concurrent requests. The container makes resolving the service thread-safe, but not the service’s fields or dependencies, so unsynchronized mutation causes races and corrupted state.
- Retained memory is bounded: the singleton and its dependency graph live as long as the provider. An unbounded cache keeps growing, and failed or stale state survives far longer than intended.
- No scoped state is captured: a singleton that holds a scoped service keeps that instance beyond its intended scope. This is a captive dependency. When scope validation is enabled, it rejects constructor-injecting a scoped service into a singleton.
When any condition fails, a shorter lifetime such as scoped or transient fits better, or the state needs a different owner. Singleton work that genuinely needs scoped services can create and dispose an explicit scope for each operation.
Microsoft: Dependency injection guidelines · Microsoft: Service lifetimes · Singleton pitfalls
How can an Adapter be distinguished from a Facade when both wrap another system?
An Adapter makes an existing component usable through a contract that a client already expects. Its defining job is compatibility. A Facade gives clients a new, simpler interface to a complex subsystem. Its defining job is simplification, which can include coordinating several subsystem calls behind one operation.
Aspect Adapter Facade Problem it solves The client depends on one interface, but the existing component exposes an incompatible one. Using the subsystem directly is complex, for example because clients must know many of its components or repeat the same sequence of calls. Interface it exposes The target contract the client already uses, such as IInventoryService.A new, higher-level interface shaped around what clients need, such as a checkout operation PlaceOrderAsync.Work inside Maps method names, reshapes data, and converts results and error codes between the two contracts. Hides subsystem details behind that interface. When a use case needs several steps, it calls the components in the required order and owns that workflow. Without it The two sides cannot work together unless equivalent translation is written somewhere else, such as in the client. Clients can still use the subsystem directly, but each one has to handle its complexity. Where a workflow is involved, each client duplicates it, and the copies can drift in sequencing or error handling. A Facade may also contain some translation. The defining responsibility separates the two: compatibility for an Adapter, simplification for a Facade.
Design Patterns: Adapter/Facade (Microsoft Learn) · Adapter: Solution · Facade: Problem · Facade: Tradeoffs
When does the Bridge pattern add value beyond choosing one implementation behind an interface?
Bridge adds value when a design has two dimensions that vary independently, such as payment operations and payment providers, and operations must combine freely with providers. Choosing one implementation behind one interface varies only one dimension. That is ordinary polymorphism or a dependency-injection registration, or Strategy when the implementations are interchangeable algorithms for one behavior.
Signals that a second dimension is real:
- Both sides change: new operation variants and new providers are both being added, or are about to be.
- Code multiplies by combination: methods differ only by provider, such as
ProcessStripeRefundAsyncandProcessPayPalRefundAsync. Each new operation is then written once per provider, and each new provider repeats every operation.- Combinations are meaningful: every operation genuinely works with every provider. If subscriptions work with only one provider, the dimensions are coupled, and a Bridge advertises flexibility the system does not have.
With one operation and one implementation, two hierarchies add indirection without a payoff, and a direct implementation is easier to trace.
What is the Bridge pattern?
Bridge separates an abstraction from its implementation into two class hierarchies joined by composition, so each hierarchy can be extended independently. The abstraction holds a reference to an implementation interface and delegates low-level work to it, instead of inheriting from a concrete implementation.
Payment example:
- Abstraction hierarchy:
PaymentOperation, with variants such asSingleChargePayment,SubscriptionPaymentandPartialRefundPayment. Each variant defines what the operation does.- Implementation interface (the bridge):
IPaymentGateway, with primitives such as authorize, capture, refund and create subscription.- Concrete implementations:
StripeGateway,PayPalGatewayandBankTransferGateway. Each maps those primitives to one provider’s API.An operation receives a gateway and calls its primitives. For example, a single charge authorizes and then captures. Any operation can therefore run on any provider:
- A new provider is one new
IPaymentGatewayimplementation, and it works with every existing operation.- A new operation built from existing primitives is one new
PaymentOperationsubclass, and it works with every provider.Without the bridge, methods such as
ProcessStripeChargeAsyncandProcessPayPalRefundAsynccover every provider-and-operation pair, and their number grows as providers × operations. If only the provider side varied, one interface with several implementations would be ordinary polymorphism. The second, abstraction-side hierarchy is what makes the design a Bridge.Limit: Bridge prevents the cross-product, not change to the shared contract. An operation that needs a genuinely new primitive changes
IPaymentGatewayand every provider implementation.
How does Bridge differ from dependency injection?
Dependency injection and Bridge answer different questions. DI is an inversion-of-control technique: a class receives its dependencies from outside instead of creating them itself. Bridge is a class structure: an abstraction hierarchy and an implementation hierarchy are separated so that each can vary independently.
- What each decides: DI decides how an object obtains a collaborator. Bridge decides how a design is split into two hierarchies and where they connect.
- What each requires: DI applies to any dependency, even a single one. Bridge requires two dimensions that vary independently, such as payment operations and payment providers.
- How they combine: DI can supply the bridge, for example by injecting an
IPaymentGatewayinto aPaymentOperation. A Bridge can also be wired by hand, as innew SingleChargePayment(stripeGateway).When DI selects one implementation behind one interface, such as one
IDistributedCacheprovider, the result is ordinary polymorphism. It becomes Bridge-like only when a separate, independently changing abstraction composes with that implementation contract.Dependency injection in .NET · Bridge: Solution · Bridge: Related .NET Abstractions
What is the Composite pattern?
Composite represents a part-whole tree so that clients treat a single object and a group of objects through the same interface. A leaf implements an operation directly. A composite implements the same operation by delegating to its children, which may be leaves or other composites, and combining their results.
Example: in a product catalog,
SingleProductandProductBundleboth implementIOrderComponent.GetPrice(). A product returns its own price. A bundle sumsGetPrice()over its children and applies its discount, so a bundle of bundles is priced by the same call. A caller prices any item without checking whether it is a product or a bundle.This removes repeated type checks such as
item is ProductBundlefrom client code. A new leaf type, such as a subscription product, needs no changes to bundles or callers.Limits:
- Child management such as
AddandRemovemakes sense only for composites. Putting it on the shared interface forces leaves to reject those calls.- The tree must stay acyclic. A bundle that contains itself makes recursive operations overflow the stack.
- Composite fits a genuine part-whole hierarchy. A fixed model with two types may be clearer without it.
Composite: Solution · Composite: Pitfalls · Composite: Tradeoffs
How does Composite relate to the Visitor pattern?
Composite and Visitor are complementary. Composite defines the structure of a part-whole tree, and Visitor defines operations over the node types in that tree, so the two can be combined.
- Composite: leaves and groups share one component interface. A group implements each operation by delegating to its children, so callers treat any node the same way.
- Visitor: each operation lives in a separate visitor that handles every node type, instead of being a method on every node class.
In a plain Composite, operations such as
GetPrice()andGetItemCount()are members of the shared interface, so every node class must implement each new operation. Adding Visitor moves operations out of the nodes. That combination fits when the set of node types is stable and new operations are added more often than new node types.
When does a Composite tree become a performance problem?
A Composite tree becomes a performance problem when recursive operations walk a large tree, or walk it very often, and measurements show that the repeated traversal is a significant cost. A call such as
GetPrice()on a bundle visits every node below it each time it runs, so its work grows with the size of the traversed subtree.One remedy is to cache an aggregate, such as a bundle’s subtotal, or to keep it updated as the tree changes. That moves cost and risk to mutation: every change below a cached value must invalidate or update it, or reads return stale results. Because the optimization adds this consistency rule and makes mutation harder, measurements of the real workload justify it, not tree size alone.
What is the Decorator pattern?
Decorator adds behavior to an object by wrapping it in another object that implements the same interface. The wrapper holds a reference to the wrapped component, normally delegates calls to it and runs its own work before or after delegating. A wrapper can also end a call without delegating, as a validation decorator does when it rejects invalid input. Because the wrapper still satisfies the original contract, clients use it exactly as they would use the component, and wrappers can be stacked.
Example: a core
IOrderHandlersaves and processes an order, and separate decorators add validation, logging and metrics:IOrderHandler handler = new ValidationOrderHandler( new LoggingOrderHandler( new MetricsOrderHandler( new CoreOrderHandler(repository), metrics), logger));A call enters the outermost wrapper first and moves inward unless a layer ends it, and results return outward through the layers it passed. Each concern lives in its own class, so adding one, such as idempotency, means adding another decorator rather than editing the core handler.
Order is part of the behavior. With validation outside logging, invalid orders are rejected before they are logged. With the layers reversed, every attempt is logged. The costs are more small classes and deeper call stacks to trace.
How does ASP.NET Core Middleware implement the Decorator pattern?
Middleware that delegates to the rest of the pipeline follows the Decorator shape: it wraps the remaining pipeline and adds behavior around it, while every step keeps the same request-handling contract.
- Shared contract: each step is a request delegate that handles an
HttpContext.- Wrapped component: a wrapping middleware receives
next, the delegate for the rest of the pipeline.- Added behavior: code that runs before calling
nextacts on the request, and code that runs after it acts on the response.- Composition: middleware is added in order, and each wrapping middleware can call the next. The registration order sets the request order, and the response passes back through the same middleware in reverse.
- Boundary: a terminal delegate, such as one added with
Run, receives nonextand ends the pipeline. In the Decorator lens it is the innermost component rather than a decorator.As with stacked decorators, order is part of the behavior. A wrapping middleware can also end the request without calling
next. This short-circuiting means later middleware never runs, so placement decides which middleware still sees the request.Decorator is one useful lens on this structure. The ASP.NET Core documentation describes it as a pipeline of request delegates.
When should Decorator replace inheritance for added behavior?
Decorator fits when the added behavior is optional, must be combined with other behaviors in a chosen order, or must wrap a type that cannot be subclassed. An example is a sealed or third-party implementation of an interface the client uses. Inheritance fits when the new type is a stable subtype of its base class rather than an add-on around it.
Aspect Decorator Inheritance How behavior is attached A wrapper implements the same interface, holds the wrapped object and runs work before or after delegating to it. A subclass extends or overrides members of its base class. Combining behaviors Separate wrappers, such as validation, logging and metrics, are stacked when objects are composed, and the order is explicit there. Combinations are fixed in the class hierarchy when the code is written. Coupling Depends only on the shared contract, not on a base-class implementation. Ties the added behavior to the base-class implementation. Cost Adds an object, a delegation call and another layer to trace while debugging. No wrapper object or extra delegation layer.
What's the performance cost of a deep decorator chain?
Each layer can add a cost to every call that passes through it: one more method call that delegates to the inner object. That cost is not fixed. It depends on the generated code, and runtime optimizations such as devirtualization and inlining can reduce it. A wrapper written as an
asyncmethod that awaits the inner call can also carry the bookkeeping of an asynchronous method, which suspends and resumes it when the inner task has not yet completed. The potential overhead grows with the number of layers.Whether it matters depends on the path. On a CPU-bound hot path called very often, the extra calls can be measurable. When the wrapped operation performs I/O, such as a database or HTTP call, the I/O can dominate and make the wrapper overhead small by comparison. Profiling the specific path shows whether the layers are significant. Collapsing layers without that evidence gives up the separation of concerns the decorators provide.
Asynchronous programming scenarios (C#) · Performance Improvements in .NET 10 · Decorator: Pitfalls
When does a Facade become a "god class" anti-pattern?
A Facade simplifies access to a subsystem by coordinating its collaborators. It becomes a god class when it absorbs business rules or mutable domain state instead of delegating those responsibilities to domain services/objects. Difficulty testing it without recreating the domain is a stronger signal than line count. Facade tradeoffs
How does the Flyweight pattern reduce memory use?
Flyweight reduces memory by storing state that many objects have in common once and sharing it, instead of copying it into each object.
- Intrinsic state: the shared, context-independent part, such as a product category’s name, tax rate and shipping rules. It lives in the flyweight and should be immutable, because many owners see the same instance and a write to it would change what all of them observe.
- Extrinsic state: the part unique to each use, such as a product’s ID, SKU, price and stock. It stays with each owner or is passed in by the caller.
- Factory or interning table: returns the existing flyweight for equal intrinsic state and creates one only on the first request.
Example: when 100,000 products each reference one of 50 shared category objects, the category data is stored 50 times instead of 100,000 times.
string.Intern()applies the same idea to strings: the runtime returns one canonical instance for equal values.Flyweight is not object pooling.
ArrayPool<T>lends a mutable buffer to one caller at a time and takes it back. A flyweight is shared by many owners at once.
When is Flyweight not worth the complexity?
Flyweight is not worth its complexity when duplicated state is not a measured memory problem: the object population is modest, the shared part is small, or the state cannot be split cleanly into a shared part and a per-use part. In those cases it adds a factory, a lookup path and an extra level of indirection without relieving a real constraint. It is also a poor fit when the shared part must change per owner, because mutating a shared instance silently changes what every owner sees.
It earns its cost when many objects repeat substantial immutable state and a memory profile shows that the duplication materially contributes to retained memory. The saving grows with the ratio of objects to distinct shared values. For example, 100,000 products can reference 50 shared category objects instead of each holding its own copy of the category data.
What's the difference between a Proxy and a Decorator in terms of intent?
Both wrap an object behind the same interface, so their structure can look identical. A Proxy exists to control access to the real subject. A Decorator exists to add a responsibility around the object it wraps.
Aspect Proxy Decorator Intent Decides whether, when or how the real subject is reached. Extends what a call does by adding a responsibility around the wrapped object. Effect on the real call Can defer creating the subject, return a cached result or reject the call before the subject runs. Normally runs added work before or after delegating to the wrapped object. It can also end a call before delegating as part of that responsibility, as validation does when it rejects invalid input. Typical forms Virtual (lazy creation), caching, protection (authorization) and remote proxies. Logging, validation and metrics layers stacked in a chosen order. Mixing both concerns in one wrapper makes ordering and failure behavior harder to reason about. Order matters even with separate wrappers: if a caching proxy wraps an authorization proxy, a cache hit can return data without the authorization check running.
What is event-driven architecture?
Event-driven architecture (EDA) is a style in which components coordinate by publishing and consuming events. An event records a fact that has already happened, such as
OrderPlaced. A producer publishes the fact without naming the consumers, and each consumer decides what that fact means for its own state. When event communication is asynchronous, the producer also does not wait for the consumers.
Aspect Synchronous direct call Event-driven Knowledge of the receiver The caller names the service it calls. The producer does not know which consumers exist, so new consumers can subscribe without changing it. Timing The caller waits for the response, so both sides must be available at the same time. With asynchronous delivery, consumers can react later, which removes that temporal coupling when an immediate answer is not needed. Outcome The caller learns the result before continuing. With asynchronous delivery, the producer continues without learning the consumers’ results. Other services update their state after processing the event, so cross-service consistency is usually eventual. Not naming consumers does not by itself mean not waiting. In-process domain events can be dispatched synchronously, so the producing code waits for its handlers even though it does not name them.
EDA is a communication model, not a synonym for using a queue. A message broker or retained log is a common transport for durable cross-process events. In-process dispatch, database change streams and HTTP webhooks can also carry events, each with different durability and coupling. Synchronous APIs remain the better fit for queries and decisions that need an immediate answer.
Event-driven architecture style (Azure Architecture Center) · Microsoft: Domain events: design and implementation
What determines whether an event-driven workflow uses orchestration or choreography?
The choice depends on whether one component must own the workflow’s order, state and recovery. Choreography fits when services can react to published events independently and no component needs to own the whole workflow. Orchestration fits when ordered steps, current progress and failure recovery must be tracked and controlled in one place. Asynchronous messaging supports both styles, so using events does not force choreography.
Aspect Choreography Orchestration Who decides the next step Each service decides how to react to the events it receives; no component directs the others. A coordinator decides which step runs next and tells the participant to perform it. Workflow state Spread across the participants, so no single component knows the overall progress. Held by the coordinator, which knows which steps have completed. Failure and recovery Each participant reacts to failure events itself, so recovery logic is distributed. The coordinator detects a failed step and triggers retries or compensating actions. Adding a reaction A new subscriber can start consuming an event without any change to the producer. A new step usually means changing the coordinator’s workflow. Example: after
OrderPlaced, sending a confirmation email and updating analytics are independent reactions, so choreography fits. A checkout reserves stock, then takes payment, and releases the stock if payment fails. Ordered steps and compensation alone do not require an orchestrator, because a choreographed saga can also chain local transactions and compensations through events. Orchestration fits this checkout when one component must explicitly own its order, progress and recovery. Both styles can exist in one system.The number of events or services does not decide the style by itself; ownership of order, state and recovery does.
Event-driven architecture style (Azure Architecture Center) · Saga distributed transactions pattern (Azure Architecture Center)
How do domain events, integration events, and event notifications differ?
Domain and integration events differ by boundary and contract. A domain event records a business fact inside one bounded context, while an integration event publishes a stable contract that other services depend on. Event notification is a different axis: it describes a payload and retrieval style in which the event only signals that something changed and consumers fetch the details themselves. An integration event can therefore be notification-style.
Aspect Domain event Integration event Purpose Records a business fact inside one bounded context, such as InvoiceIssued.Shares a fact with other services as a published contract, such as OrderPlacedIntegrationEvent.Audience Handlers in the same bounded context. Other services. When it is raised or published Raised by domain logic when local business state changes. It can be dispatched synchronously in process. Published only after the local transaction succeeds, usually through an outbox and publisher. Payload Can follow the internal model. It typically carries rich domain data, but that richness is not what makes it a domain event. A stable DTO, so consumers do not depend on the internal model. Main constraint Normally stays inside the service; leaving it requires an explicit mapping to an integration event. Versioning and backward compatibility become part of the contract, because consumers deploy independently. Event notification describes what an event carries, not where it travels:
- It carries minimal identifiers and metadata, such as
CatalogItemChanged { ItemId, ChangedAt }.- Consumers fetch current state from the source when they need more detail.
- It suits cache invalidation and broad fan-out, but couples consumers to the source’s availability and read API.
- An integration event can use this style when other services only need to know that something changed.
The domain fact is modeled first. Only the part that another service can safely depend on is mapped into an integration event.
Microsoft: Domain events: design and implementation · Martin Fowler: What do you mean by “Event-Driven”?
What is a microservices architecture?
A microservices architecture splits a system into services that can be deployed independently. Each service is built around one business capability, owns that capability’s data, and works with other services only through network contracts such as APIs and events.
- Independent deployment: a service can be released, rolled back and scaled without a coordinated release of the whole system. This is the property the style exists for; running code in separate processes does not by itself make it a microservice architecture.
- Business-capability boundaries: services such as Orders, Inventory and Billing each contain their own rules and language, instead of the system being split by technical layer.
- Exclusive data ownership: only the owning service accesses its data directly, and other services go through its contract. Separate database servers are optional; exclusive ownership is not.
- Contract-based integration: versioned APIs and events carry data between services, so one service’s internals can change without forcing another team to release.
The cost is that calls become network calls that can fail partially, workflows that span services lose single-database transactions, and every service needs its own pipeline, monitoring and on-call ownership.
Why do microservices create distributed data consistency problems, and what keeps a cross-service workflow reliable?
Each service commits only to its own datastore, so a business operation that spans several services has no single local ACID transaction by default. One service can commit while a later step fails, which leaves partial business state. While the workflow runs, other services can also observe intermediate state, so the overall result is eventually consistent rather than atomic.
A reliable cross-service workflow combines separate mechanisms, each covering a different failure:
Mechanism Failure it handles How it handles it Saga A later step fails after earlier steps have committed The workflow runs as a sequence of local transactions. When a step fails, compensating business actions counteract completed steps where possible, such as refunding a payment or releasing a stock reservation. Transactional outbox The service crashes between committing its data and publishing the related message The state change and the outgoing message are written in the same local transaction, and a relay publishes the message afterward. A committed change therefore always has a stored message waiting to be published, and a rolled-back change is never announced. Inbox or deduplication record The same message arrives again, for example after a relay retry or an unconfirmed acknowledgement The consumer records which messages it has already processed and skips repeats. Idempotent effects A command or handler is retried Repeating the operation leaves the same result as performing it once. Compensation is a new business action, not a database rollback, and it has two separate limits. Some committed steps have no compensating action at all. Such noncompensable (pivot) work cannot be undone once it commits, so compensation cannot restore the earlier state. Where a compensating action does exist, it can itself fail. The outbox does not make delivery exactly-once: a message can still arrive more than once, which is why consumers need deduplication or idempotent effects.
Azure: Saga pattern · Azure: Transactional outbox with Azure Cosmos DB · Azure: Event-driven architecture style
What architecture usually fits a new product, and what evidence would justify moving to microservices?
A new product is usually best served by a monolith, ideally a modular monolith with explicit internal boundaries. Moving toward microservices is justified one capability at a time, when measured evidence shows that a stable capability repeatedly needs independent deployment, scaling, ownership or isolation, and the team can operate the resulting distributed system.
Why one deployment usually fits first:
- Domain boundaries in a new product are still moving. Inside one deployment, moving a boundary is an ordinary refactoring; between services, it means changing contracts, data ownership and deployments.
- Calls between collocated modules stay in-process, and transactions across those modules can stay local. These internal calls need no network failure handling or cross-service consistency work yet. Calls to external dependencies, such as a database or a third-party API, can still fail.
- One deployable unit is cheaper to operate than many services, each with its own pipeline, telemetry and on-call duty.
What counts as evidence: measured costs of the shared deployment, such as release wait time, change collisions between teams, one workload saturating while the rest of the application is idle, or incidents in one area spreading to unrelated features. A preference for microservices or an expected future scale is not evidence by itself.
Operational readiness: every extracted service adds network failure modes for its callers. A workflow whose steps commit to independently owned state also becomes eventually consistent. An extraction that only serves reads need not create such a workflow. Without distributed tracing, on-call ownership and experience with asynchronous consistency, those costs outweigh the gain.
This is a conditional judgment, not a universal monolith-first rule. A monolith alongside a few extracted services can be a stable end state.
What evidence shows that an extracted service is independent rather than part of a distributed monolith?
An extracted service is independent when its owning team can change, deploy, roll back and operate it without a coordinated release elsewhere. A distributed monolith fails that test: its processes run separately, but its parts still change or fail together. Shared writable tables and paired deployments create that coupling directly. A mandatory synchronous call chain can create it too, when its contracts cannot evolve compatibly, its releases must be paired, or a dependency’s failure is not contained. Synchronous communication alone does not force lockstep change.
Evidence of independence:
- Unpaired release and operation: the service ships and rolls back on its own, and its team runs it with its own alerts and on-call duty.
- Exclusive data ownership: only the service writes its data; other components use its contract instead of reading or writing its tables.
- Versioned contract: consumers depend on a published API or event schema that can evolve compatibly, not on shared internals.
- Exercised failure behavior: dependency failures have been tested, and callers reject, queue or degrade as designed instead of letting the failure spread.
Traces and reconciliation show that synchronous calls and later messages can be followed across the boundary, and that diverging state gets detected. That is useful evidence, but it does not prove independence on its own: a fully traced service can still require lockstep releases.
Azure: Microservices architecture style · Azure: Event-driven architecture style
What is a modular monolith?
A modular monolith is a single deployable application whose code and data are divided into modules, each owning one business capability behind an explicit contract. It keeps a monolith’s single deployment and in-process calls while giving each module clear ownership.
- Code boundary: each module exposes a small public contract, such as commands, events, DTOs and interfaces. Other modules use that contract and do not reference the module’s internal domain or infrastructure types.
- Data boundary: each table has one owning module, often made visible with a separate schema and
DbContext. Other modules do not read or write those tables directly.- Runtime: modules are released together and normally run in the same process, so calls between them are in-process. With EF Core on a relational database, modules can also commit together in one local transaction when their
DbContextinstances share the sameDbConnectionandDbTransaction; using the same database alone is not enough.The boundaries hold only when they are enforced. Project references, architecture tests that fail the build, and database permissions make crossing a boundary harder than using the contract. Without that enforcement, direct table reads and internal references erode the modules into ordinary folders.
Kamil Grzybek: Modular Monolith: A Primer · EF Core: Using transactions
When is a modular monolith a better fit than microservices, and what signals justify extracting a module?
A modular monolith fits better when strong module boundaries are valuable but one deployment, in-process calls and local transactions still serve the system well. Extracting a module is justified when it repeatedly needs independent release, scaling or isolation, and that need outweighs the cost of a new network boundary.
Aspect Module inside a modular monolith Module extracted as a service Release Ships and rolls back with every other module, so unrelated changes can delay it Ships and rolls back on its own schedule Scaling Scales only by replicating the whole application Scales independently Calls In-process, with no network between caller and module. A call can still hang, or throw after producing effects Over the network, adding latency and transient failures. After a timeout or lost connection, the caller may not know whether the operation completed Transactions Can share one local transaction with another module when both DbContextinstances use the sameDbConnectionandDbTransaction; using the same database alone is not enoughCommits only to its own store and cannot join the caller’s transaction. When a local commit must also announce a change, an outbox records the message in that commit for later publication. Workflows spanning several services’ commits may also need idempotent consumers and sometimes a saga Operations Shares one pipeline and runtime with the rest of the application Needs its own pipeline, telemetry and on-call ownership The extraction signals are recurring costs of the shared unit. The module’s changes keep waiting behind unrelated releases, its load forces the whole application to scale, or its failures and resource use must be contained away from other features. Keeping the same contract interface does not make a remote call local. Deadlines, safe retries, cross-boundary observability and transaction boundaries must therefore be designed explicitly, with an outbox wherever a local commit must durably publish a message.
Azure: Microservices architecture style · Azure: Saga pattern · EF Core: Using transactions · Azure: Transactional outbox with Azure Cosmos DB · Azure: Retry pattern
When a modular-monolith module is extracted into a service, what must be redesigned even if its contract interface stays the same?
The interface can keep the same methods, but each call now crosses a network, which changes its latency, failure and transaction behavior. Local calls can already hang, or throw after producing an effect. The network adds another kind of uncertainty: after a timeout or lost connection, the caller may not know whether the service completed the operation. The remote call also can no longer join the caller’s local transaction.
What has to be redesigned:
- Deadlines and cancellation: remote requests get time limits, and callers need a defined response when the service is slow or unavailable, such as rejecting, queuing or degrading.
- Retries: a request is retried only when repeating it is safe. Either the operation is naturally idempotent, or it carries an idempotency key that stays the same across retries of one logical request, so the service recognizes a repeat after an attempt that may already have succeeded. Retries are capped so a partial outage does not become a retry storm.
- Observability: trace context crosses the boundary, and dependency latency, errors, saturation and retries are measured before traffic moves.
- Transactions: the shared local transaction no longer covers both sides, so each side commits only its own state. When a local commit must also publish a message that others depend on, an outbox records the message in the same commit for later publication, so a committed change cannot lose its message. Longer workflows may also need compensation or a saga.
- Data: the module’s data moves to its own store through an explicit migration with backfill, reconciliation and rollback.
Good module boundaries reduce how much code changes during extraction, but they cannot make a remote operation behave like a local method call.
Azure: Retry pattern · Azure: Transactional outbox with Azure Cosmos DB · Azure: Saga pattern
What evidence justifies extracting a service from a monolith?
Extraction is justified when a boundary is already able to stand alone and recurring, measured pressure requires it to. Without both conditions, extraction moves the coupling onto the network instead of removing it.
The boundary is ready:
- one team owns the capability and its changes;
- other code uses an explicit, stable contract instead of internal classes;
- the capability exclusively owns its data, and no other module reads or writes those tables directly.
There is recurring pressure for independence:
- its changes repeatedly wait for unrelated releases;
- its load needs to scale separately from the rest of the application;
- its failures or resource use need runtime isolation from other features.
When the pressure is missing, the network hop buys no autonomy that matters. The coupling that was visible in code becomes operational coupling, with added latency, partial failure and another deployment to run. When the boundary is not ready, shared tables or paired releases make the extracted service part of a distributed monolith.
How does service-oriented architecture (SOA) differ from microservices?
Both organize a system as services behind explicit contracts. They differ mainly in where change is controlled. Typical enterprise SOA centralizes integration and governance to connect existing systems that cannot change together. Microservices push ownership, data and deployment decisions into independently owned services.
Aspect Typical enterprise SOA Typical microservices Main pressure Make heterogeneous systems, such as SAP, mainframes and custom applications, interoperate through stable contracts Let teams own, change and deploy bounded business capabilities independently Service size Often coarse business services shared by several business units Usually smaller services, each aligned to one bounded capability Integration logic Often placed in a central ESB or orchestration layer that routes, transforms and mediates protocols Kept near service endpoints or in a workflow engine, with comparatively simple transport Data Shared enterprise stores are common Each service owns its data Governance Central contract and schema review gives controlled interoperability at the cost of slower local change Teams own their contracts within platform guardrails Deployment Services may share infrastructure and coordinate releases Independent deployment is a core goal These are tendencies rather than strict definitions. Many organizations use both styles: governed integration services around existing systems, and microservices for newer product capabilities.
Which part of SOA remains useful without SOAP, WSDL, or an ESB?
The durable idea is an explicit contract around a cohesive business capability: consumers depend on the published boundary, not the service’s internal code or direct writes to its data. SOAP/WSDL or an ESB were implementation/governance choices; HTTP, gRPC, or messaging can also carry the contract if compatibility is managed. SOA Manifesto
What makes a function "pure" and why does purity matter for testing?
A function is pure when its result depends only on its explicit arguments and evaluating it has no observable side effects. The same inputs always give the same output. The function does not read hidden state, such as the clock, a global variable or configuration. It also changes nothing outside its return value, such as shared state, a database or a log. A call can therefore be replaced by its result without changing program behavior. This property is called referential transparency.
For example,
CalculateTax(amount, rate) => amount * rateis pure. A version that reads_config.TaxRateand writes a log entry is impure. Its result depends on hidden state, and running it has an effect.Purity matters for testing because every dependency is an argument:
- Direct checks: a test passes input values and asserts on the returned value. No database, clock or global has to be arranged, faked or reset first.
- Repeatable results: the same inputs give the same result on every run and in any test order, so hidden state cannot make the test flaky.
- Effects at the edges: code that reads the clock or database and writes results stays in a thin surrounding layer. The decision logic is tested with plain values, and that layer is tested separately against its real boundary.
Purity and totality are separate properties. Totality is judged against a stated input domain: a function is total on that domain when every input in it produces a result. Suppose the stated domain is every value of the parameter types. A function that deterministically fails for some of those values, for example by throwing, is then partial on that domain. The same function can still be total on a narrower stated domain that excludes those values. Haskell’s semantics models such an evaluation error as bottom (⊥), like non-termination. That is a language-semantics model, not a general rule for C# exceptions. When invalid input is an expected case, it can be returned as a value, such as a
Result<T>. The function then returns a result for those inputs too. The outcome stays explicit, and a test can assert it like any other result..NET: Functional vs. imperative programming · Haskell 2010 Report §3.1: Errors · Lean reference: Functions
What is test-driven development, and how does its red-green-refactor cycle work?
Test-driven development (TDD) builds software in short cycles. A test for the next small behavior is written first, then just enough code to pass it, and then the design is cleaned up while all tests pass.
- Red: a test is written for the next behavior, usually chosen from a short list of examples still to cover. It must fail, and for the expected reason. The failure shows that the test can detect the missing behavior. A compile error because a type does not exist yet can be the first red state. It is fixed quickly so that the test fails on the behavior itself rather than on setup.
- Green: the simplest production change that makes the test pass is written, without speculative extras. Behavior grows only as far as the examples require.
- Refactor: with the suite green, production and test code are reviewed for naming, duplication and boundaries, and improved where that helps. The passing tests show that behavior did not change. This step is a deliberate check; code that needs no improvement is left alone.
The next example is then taken from the list and the cycle repeats. Each cycle stays small enough that the reason for a failure and the production change remain obvious. The design value depends on doing the refactor step; skipping it leaves passing tests without deliberate design improvement. TDD describes the order of work, so tests written after the implementation can be useful regression checks but are not TDD.
How can TDD improve design beyond increasing test coverage?
TDD improves design by making each test the first caller of the code it specifies. Because the test is written before the implementation, construction, inputs, results and failure behavior have to be decided from the caller’s side. Awkward construction or unclear results then show up while they are still cheap to change.
The rest of the loop adds two more design effects:
- Safe refactoring: red-green-refactor leaves a passing example for each completed behavior. Names, duplication and boundaries can then be improved while those examples show that behavior is unchanged.
- Decomposition: a list of examples, worked through in small steps, breaks a large behavior into observable increments.
These benefits are conditional. They depend on choosing informative examples and actually doing the refactor step. A poorly chosen test can drive an awkward interface as easily as a good one clarifies it. Difficult test setup is a signal to investigate, for example for hidden time, global state or too many responsibilities. It does not by itself prove that a class needs splitting.
When is TDD not worth the overhead?
TDD is poor value when its test-first loop yields neither useful design feedback nor lasting regression evidence. Typical cases:
- Disposable spike: the goal is to learn whether an approach works, and the code is discarded or rebuilt before production. Durable tests would protect code that will not ship, and the loop adds overhead to exploration. The exception lasts only while the spike stays disposable.
- Declarations the platform already guarantees: the language or framework may already guarantee a plain declaration’s behavior, as with an unmodified property accessor. A test for it mostly restates that guarantee. A declaration that adds custom validation, binding or serialization is no longer in this category.
- Visual rendering: when the useful feedback is how output looks, inspecting the rendered result gives that feedback more directly than a code-level red-green loop.
Skipping TDD does not remove the need to verify behavior that ships. That behavior still needs tests, written before or after the code, at the layer that can expose its realistic failure. Configuration, persistence and protocol wiring, for example, fit an integration-level test loop better than unit-level TDD. No universal coverage percentage or time-cost threshold decides the question.
What is a unit test, and what counts as the unit?
A unit test verifies one small behavior quickly and deterministically, without crossing slow or nondeterministic process boundaries such as a database, remote HTTP, the filesystem or uncontrolled time. The unit is defined by the behavior under test, not by code size. It can be one method, one class or a small group of collaborating objects.
- Kept real: ordinary in-memory collaborators can run as they are. They do not need to be mocked for the test to count as a unit test. Replacing them anyway is a design style, the mockist or London school, not part of the definition.
- Replaced: process boundaries the test does not own are represented by test doubles or plain values, such as a fixed clock instead of the system time.
- Result: fast, local evidence and a failure that points to a specific behavior.
A test that reaches a real database or filesystem has crossed a process boundary, whatever it is called. It becomes slower, and its failures include infrastructure state, so that behavior belongs in an integration test.
What is the difference between a stub and a mock?
A stub supplies canned answers so the code under test can run, and the test then checks the result. A mock is set up with expected calls, and the test checks that those interactions actually happened.
Aspect Stub Mock Role Feeds inputs to the code under test, such as a repository that returns a fixed list of orders. Observes outgoing calls, such as whether an email sender was asked to send a message exactly once. What the test asserts The outcome: a returned value or other observable state. The interaction: which calls were made, with which arguments and how often. Coupling to implementation Lower. Internal refactoring does not break the test while the outcome and the stubbed contract stay the same. Higher. Changing how collaborators are called can break the test even when the outcome is unchanged. Typical fit Dependencies whose answers drive the logic. Commands whose observable result is the call itself, such as publishing an event. The typical-fit row is a heuristic, not a type rule. The distinction lies in how the test uses the double, not in the tool that created it. In Moq, the same
Mock<T>acts as a stub when it only configuresSetup(...).Returns(...), and as a mock when the test callsVerify(...).Two related doubles complete the vocabulary. A spy records the calls it receives so the test can assert on them afterwards. A fake is a simplified working implementation, such as an in-memory repository backed by a
Dictionary.
When is a unit test the wrong testing layer?
A unit test is the wrong layer when the realistic failure comes from a boundary that the unit test replaces or never runs. A unit test substitutes doubles or plain values for databases, remote services and other process boundaries, so it cannot observe failures that only the real boundary produces. Using framework code does not by itself rule out a unit test. Deterministic in-memory framework behavior, such as
JsonSerializer.Serializeturning an object into a JSON string, can run inside one. The limit applies when the test replaces that behavior or never runs the real configuration, host or wire exchange around it.Typical cases:
- Provider SQL and persistence: a stubbed repository cannot show whether the database provider translates a query correctly or whether the query matches the real schema. An in-memory database provider is not a faithful substitute for relational behavior either.
- Serialization and protocols: field names, formats and wire compatibility are decided by the real serializer or HTTP stack.
- Dependency registration and configuration binding: a missing service registration or a misbound setting appears only when the real container and configuration are built and used.
- Rendering: markup and layout come from the UI framework, not from isolated logic.
These failures need a component or integration test that exercises the boundary with a real or appropriate implementation. The guiding rule is the lowest-cost layer that can still expose the realistic failure. Deterministic domain decisions belong in unit tests, while wiring and provider behavior need integration evidence.
.NET unit-testing best practices · .NET: Serialize objects to JSON with System.Text.Json
When is classic machine learning a better fit than an LLM?
Classic machine learning is the better fit when two conditions hold. First, the task is a defined prediction with representative labeled examples and a measurable objective. Second, measurement on the actual workload shows that classic ML meets that objective with a better balance of quality, latency, cost and maintenance than an LLM. The first condition makes classic ML a strong candidate. Only the second decides.
- Task and data fit: the task has a fixed output to predict, such as a class, a number or a ranking. Labeled examples represent the inputs seen in production, and a metric measures success directly. A model trained on those examples learns that mapping, and the metric checks its quality.
- Why fit alone does not decide: generative models can also perform classification and regression, and an LLM can be the stronger option for some text-classification tasks. A labeled prediction task can therefore have more than one viable approach.
- Measurement: both options are evaluated against the same simple, observable objective on the same data. Neither approach is inherently more accurate, faster, cheaper or easier to maintain.
Prompting can also try an LLM on a task while its requirements are still changing. Those results need the same measurement before either approach is chosen.
Google: ML problem framing · Google: Rules of Machine Learning
What is an embedding, and how does it support semantic retrieval?
An embedding is a vector of numbers that a model produces for a piece of text. The model places texts with related meaning close together, so the distance between two vectors measures how related the texts are. The vector’s length is set by the model and, for some models such as OpenAI’s
text-embedding-3family, by a requested output size.Semantic retrieval relies on that geometry:
- Indexing: the embedding model converts each document chunk into a vector, which is stored in a vector index.
- Query: the same model, with the same output size, converts the query into a vector in the same space.
- Ranking: a similarity measure, such as cosine similarity, ranks the stored vectors by closeness to the query vector. The nearest chunks become retrieval candidates.
Because matching depends on meaning rather than shared words, a query about throttling partner API traffic can find a document about rate limiting for partners. A high similarity score is only a relevance signal. It does not prove that a chunk answers the question, so retrieval quality still has to be evaluated.
Vector embeddings (OpenAI) · Vector search overview (Azure AI Search)
How do Matryoshka embeddings reduce vector storage, and how should a shorter vector be validated?
Matryoshka Representation Learning (MRL) trains an embedding so that its first dimensions already form a useful coarse representation, and later dimensions add finer detail. A prefix of the vector, such as its first 256 values, therefore still works as a lower-dimensional embedding. Storing only that prefix reduces vector storage.
- Storage: at the same data type, 256 values instead of 1536 is one-sixth of the raw vector payload. IDs, metadata and index structures do not shrink by the same factor, so the whole index shrinks less.
- Search cost: shorter vectors also reduce distance-computation work.
- Availability: shortening is model-specific. OpenAI’s
text-embedding-3models accept adimensionsparameter. A vector truncated manually is re-normalized before cosine or dot-product search.- Risk: fewer dimensions can lower retrieval quality on a particular corpus.
A shorter vector is validated by running the same labeled queries at each candidate dimension and comparing Recall@k and latency. The preferred dimension is the smallest one that still meets the recall and latency targets. A design that re-scores candidates with full-length vectors has to store them elsewhere or recompute them, which adds storage, API cost or latency.
Embeddings guide (OpenAI) · Matryoshka Representation Learning (arXiv)
What happens to retrieval when queries use embeddings from a new model but the corpus still has vectors from the old model?
Retrieval quality breaks. The query vector and the stored vectors come from different vector spaces, so the distances between them do not measure relatedness, and the ranking becomes effectively arbitrary.
Each embedding model learns its own coordinate system, so the same text receives unrelated vectors from two different models. If both models produce vectors of the same length, the search can still run without any error while returning poor results. That makes the mismatch easy to miss.
A model change therefore requires re-embedding the whole corpus with the new model. A safe migration builds and validates the new index first and only then switches query traffic, for example by moving a collection alias. Embedding-cache keys include the model name and version, so cached vectors from the old model are not reused with the new one.
What is LoRA, and why is it cheaper than full fine-tuning?
LoRA (Low-Rank Adaptation) fine-tunes a pretrained model by freezing its weights and training only a small low-rank update for selected layers. Because the trainable part is a tiny fraction of the model, training needs gradients and optimizer state only for that update, and the result is a small adapter instead of a full model copy.
- A chosen weight matrix
W0stays frozen.- Its change is represented as the product of two small matrices,
B·A. Their shared inner dimension, the rankr, is much smaller than the layer’s dimensions.- Only
AandBare trained.- The adapted layer computes
W0·x + B·A·x.The small trainable set lowers training memory compared with updating every weight. Adapters for different tasks can share one base model and be swapped. An adapter can also be merged into
W0, so serving needs no extra computation. Whether a LoRA adapter matches full fine-tuning on a particular task depends on its configuration, such as rank and target layers, and held-out evaluation establishes it.Sources: Hu et al.: LoRA · Hugging Face PEFT: LoRA
Why can LoRA still cause capability regressions if the base weights are frozen?
Freezing the base weights fixes what is stored and updated during training, not what the deployed model computes. LoRA adds a trainable low-rank update to selected layers. While the adapter is active, each adapted layer computes the frozen weight’s output plus the adapter’s contribution:
W0·x + B·A·x.That added term applies to every input passing through those layers, not only to inputs that resemble the training data. Training optimizes it for the target examples alone, so it can also shift behavior the data never covered, such as instruction following, safety behavior or general reasoning. Frozen base weights therefore do not guarantee unchanged outputs. The full base-plus-adapter model needs evaluation on both the target task and broad held-out sets.
Freezing does make rollback easier. A separate adapter can be disabled, which returns the model to the base behavior. Merging folds the update into the weights, so rollback after a merge depends on a supported unmerge or an unmodified copy of the base checkpoint.
Sources: Hu et al.: LoRA · Hugging Face PEFT: LoRA · More Than Catastrophic Forgetting
Why is adjusting
temperatureandtop_pat the same time discouraged?Both settings control sampling randomness by reshaping the same next-token distribution. When both change at once, a change in output cannot be attributed to either setting. Tuning is therefore clearer when one setting moves while the other stays at its default.
temperaturerescales the logits before sampling. Lower values concentrate probability on the most likely tokens; higher values flatten the distribution and admit more unlikely tokens.top_p(nucleus sampling) samples only from the smallest set of tokens whose cumulative probability reaches the threshold, cutting off the low-probability tail.Both settings change how often low-probability tokens appear, so their effects overlap rather than controlling separate properties. OpenAI’s API reference currently recommends altering one or the other, not both. This is tuning guidance, not a hard limit. Advice differs across providers, and some models fix or reject these parameters, so the deployed model’s documentation and task evaluation decide the final settings.
Sources: OpenAI API reference: Create chat completion · Hugging Face Transformers: Generation
Why can a response with valid citation markers still contain unsupported claims?
A citation marker only links a claim to a source location. It does not show that the cited passage supports the claim. Markers on some sentences also say nothing about sentences that have none.
- Wrong support: the model generates the citation as well as the claim, so it can attach a real, on-topic passage to a statement that the passage does not entail. For example, a passage describing a refund policy may not contain the refund limit that the answer attributes to it.
- Missing support: a response can mix cited sentences with uncited claims drawn from the model’s own memory.
Citation validity and claim support are therefore separate checks. Support is checked after generation by splitting the answer into claims and testing each claim against its cited passage, through human review or an automated verifier matched to the risk. Natural Language Inference classifiers and fact-checking models such as MiniCheck are automated options. They also make errors, so their decisions need validation against labeled examples.
Sources: Anthropic: Citations · Tang et al.: MiniCheck
What does an explicit execution graph define in an agent workflow?
An explicit execution graph defines an agent workflow’s control flow: the units of work, the transitions allowed between them and the state carried from step to step.
- Nodes do the work. A node can be ordinary code, a tool call, one model call, a complete agent loop or a nested graph.
- Edges route execution. A fixed edge always leads to the same successor, a conditional edge chooses the successor from an outcome or the current state, and a terminal edge ends the run.
- State records progress. Nodes read defined state and return updates, and the runtime’s merge rules decide how concurrent updates combine.
A loop can run inside one node or appear as a cycle across nodes. The graph models control flow, not relationships in data as a knowledge graph does. The framework decides when nodes run, when their updates become visible and where checkpoints are taken.
Sources: Microsoft Agent Framework: Workflows · Microsoft Agent Framework: Checkpoints
Why does an explicit graph not make an agentic system deterministic?
An explicit graph fixes which paths are legal, not what happens along them. Fixed edges and deterministic guards make parts of the control flow repeatable, but other sources of variation remain:
- Model calls inside nodes can return different outputs, and a model-driven routing decision can choose a different legal edge.
- External services and mutable inputs can return different data on different runs.
- Concurrent work can interleave differently from run to run.
The graph constrains the application’s routing and makes it easier to inspect, but it does not guarantee repeatable results. Reproducing or debugging a run therefore depends on recording its inputs, state, node outputs and routing decisions.
Why does checkpointing require idempotent node design?
Resuming from a checkpoint can run a node’s work again, so any external side effect in that node must be safe to repeat. In Microsoft Agent Framework, checkpoints are taken when a superstep completes. Work done after the last saved checkpoint is not part of any saved state, so resuming can repeat it.
- The checkpoint after superstep N is saved.
- A node in superstep N+1 sends an email.
- The process stops before the checkpoint after superstep N+1 is saved.
- Resuming from the superstep N checkpoint runs the node again, and a second email is sent.
An idempotent operation prevents the duplicate. If the email service accepts an idempotency key, the node sends a key that identifies this one logical operation. The key stays the same on every retry or replay, so the service recognizes the repeat and does not send again. The key is built from checkpointed values, such as the run, the node and the loop iteration, rather than generated fresh inside the node, because a fresh value would differ on replay. A later send that is meant to happen, such as a reminder in a later iteration, gets a different key.
When the side effect is itself a database write, atomic deduplication also works. A unique constraint on the operation’s key makes a replayed insert fail instead of creating a second record. A separate check-then-write does not work, because the process can stop after the effect but before recording that it happened. Checkpointing provides resumable state, not exactly-once side effects.
Sources: Microsoft Agent Framework: Checkpoints · Amazon Builders’ Library: Making retries safe with idempotent APIs
How do encoder-only, encoder-decoder, and decoder-only transformers differ?
The three families differ in which transformer stacks they use, and therefore in how attention sees the input and what the model produces. Encoder-only models build contextual representations of an input. Decoder-only models generate a continuation one token at a time. Encoder-decoder models read a source sequence and generate a separate target sequence.
Aspect Encoder-only (BERT) Encoder-decoder (T5) Decoder-only (GPT-style) Attention Bidirectional: every token can attend to the whole input The encoder reads the source bidirectionally; the decoder attends causally to its own earlier tokens and to the encoder’s output Causal: each position attends only to earlier positions Pretraining objective Predict masked tokens from the context on both sides The encoder reads input with spans removed, and the decoder generates the missing spans Predict the next token Output One contextual vector per token, or a pooled vector, passed to a task head A generated target sequence A generated continuation of the prompt Typical use Classification, extraction, reranking, embeddings Translation, summarization and other text-to-text tasks Chat, code, completion, tool-call generation An encoder-only model has no autoregressive decoder, so it is not used for open-ended generation. The causal mask is what lets one decoder-only stack train on next-token prediction and then continue a prompt token by token. The typical uses describe common fits, not hard limits.
Sources: Attention Is All You Need · BERT · T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer · Language Models are Few-Shot Learners
How should prompting, RAG, and fine-tuning be chosen for an LLM system?
The choice follows the gap that evaluation reveals. A behavior gap is first addressed with clearer instructions and examples. A knowledge gap is closed by putting the facts into context. Small, stable facts can be written directly into the prompt. Large, private or changing facts are retrieved per request with RAG. Fine-tuning addresses behavior that stays inconsistent after prompt work. The techniques solve different problems and can be combined.
Technique What it changes Fits when Main limit Prompting The instructions, examples and any fixed reference text sent with each request The model can do the task but needs a clearer specification, output format or examples, or the facts it needs are few and stable enough to write into the prompt Instructions alone add no knowledge. Facts written into the prompt are fixed in advance and add tokens to every call, so this approach does not scale to large or changing knowledge RAG The evidence selected and placed in the context at request time Answers depend on current, private or citable information that is too large or changeable to include in every prompt Answer quality is limited by retrieval quality, and retrieved text does not reliably teach a new format or style Fine-tuning The model’s weights, through training on examples Format, style or task behavior remains inconsistent after prompt work It needs training data and evaluation, and facts stored in weights are harder to update than retrieved documents Prompting is the usual starting point because it is the cheapest to iterate, but this is a diagnostic default, not a mandatory sequence. A fine-tuned model can still use RAG: training shapes behavior while retrieval supplies changing facts. Each change is compared on the same representative evaluation set, so an improvement can be attributed to that change.
Source: OpenAI: Optimizing LLM accuracy
How does a sparse mixture-of-experts model work, and what does sparse activation actually save?
A sparse mixture-of-experts (MoE) model replaces some dense feed-forward layers with several expert networks and a learned router. Each token is processed by only a few selected experts. Compared with evaluating every expert on every token, this saves feed-forward arithmetic, so total parameter capacity can grow while per-token computation stays roughly constant. It does not shrink the parameter footprint: every expert still has to be stored and placed in device memory, and the rest of the transformer still runs for every token.
Routing a token through an MoE layer:
- The router scores every expert for the token’s hidden state.
- The top-k experts are selected.
- The token is dispatched to those experts, possibly on other devices.
- Their outputs are combined using the router’s weights.
Measure What it reflects Total parameters Checkpoint size and the device memory needed to hold every expert Active parameters per token Part of the arithmetic performed for each token Measured latency and throughput Arithmetic plus router work, cross-device dispatch, load imbalance, batching and kernels If many tokens choose the same expert, its device becomes a bottleneck while others sit idle. Capacity limits and load-balancing objectives counter this. Depending on the implementation, tokens beyond an expert’s capacity are dropped or rerouted.
The saving is relative to evaluating all experts. It is not a guaranteed advantage over every alternative. A dense model with a similar number of active parameters can be faster than an MoE when expert traffic is communication-bound, because the MoE also pays for routing, dispatch and imbalance. Parameter counts alone cannot predict either outcome. Only measurement on the intended serving workload shows the real cost.
MoE routing happens inside one model and is separate from application-level routing between different models.
Source: Switch Transformers
How should candidate LLMs be compared for a production workload?
Candidate LLMs are compared empirically. Every candidate runs on the same representative, versioned workload and is judged against the same gates for task quality, safety, latency and cost. A common decision rule then selects the least expensive candidate that clears every required gate.
What the comparison measures:
- Task quality: correctness, groundedness, tool-call success and failure slices, measured on a labeled set drawn from the product’s own traffic and rubric.
- Safety and policy: refusal behavior, data handling and provider terms.
- Latency: p50 and p95 at the expected prompt size, output length and concurrency.
- Cost per successful task: input, output and tool usage, plus retries and escalations. A model with a low price per token can still cost more per completed task if it often needs a retry or a human.
- Capabilities and operations: required context length, structured outputs, tools, rate limits and version pinning.
Public leaderboards help build a shortlist, but they measure a different distribution and rubric. The evidence also expires. A change in traffic, prompts or prices invalidates earlier results, as does a new model version behind a provider alias, so the comparison is rerun when any of these occur.
Sources: OpenAI evals · OpenAI model optimization
How do classifier routing and cascading differ when choosing an LLM for each request, and how can they be combined?
A classifier router predicts which model should handle a request before anything is generated. A cascade generates with a lower-cost model first and escalates to a stronger model only when an observable check on that output fails.
Aspect Classifier routing Cascade When the decision is made Before generation, from request features such as task type or predicted difficulty. After a first generation, from a gate on the output such as schema validation, a groundedness check, calibrated confidence or an explicit unsupported result. Generations per request One, so no request pays for a discarded attempt. One when the gate passes; at least two when the request escalates. Typical failure A false “easy” prediction sends a hard request to a weaker model. Quality drops silently unless something downstream checks the answer. Escalated requests pay for duplicate generation and extra latency, which can dominate p95. If many requests escalate, the cascade can cost more than always using the stronger model. What it depends on Predictable difficulty signals, and a classifier evaluated on per-route recall for hard and safety-sensitive requests. A cheap, reliable gate. A weak gate escalates too often or lets bad answers through. Combining them: the classifier chooses the starting model, and a cascade gate checks that model’s output. Requests predicted to be hard go straight to the stronger model. Requests predicted to be easy start on the cheaper model with the gate behind it. The gate then catches some misroutes that classifier routing alone would serve unnoticed.
The combined policy is judged on end-to-end task success, cost per successful task and tail latency, not on classifier accuracy alone. Managed routers such as Microsoft Foundry Model Router package per-request model selection as a product feature, and they still need the same evaluation on the target workload.
Sources: FrugalGPT (Chen et al.) · Microsoft Foundry Model Router · OpenAI evals
When does a workflow fit better than an autonomous agent?
A workflow fits better when the steps and branches can be defined in advance. Code then controls the path, and the model handles individual steps inside it. An autonomous agent is justified only when the next action must depend on intermediate results that cannot be anticipated, and that flexibility is worth its extra cost and risk.
Aspect Workflow Autonomous agent Who chooses the next step Application code follows predefined paths. The model works inside individual steps or picks one of several predefined branches. The model decides its next action and tool use from the results so far. Predictability The possible paths are known before the run, so each one can be tested and reproduced. The path emerges at runtime, so two runs of the same task can take different routes. Cost and latency Easier to predict, because the possible calls follow from the defined paths. Depend on how many model and tool calls the agent decides to make. Failure exposure Failures occur at known steps, where code can add checks before the next step runs. An early mistake can become the premise for later steps and compound. The usual default is the simplest design that meets the need: a single well-prompted call first, a workflow when one call is not enough, and an agent only when no fixed path can express the task.
How can errors compound during an autonomous agent run, and what controls contain them?
Errors compound because each step builds on what earlier steps produced. An agent reasons over earlier observations, acts, and feeds the new result into its next decision. A wrong assumption or misread tool result therefore does not stay one bad output: it becomes the premise for later actions. For example, an agent that looks up the wrong order can then refund, email and log against that order, with every step consistent with the original mistake.
Several design controls contain this risk:
- Step, cost or time limits set for the task end a run that keeps acting without converging, so drift has a bounded cost.
- Checks against expected outcomes, such as tests or validation of tool results, can catch bad progress before more steps depend on it.
- Recovery or escalation paths define what happens when the run cannot proceed safely, such as stopping with a report or asking a human for input.
- Traces of decisions, tool calls and results show where the run diverged, which makes the failure diagnosable.
These controls reduce risk; they do not guarantee that the agent recovers or reaches a correct result.
ReAct: Synergizing Reasoning and Acting in Language Models · Building Effective AI Agents
What makes a task a good fit for an autonomous agent?
An agent is useful when the path cannot be fully specified in advance and intermediate results must influence the next action. The outcome still needs criteria that can eventually determine success or failure; delayed feedback can work when execution has a timeout, checkpoints, and a recovery plan. If success stays vague, the system cannot tell whether another action helped. Building Effective AI Agents
How do prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer differ in who controls the next step?
They differ in how much of the control flow code fixes in advance. In prompt chaining, routing and parallelization, application code defines every possible path, and a model at most picks one predefined branch. In orchestrator-workers, a model decides the subtasks at runtime. In evaluator-optimizer, a model’s evaluation decides whether the loop repeats, within a cap set in code.
Pattern Who decides the next step What code fixes in advance Prompt chaining Code. Each step always follows the previous one, and programmatic gates can stop a bad intermediate result. The whole sequence of steps. Routing A classifier, often an LLM, reads the input and picks one branch. The set of branches and how each one is handled. Parallelization Code. It starts the calls together and aggregates them, either splitting independent sections or voting over repeated runs. The subtasks and how their results are combined. Orchestrator-workers The orchestrator model decides how to split the task, what to delegate and how to synthesize the results. Only the outer frame; worker count and assignments appear at runtime. Evaluator-optimizer An evaluator model decides whether a draft passes explicit criteria or returns to the generator with feedback. The acceptance criteria and an iteration cap that ends the loop. Each additional model-owned decision tends to make cost less predictable and failures harder to reproduce, so the simplest pattern that fits is usually the easiest to test.
Why is the orchestrator-workers pattern harder to operate than ordinary parallelization, even though both fan out work to several calls?
Orchestrator-workers is harder to operate because a model, not code, decides the structure of the fan-out. In ordinary parallelization, code defines the subtasks and the aggregation in advance. In orchestrator-workers, a model chooses the subtasks for each input at runtime and also synthesizes the results, so the set of calls is not known before the run.
- Ordinary parallelization: application code starts a predefined set of independent calls, such as one per code-review concern, and aggregates their outputs programmatically. The split and aggregation rules can be read, reviewed and tested before any run.
- Orchestrator-workers: an orchestrator model reads the input, decides how to split it and what to delegate, then synthesizes the workers’ results. The worker count and assignments exist only at runtime.
Fixed rules make the structure inspectable, but they do not make outcomes repeatable. In both patterns, the workers are model calls whose outputs can differ between runs, and how well a split suits the task depends on the input. Orchestrator-workers adds model-chosen structure on top of that variability:
- Work and cost vary: the number of workers, and therefore the calls and tokens spent, can change between inputs and even between runs on the same input.
- The split itself can fail per run: any fan-out can miss part of the task, duplicate work or combine results poorly. With a fixed fan-out, the split and aggregation rules can be checked before deployment. With orchestrator-workers, a flawed decomposition or synthesis exists only in a particular run’s decisions. It cannot be reviewed in advance and can appear for some inputs or runs but not others. It is therefore harder to reproduce and to catch with a few fixed test cases.
These are risks, not guaranteed outcomes. A trace of each run’s decomposition and synthesis shows where a particular run went wrong.
Why can one base model produce very different results on the same agent benchmark?
Because an agent benchmark scores a whole system, not the model alone. The result reflects the model together with its scaffold: prompts and policies, tool descriptions, planning and retry logic, context management, and the loop that runs it. Changing the scaffold can move the score even when the base model stays the same.
Run conditions also matter. An agent can solve a task on one attempt and fail it on the next. τ-bench reports
pass^k, the chance that allktrials of a task succeed, which exposes inconsistency that a single-attempt score hides.A fair model comparison therefore holds the task set, scaffold and run conditions constant and varies only the model. Comparing scaffolds is a separate experiment with the model held fixed. A leaderboard score describes one benchmark setup, not intrinsic model quality.
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Why should tool selection and argument accuracy be measured separately?
Because they score different decisions that fail for different reasons. Tool selection asks whether the agent called the right tool for the current step, including recognizing when no call is needed. Argument accuracy asks whether a correctly chosen tool received the right values.
A combined score hides which decision failed. For example, 95% selection accuracy and 70% argument accuracy average to 82.5%, and that single number no longer shows that arguments are the weak point. Separate scores point an investigation in a direction:
- Selection failures often suggest unclear tool descriptions or a routing problem.
- Argument failures often suggest weak grounding in the conversation or a loosely constrained schema. A call can name the right tool in schema-valid form and still send order
4851when the conversation identifies order4815.These patterns are leads to investigate, not proof of the root cause.
How argument accuracy is scored also depends on the field. Exact matching fits identifiers, enums and booleans with one defined value. Free-text fields need normalization or a semantic check, which is valid only where the tool contract accepts equivalent wording.
When is reference matching a better way to score an agent's trajectory than an LLM judge?
Reference matching is the better scorer when the acceptable tool-call paths are few enough to write down and the rule for comparing them is clear. It is then cheap, deterministic and repeatable. An LLM judge fits better when many different paths can be correct, because it scores the trace against a rubric instead of a list of paths. The cost is an extra model call and less deterministic scores.
Aspect Reference matching LLM judge over the trace What it needs One or more known-good tool-call paths and a comparison mode. A rubric describing good behavior, such as plan quality, tool choice, wasted work and recovery. Alternative valid paths Accepted only if listed or tolerated by the mode; an unanticipated correct path fails. Can be accepted without being listed, because the rubric describes qualities rather than one path. Cost and repeatability No model call; the same trace always gets the same score. A model call per scored trajectory; scores for the same trace can vary between runs. The comparison mode sets how much variation counts as acceptable:
- Strict: same calls in the same order; fits a task with one valid procedure.
- Unordered: same calls in any order; fits independent reads.
- Subset: every agent call must appear in the reference, so unlisted tools fail. This can forbid writes in a read-only task.
- Superset: every reference call must appear in the trace, and extra calls are allowed. This enforces required steps, but a wasteful trace that contains them still passes.
Several valid paths do not automatically require a judge. A richer set of references, or a hybrid rule such as a subset gate plus a judge for path quality, may be enough.
Why can adding more context make answers worse instead of better?
Because a model does not use every part of a long input equally well, so extra material can bury or dilute the evidence that the answer depends on.
- Position effects: in the long-context tasks evaluated in Lost in the Middle, some models used relevant information near the beginning or end of the input more reliably than the same information placed in the middle. The size of the effect depends on the model and task.
- Dilution: irrelevant or redundant text competes with the useful evidence, so adding it can lower answer quality instead of adding information.
- Cost: extra input tokens also add cost and can add latency, even when they do not improve the answer.
Added context is not always harmful. The aim is complete, relevant material placed deliberately, with answer quality and input cost measured on the actual workload as the context grows.
How can a long-running agent keep useful context within a finite window?
By treating the context window as a working set. Each call contains the instructions, evidence and state needed for the next decision. Everything else stays outside the prompt but remains retrievable.
The prompt, conversation history, tool results and space reserved for the response all draw from one finite token budget, so they are managed together:
- Selection and ordering: only material relevant to the current step enters the window, and it is placed deliberately rather than replayed in full.
- Trimmed tool results: tool output keeps the fields the next decision uses, and stale results that no longer matter are cleared.
- Compaction: older turns are replaced by a short record of decisions, constraints and open work before history crowds out the task.
- Offloading: bulky state, such as notes, files or full documents, lives outside the prompt and is reloaded by reference when needed.
Compaction trades space for possible detail loss: a summary can drop a subtle fact that later matters. Where exact recovery matters, the source material is kept outside the window instead of being trusted to the summary.
How should chunking be chosen for a RAG corpus?
Chunking should be chosen from the corpus’s structure, the queries it must answer and the failures that evaluation exposes. No splitter or chunk size is best for every corpus.
Chunk size sets the basic trade-off:
- Larger chunks keep related statements together, but they can dilute the match and spend context on irrelevant text.
- Smaller chunks can match a query more precisely, but they can also separate a rule from the condition or exception that changes it.
Each chunk must also fit the embedding model’s input limit. Overlap between neighboring chunks lowers the chance that a boundary cuts a statement apart, at the cost of some duplicated text.
A simple rule-based splitter, such as fixed-size or recursive splitting, provides a cheap baseline. A more specialized strategy earns its extra ingestion and maintenance cost only when evaluation traces failures to the problem it addresses:
Observed failure Strategy that targets it Why it helps Tables, code blocks or clause–exception pairs are split apart Structure-aware splitting A format parser keeps those layout units intact. The right passage is found, but nearby constraints are missing from the answer Parent-child retrieval Small chunks are searched, and their larger surrounding section is passed to generation. Long prose without reliable headings mixes several topics in one chunk Semantic splitting Boundaries are placed where adjacent text stops being similar. A change is kept only when it improves retrieval and end-to-end answers on that corpus.
Sources:
How does parent-child chunking change the context returned for a matching passage?
Parent-child chunking searches small child chunks but gives the generator the larger parent section that contains the match. Searching small children can keep matches precise, while generation receives surrounding text that the child alone may lack.
The flow:
- Ingestion splits each parent section into small child chunks and stores the parent’s identifier with every child.
- A query is matched against the child chunks.
- The application maps each matched child to its parent through that identifier. This is a lookup, not another similarity search, and a parent shared by several matched children is included once.
- The parent text, rather than the child alone, is assembled into the generator’s context.
The parent can restore a nearby prerequisite, condition or exception that the matching child omitted, such as an exception clause that follows a retention rule. It also adds tokens. A parent that is too broad brings back the irrelevant text that small children were meant to avoid. Expansion therefore helps only when evaluation on the corpus shows better retrieval and end-to-end answers.
In Azure AI Search, index projections can index child chunks with a field that links each one to its parent document. Expanding results to parent text is logic the application implements.
Sources:
When should a team move from recursive to structure-aware chunking?
The move is justified when evaluation repeatedly traces retrieval or answer failures to document structure that the recursive splitter breaks, and the measured gain outweighs the cost of maintaining parsers.
Aspect Recursive splitting Structure-aware splitting How boundaries are found An ordered list of separator strings is tried, such as blank lines, then newlines, then spaces. Smaller separators are used only when a piece is still too large. A parser for the source format identifies headings, tables, code blocks and list items. What it keeps together Paragraphs and sentences, as far as the separators match the format. Logical units such as a whole table or a clause with its exception, plus metadata such as the heading path. Typical failure Boundaries fall wherever a separator occurs. Table rows can be separated from their header, and code blocks or clause–exception pairs can be divided. A parser can misparse silently when a source changes format, and very large units still need a size limit. Cost Low: rule-based, with separators configurable per format. Higher: one parser per source format, plus ongoing maintenance. A generic recursive splitter, such as LangChain’s
RecursiveCharacterTextSplitter, works on separator strings and does not parse the document format. Some failures can be fixed within recursive splitting by adding format-specific separators, or by extracting tables and code blocks as whole units first. Structure-aware splitting earns its cost when those adjustments are not enough.Sources:
Why is semantic chunking not always superior to simpler rule-based approaches?
Semantic chunking places boundaries from embedding scores and a tuned breakpoint rule. That adds ingestion cost and a setting whose behavior can shift with the model and corpus. It adds little when the document’s structure already marks the topic boundaries.
How a semantic splitter places boundaries:
- The text is split into sentences or small groups of sentences.
- Each group is embedded, and the distance between neighboring groups is computed.
- A boundary is placed where a distance exceeds a breakpoint, such as a percentile or a number of standard deviations computed over that text’s distances.
Why that is not automatically better:
- Cost: every span needs an embedding call during ingestion, which rule-based splitters avoid.
- Instability: the distribution of distances depends on the embedding model and the corpus. A breakpoint tuned in one setting can, in another, fragment text into tiny chunks or leave several topics in one chunk. It must therefore be re-evaluated after the model or corpus changes.
- Redundancy: when reliable headings or markup already separate topics, a structure-aware or recursive splitter finds the same boundaries more cheaply and predictably.
Semantic splitting earns its cost for long prose that changes topic without structural markers, such as transcripts or email threads, when evaluation shows better retrieval.
Sources:
How do offline evaluation and production monitoring work together for a RAG system?
Offline evaluation decides whether a candidate version is good enough to release. Production monitoring checks whether the released version keeps working on real traffic. Confirmed production failures are added to the offline dataset, so later releases are tested against them.
Aspect Offline evaluation Production monitoring Data A controlled, versioned dataset with relevance labels or reference answers Live requests, whose query mix and source documents change over time Timing Before release, comparing a candidate with the current baseline After release, continuously Signals Retrieval metrics against labels and answer correctness against references Per-stage traces, latency, errors and cost, cheap checks on every request, and judge scores on a sample Blind spot Query patterns and data changes that the dataset does not contain Usually no reference answer, so quality scores are only as valid as the sampling and the evaluator The feedback loop:
- A release passes offline gates against the baseline.
- Monitoring flags an anomaly, such as sampled faithfulness falling for one segment or empty results rising.
- Traces are triaged to confirm real failures and locate the failing stage.
- Confirmed cases, with verified labels, join the offline dataset.
- The fix is evaluated offline against the expanded dataset before the next release.
Sampled quality scores need representative or stratified sampling, and a judge whose agreement with human labels is checked. Without both, monitoring can look healthy while quality falls.
Sources:
What does query translation change in a RAG retrieval pipeline, and what can it not fix?
Query translation changes only the queries sent to retrieval. A model turns the user’s request into one or more retrieval inputs, such as paraphrases, a broader step-back question, sub-questions or a hypothetical answer. Each input is then searched against the existing index. Translation cannot fix problems in the corpus or in the index itself.
- What it can help: vocabulary mismatch, where the user and the corpus describe the same idea in different words; requests whose parts need separate searches; and questions that also need broader background. For example, “Can partners burst above limits now?” and a document titled “Partner Tier Burst Allowance” describe the same policy, and a rewrite closer to the document’s wording can reach it.
- What it cannot fix: evidence missing from the corpus, chunks that split the needed text, a weak embedding model or a broken index. Every variant still searches the same chunks and vectors.
The work after search depends on the technique. Lists retrieved for several phrasings of one question are usually deduplicated and fused before reranking or generation. Decomposition can instead answer each sub-question from its own evidence and then synthesize the partial answers into one response. In every case, the benefit is measured against direct retrieval on the same queries.
Sources:
How can query translation change retrieval quality, and how should that change be evaluated?
Translation can raise recall by searching wordings closer to the corpus’s vocabulary. It can lower precision when a rewrite drifts from the request’s entities or constraints. Its effect is measured by comparing translated retrieval with direct retrieval on the same labeled queries.
- Gain: a rewrite such as “partner tier throttling policy” can match documents that use words the user did not, so relevant evidence missed by the original wording enters the results.
- Loss: a rewrite can stay on topic yet change the request. When “
HttpClientconnection timeout” becomes “network timeout troubleshooting”, operating-system material can crowd out .NET-specific evidence. Fusing the result lists does not remove drift, because a document ranked highly by one bad variant can still pass the cutoff.How the change is evaluated:
- Direct retrieval is the control, run on the same labeled queries.
- Results are split by query class, such as vague questions and identifier-heavy ones, because a technique can help one class and hurt another.
- Recall@k and Precision@k are measured at the cutoffs the pipeline actually uses. A rise in Recall@20 with a fall in Precision@5 means more relevant candidates appear within 20 results while the top five carry more noise. By itself, that result does not show what survives reranking or reaches the generator.
Two common controls limit drift: the original query stays among the searched variants, and rewrites that alter entities, identifiers, versions or other material constraints are rejected.
Sources:
When should a RAG system decompose a question instead of generating multiple query phrasings?
Decomposition fits a request whose parts need different evidence. Multiple phrasings fit a single, clear information need whose wording may not match the corpus.
Aspect Decomposition Multi-query Fits when Parts of the request need separate evidence, as in comparing two systems or assembling a timeline One intent is clear, but the corpus may describe it in other words What is generated Different sub-questions, each retrieving its own evidence Several phrasings of the same question Main failure Sub-questions drop constraints from the original request, so retrieval returns generic evidence that synthesis cannot make specific again Paraphrases drift from the intent and pull in related but wrong material Execution Independent sub-questions can run concurrently. Multi-hop sub-questions depend on earlier answers and run in sequence. Variants can run concurrently, and their result lists are merged and deduplicated Added work Generating sub-questions, one retrieval per sub-question and a synthesis step that combines the partial answers Generating variants and one retrieval per variant Example: “How does
Taskcompare toValueTaskfor high-throughput API endpoints?” needs evidence onTaskallocation,ValueTaskconstraints and throughput behavior, which a single passage is unlikely to contain. Decomposition fits, provided each sub-question keeps the high-throughput constraint and the synthesis step receives the original request.Sources:
What must a RAG retrieval-cache key include to keep cache hits correct?
The key must include every input that can change the ranked candidate list, so that changing any of them produces a different key. A retrieval cache stores document IDs and scores. If a result-affecting input is missing from the key, a hit returns a list computed under different conditions.
Key field Failure if it is missing Processed query, plus the version of the canonicalization or translation that produced it After query processing changes, hits can return lists produced by the old processing. Embedding model and its configuration After an upgrade, lists ranked in the old embedding space are reused. Index or corpus version Added, changed or deleted documents stay invisible to cached queries. Tenant and authorization-context hash One caller receives candidates computed for another caller’s permissions, which is a data leak, not just staleness. Filters, top-k and retriever or reranker configuration A request with different settings receives a list built for other settings. The query part of the key can be processed text or embedding bytes. Text is inspectable and stays stable while its canonicalization is versioned. Embedding bytes are deterministic only while the encoder, model and configuration are pinned. Either works when every other result-changing input is also in the key.
For protected content, a complete key is necessary but not sufficient. A read still checks the caller’s current access, because permissions can change after an entry is stored.
Sources:
How do caches help a RAG pipeline, and what makes a cache hit safe?
Each cache layer skips a different expensive step: an embedding cache skips the embedding call, a retrieval cache skips search and reranking, and a response cache skips generation. A hit is safe only when three conditions hold: the key or a validation check covers every input that could change that stage’s result, the caller is currently allowed to see the value, and the entry is fresh enough for its sources.
Layer Stores Work skipped on a hit Key must cover Main risk Embedding The vector for a processed text The embedding call, at ingestion or query time Text hash, embedding model, dimensions and preprocessing version Vectors from an old model mixed with new ones, which makes similarity scores meaningless Retrieval Ranked document IDs and scores Vector search and reranking Processed query, index version, filters, top-k, retriever settings, tenant and authorization context Stale lists after an index update, or results computed for another tenant Response The final answer Generation The retrieval inputs plus prompt template, selected evidence, generation model and settings, and any conversation state that changes meaning Answers grounded in outdated evidence, or served to a caller who may not see that evidence Freshness follows the sources. A time-to-live bounds staleness, and known source or permission changes invalidate entries before it expires. Fast-changing evidence can justify caching embeddings and retrieval results while not caching final answers.
Sources:
What kind of RAG question can justify adding a knowledge graph to vector retrieval?
A knowledge graph can be justified when the answer depends on explicit relationships between entities, or on synthesis across much of a corpus, rather than on a few passages that resemble the question. A question such as What are the main themes across these documents? is an example: no small set of similar chunks contains its answer.
- Vector retrieval returns the top-k chunks closest to the query. That suits direct fact lookup, where one or a few passages hold the answer.
- GraphRAG uses a language model to extract entities and relationships into a graph, groups closely related entities into communities and pre-generates a summary of each community. Corpus-wide questions can then be answered from those summaries instead of from isolated chunks. Questions about dependencies or impact can follow explicit edges between entities.
The graph adds model-driven extraction, indexing and maintenance cost, and extraction or linking errors become wrong edges. Microsoft Research evaluated GraphRAG on corpus-wide, query-focused summarization, so its benefit for other question types has to be measured. The decision therefore rests on a comparison with a vector-retrieval baseline on representative questions.
From Local to Global: A Graph RAG Approach to Query-Focused Summarization (Microsoft Research)
What is retrieval-augmented generation (RAG)?
Retrieval-augmented generation (RAG) answers a request in two steps. It first retrieves relevant evidence from an external corpus, then gives that evidence to a language model, which generates an answer grounded in it. Knowledge therefore comes from the corpus at request time, not only from what the model learned in its weights.
A typical request passes through these stages:
- Query processing: the question is used as is or rewritten into a better search query.
- Retrieval: a search over an index of document chunks returns candidate passages. The search can use vector similarity, keywords or both.
- Reranking (optional): a second stage reorders the candidates so the most relevant come first.
- Context assembly: the best passages are placed into the prompt within a token budget.
- Generation: the model answers from that context, ideally citing the passages it used.
Compared with relying on the model’s weights alone, RAG changes two things. Knowledge can be updated by changing the corpus and its index instead of retraining, and an answer can point back to its sources. Each stage also limits the next: generation cannot use evidence that retrieval never returned.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al.)
How should an advanced RAG stage, such as reranking or query rewriting, be introduced and evaluated?
An advanced stage is introduced as a controlled experiment. It targets a specific failure measured on the current pipeline, and it stays only if it beats that baseline by enough to justify its added latency, cost and maintenance.
- Baseline: the current pipeline is measured on representative cases, and the failure the new stage should fix is identified. An example is relevant evidence ranked too low to reach the prompt.
- Isolated change: where practical, only that stage changes. A broad change to several stages can hide which component helped and which one regressed.
- Same comparison: baseline and candidate run on the same cases. They are measured on the targeted metric, on end-to-end answer quality and on operational measures such as p95 latency and cost per query.
- Decision: the stage is kept only when its quality gain outweighs its latency, cost and maintenance burden. Otherwise the simpler pipeline remains.
Changing exactly one thing is not always feasible. A feature flag, a controlled rollout or an ablation that removes one component at a time can still attribute an effect to a specific stage.
When a RAG answer is wrong, how can the failing pipeline stage be located?
The diagnosis starts from the exact context and request the model received for that answer. Whether the needed evidence was there decides which part of the pipeline to examine, and tracing backward from there finds the stage that failed.
- Actual model input: the logged prompt and assembled context for the failing request are inspected, rather than assumptions about what retrieval probably returned.
- Evidence missing or unsuitable: the fault is upstream of generation. Tracing back through context assembly, ranking, filtering, retrieval and query transformation shows where the evidence was lost. It may have been retrieved but cut by the final top-k, removed by a filter, or never retrieved because a rewritten query changed the intent.
- Evidence present but misused: the relevant evidence reached the model, but the answer ignored, contradicted or went beyond it. That points to generation: the prompt, the model or its settings.
- Measurement itself: citation handling and the reference answer or evaluator are checked too, because a broken citation mapping or a wrong label can make an answer look wrong.
Component metrics, such as retrieval relevance and groundedness, narrow the hypotheses. End-to-end evaluation then confirms whether a fix improves the task outcome.
Why does a RAG pipeline retrieve candidates first and then rerank them with a cross-encoder?
The two stages split the work between a fast scorer that can cover the whole corpus and a slower, finer-grained scorer that can only afford a short list. First-stage retrieval cheaply finds candidates across the corpus. A cross-encoder then rescores only those candidates; running it over every document would be far too slow.
Aspect Bi-encoder (first-stage retrieval) Cross-encoder (reranker) How it scores Embeds the query and each document separately. Document vectors are computed once at indexing time and compared with one query vector Reads the query and one candidate together in a single model pass and outputs a relevance score Relevance judgment Coarser, because query and document are encoded without seeing each other Can be finer, because attention spans both texts, so a qualifier or negation in the query can change the score. This often ranks better, but the gain depends on the model and the domain Cost per query One query encoding plus a fast index search One model inference per candidate Role Recall: getting the relevant evidence into a candidate list, often tens of items Precision: putting the best candidates into the small top-k that reaches the prompt Keyword search such as BM25 can also fill the first-stage role. Whatever the first stage is, the reranker only reorders what it returned: if the relevant document is not among the candidates, no reranker can recover it. Because the ranking gain is not guaranteed, it is confirmed on domain queries against the first-stage order before the reranker is adopted.
Why can a reranker improve retrieval metrics without improving the generated answer?
A retrieval metric can improve in ranking positions that never reach the generator. Even when better evidence does reach the generator, the model still has to use it correctly.
- Outside the final cutoff: a ranking metric is often computed over more results than the pipeline puts into the prompt. In this example, only the top 5 reranked chunks are assembled. Moving a relevant chunk from position 15 to position 8 raises a rank-sensitive metric such as nDCG@20, but the model input stays identical.
- Generation-side limits: retrieval and generation fail independently. If the needed evidence was already in the prompt, reordering it may not change the answer. If the model ignores, misreads or contradicts the evidence, better retrieval does not fix that.
Finding where the gain lands therefore requires following the same queries through each stage. The stages are candidate Recall@N before reranking, the ranking after reranking at the cutoff actually used, and the assembled context. Answer quality is then evaluated on representative queries. A retrieval metric gain alone does not establish a user-visible benefit.
Retrieve & Re-Rank (Sentence Transformers) · Optimizing LLM accuracy (OpenAI)
When can reranking worsen retrieval quality, and when can it fail to improve it?
A reranker trained on short web passages may misjudge long technical documents or unfamiliar terminology. It can then worsen results by pushing relevant evidence from inside the first-stage top-k to below that cutoff. A small candidate set or low first-stage recall causes a different failure. A reranker only reorders the candidates it receives, so it cannot add relevant evidence that is missing from them.
Domain queries tell these cases apart when recall and precision are measured at matched points:
- Missing candidates: low candidate Recall@N before reranking means the evidence was absent before reranking began. That is a first-stage limit.
- Demotion: the comparison cuts the first-stage order and the reranked order at the same final k. Demotion means relevant evidence was inside the first-stage top-k but falls outside the reranked top-k. A net drop in Recall@k between the two orders signals it in aggregate. A low absolute Precision@k does not show demotion. With one relevant document ranked first in both orders, Recall@N and Recall@10 are both 1.0, yet Precision@10 is only 0.1, and nothing was demoted.
- No promotion into top-k: when relevant candidates fall outside top-k in both orders, their ranks in the two orders show whether reranking helped. A move from rank 50 to rank 11 at k = 10 is a partial improvement that still misses the final cutoff. An unchanged or worse rank means the reranker did not help that evidence. Exclusion in both orders does not by itself show that k is too small. Cutoff capacity matters only when the query needs more distinct relevant pieces than k can hold.
Why does hybrid retrieval combine keyword search and vector search?
Hybrid retrieval combines them because they fail in complementary ways. Keyword search matches the exact terms in a query, while vector search matches meaning even when the wording differs. Running both covers queries that either one alone would miss.
Aspect Keyword search, such as BM25 Vector search What it matches Documents that share the query’s terms, with rarer terms counting more Documents whose embeddings are close to the query’s embedding, so related meanings match Strong on Exact identifiers such as error codes, product names and version numbers Paraphrases and natural-language questions worded differently from the source Weak on Synonyms and rephrasing with no word overlap, such as credential validation error for authentication failure Exact tokens, which can blur inside the embedding and surface topically similar but wrong chunks What it needs A lexical (inverted) index An embedding model and a vector index In a hybrid search, both queries run in parallel and their ranked lists are merged, commonly with Reciprocal Rank Fusion (RRF). RRF scores documents by their rank positions, so the two incompatible score scales never need to be normalized. For a query about error 429 behavior for the partner tier, keyword search can catch
429while vector search finds rate-limiting passages phrased differently.Hybrid is a common starting point rather than a guaranteed improvement over either mode alone.
When can hybrid retrieval perform worse than single-mode retrieval?
Hybrid retrieval can perform worse when one retrieval path is much weaker on a particular corpus and query mix. Fusing its results pulls poor candidates into the top results. There they displace good results that the stronger path would have returned on its own.
The mechanism:
- Keyword search, such as BM25, and vector search run in parallel for the same query.
- Their ranked lists are merged, commonly with Reciprocal Rank Fusion (RRF), which scores each document from its rank positions rather than from raw scores.
- Standard RRF gives each list equal influence. For documents that appear in only one list, rank 1 in the weak list scores the same as rank 1 in the strong list, and higher than rank 3 in the strong list.
- With a small final cutoff, those weak candidates push relevant results out.
Whether this happens depends on the workload. In a homogeneous corpus where one mode clearly dominates, the other mode has little useful to add. A large top-k can also carry weak tail candidates into the context when no reranking or deduplication follows.
Hybrid is therefore justified by measurement rather than assumed. Keyword-only, vector-only and hybrid retrieval are compared on representative queries, using the retrieval or end-to-end metric that matches the product goal. When hybrid loses, the fusion can be weighted toward the stronger path, or the extra path can be dropped.
What role does a vector database play in a RAG system?
An embedding model creates numeric representations; a vector database or search index stores and indexes them, then retrieves nearby vectors for a query. The service can return associated IDs or fields and apply metadata filters; application code resolves the results to source text and assembles prompt context. Vector similarity finds candidates, but does not prove relevance or replace an authorization check. Vector search overview
Why do vector indexes use approximate nearest-neighbor search instead of exact search?
Exact search must compare the query with every stored vector, so its work grows with the size of the collection. An approximate nearest-neighbor (ANN) index examines only a promising subset of vectors. This often cuts query work sharply, at the cost of sometimes missing true nearest neighbors.
- Exact (flat) search: one distance computation per stored vector, about N × d arithmetic for N vectors of dimension d. The results are exact, and the cost is acceptable for small collections.
- ANN search: the index structure limits which vectors are compared. An IVF index clusters the vectors and searches only the clusters nearest the query. An HNSW index walks a proximity graph toward closer neighbors. Parameters such as
nprobefor IVF orefSearchfor HNSW control how much is examined, trading latency for recall.Missed neighbors raise no error. Recall is therefore measured against exact-search results as ground truth, on realistic queries and filters, and the index settings are chosen to meet both the recall and the latency targets. The N × d figure describes arithmetic work, not a guaranteed wall-clock time.
How do Recall@k, Precision@k, HitRate@k, MRR and nDCG@k differ as retrieval metrics?
All five compare a ranked result list with labeled relevant documents, but each rewards something different: coverage, low noise, any success, an early first hit or a well-ordered ranking.
Metric What it measures Effect of rank within the top k What it misses Recall@k Share of a query’s relevant documents that appear in the top k None How high the relevant documents rank and how much noise surrounds them Precision@k Share of the top k results that are relevant None Relevant documents that were never retrieved HitRate@k Share of queries with at least one relevant result in the top k None Every missing document after the first hit MRR Mean, across queries, of 1 / rankfor the first relevant result, counting0when none is foundStrong, but only for the first hit Everything after the first relevant result nDCG@k Sum of the relevance grades in the top k, each discounted by its position, divided by that sum for the ideal ordering Applies across the whole top k Whether a low score comes from missing documents or from poor ordering Example: a query has four known relevant documents, and its top five results are
[irrelevant, relevant, relevant, irrelevant, relevant].
- Recall@5 is
3/4 = 0.75, because three of the four relevant documents appear.- Precision@5 is
3/5 = 0.6, because three of the five results are relevant.- The query counts as a hit for HitRate@5.
- Its reciprocal rank is
1/2 = 0.5, because the first relevant result is at rank 2.Recall sets a ceiling for generation, because the generator cannot use evidence it never receives. Precision matters when context space or token cost is tight. MRR fits a generator that reads only the top one or two chunks. nDCG@k fits a generator that uses all k chunks and benefits from seeing the strongest evidence first.
Sources:
Why can aggregate retrieval metrics hide regressions, and when is a segment drop actionable?
An aggregate metric averages over all queries, so a gain on large, healthy segments can outweigh a serious loss on a small one. A segment drop is actionable when it matters to that segment’s users and is too large to explain as sampling noise.
Example: a change raises overall Recall@5 by 2%, while recall for one tenant’s query cluster falls by 15%. A gate on the global average passes, yet that tenant now receives worse evidence.
Slicing results by dimensions that change the retrieval problem, such as tenant, language, query class or document source, exposes such losses. Slicing has its own risk. A small segment has few labeled queries, so its score depends heavily on which queries happen to be in the set. A segment drop is credible and actionable when:
- the segment has enough labeled queries, representative of its real traffic, for its score to be meaningful;
- the drop exceeds the uncertainty that comes from that limited query sample, for example by falling outside a confidence interval computed over the segment’s queries;
- it crosses a threshold set in advance from the product’s risk for that segment.
Rerunning the same evaluation on the same queries does not satisfy the second condition. Repeated runs show whether scores stay stable when retrieval or judging is nondeterministic. They add no queries, and they do not make the existing queries more representative.
A material, credible segment regression remains a regression even when the global average improves.
Sources:
In a RAGAS evaluation, what does high Faithfulness with low Context Recall suggest, and what should be checked first?
It suggests that the answer stays consistent with the context it received, but that context lacks evidence the reference answer needs. The first check is the measurement itself: the reference answer, the captured context and the metric configuration. Retrieval is investigated once those are confirmed sound.
- Faithfulness: the share of claims in the response that the supplied context supports. It does not show whether that context was complete.
- Context Recall: the share of claims in the reference answer that the retrieved context supports. It requires a reference and measures coverage of the evidence an answer needs.
Together, the scores fit a generator that stays within limited evidence, which can produce a grounded but incomplete answer.
The order of investigation:
- Measurement: the reference answer is correct, the scored context is what the generator actually received, and the metric variant, judge model and inputs are as intended. A flawed reference or a logging gap can produce the same pattern.
- Retrieval path: missing or unindexed documents, metadata filters, the candidate cutoff and context assembly.
- Generation: changed only after the retrieval path is ruled out, because a prompt cannot make use of evidence that never arrived.
Neither score alone proves the cause. The combination is a hypothesis to test on representative samples.
Sources:
What do retrieval, response, and end-to-end evaluations each show about a RAG system?
Each evaluation answers a different question, so separate results show where a RAG system failed rather than only that it failed.
Evaluation Question it answers A failure points toward What it cannot show Retrieval Was relevant evidence found, ranked well and selected for the context? Indexing, chunking, query handling, search or ranking Whether the answer then used good evidence correctly Response Given the supplied evidence, is the answer grounded in it, relevant to the question and, when a reference exists, correct? The prompt or the generator’s use of the evidence Whether better evidence existed and was missed End-to-end Did the complete system meet the task’s outcome criteria? A user-visible miss somewhere in the pipeline Which stage caused the miss Combined, the results localize a likely cause. A grounded answer built on poor or incomplete context points upstream to retrieval. Good context paired with a wrong answer points to generation.
Example: a support bot retrieves the correct policy but misreads its date constraint. Retrieval passes and the response check fails, so more search tuning cannot fix that answer.
Sources:
Which evaluation concerns are specific to RAG, and which can reuse general LLM evaluation methods?
RAG adds retrieval to evaluation: whether relevant evidence was found, ranked well and supplied in sufficient quantity. Retrieval metrics that score against labeled relevance judgments need relevance labels that other LLM tasks do not. Reference-free retrieval evaluation does not use such labels. Answer-quality checks and the evaluation methods themselves are largely reusable.
Aspect Specific to RAG Reusable from general LLM evaluation What is measured Retrieval relevance, ranking quality and whether the context covers the evidence an answer needs Correctness, relevance to the question, safety and format. Groundedness also applies to any answer constrained by supplied text, such as a summary of a given document. Ground truth For label-based retrieval metrics, relevance labels per query, which must allow several acceptable chunks or require a set of complementary ones Reference answers and rubrics, built as for any task Methods Ranked-retrieval metrics, such as Recall@k or nDCG, over those labels Golden datasets, deterministic checks, LLM judges, human review and online evaluation of production traces Any evaluator must match the claim being measured. Its limits need checking, as do any labels it uses. A judge reused from another task is not automatically valid for a new one.
Sources:
How should retrieval metrics and cutoffs be chosen when a query has several relevant chunks?
The metric depends on how the relevant chunks contribute to the answer, and the cutoff must match what each pipeline stage actually passes on.
Relevance shape Success means Fitting metric Misleading metric Substitutable: any one chunk answers, as with duplicated policy text At least one valid chunk appears, ideally early HitRate@k, plus MRR when the first relevant rank matters Strict Recall@k, which penalizes missing copies the generator never needed Complementary: the answer needs the whole set, as in a comparison Every required chunk appears Recall@k or RAGAS Context Recall HitRate@k, which passes after the first hit and hides the missing pieces Graded: some chunks are more useful than others The most useful chunks rank highest nDCG@k with graded labels, such as 2for direct evidence,1for support and0for irrelevantBinary recall or precision, which discard the grades Example: “How do plans A and B differ on refunds?” needs one policy chunk per plan. A result containing only plan A has
HitRate@5 = 1but complementary recall of1/2.Cutoffs:
- The final k equals the number of chunks the generator receives. A larger k credits evidence that never reaches the model.
- For complementary evidence, k must be at least the size of the required set. Otherwise recall cannot reach
1.0.- In a retrieve-then-rerank pipeline, first-stage Recall@N shows whether the evidence entered the candidate set, and the final top-k shows what reaches generation. Either measurement alone can hide a loss at the other stage.
Sources:
How can chunk-generated questions produce misleading retrieval evaluation labels?
A question generated from one chunk is labeled with only that chunk as relevant. That label can miss other valid answers, and the question can be easier to retrieve than real user queries.
- Missing positives (false negatives): a duplicate, an overview section or another passage may answer the question equally well. If it is not labeled, a retriever that returns it is scored as wrong, and comparisons between retrievers can be distorted.
- Lexical leakage: the question may copy distinctive phrases from its source chunk. Exact-term overlap then makes keyword retrieval look stronger than it would on real queries.
Both are risks to check, not certainties. Common controls:
- A second retrieval pass runs each generated question, and the top candidates are judged. Other valid chunks are added to the labels, or questions that stay ambiguous are dropped.
- Generation prompts ask for paraphrased, realistically incomplete wording, and the token overlap between each question and its source chunk is inspected.
Sources:
When should a team add human-labeled evaluation cases instead of relying only on synthetic generation?
Human-labeled cases become necessary when synthetic cases can no longer be trusted to represent the workload or to label it correctly. That happens when correctness depends on expert judgment, when failures are costly, or when production inputs differ from the generated ones.
- What synthetic generation does well: a model writes plausible requests from source material with known provenance. Coverage and a working evaluation harness therefore arrive quickly and cheaply.
- Where it falls short: generated cases share the generator’s phrasing and preferred difficulty, so ambiguous, messy or adversarial real inputs are under-represented. Generated labels can also miss subtle domain distinctions.
- What human labels add: reviewed cases ground the evaluation in real inputs and expert judgment. They also show whether synthetic labels and automated judges agree with people. ARES, for example, pairs synthetic training data with a small human-annotated set that corrects its automated estimates.
A practical evaluation set often combines synthetic breadth, human-reviewed cases and regression cases taken from real failures. The balance depends on the task’s risk and on how closely synthetic cases match production traffic.
ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems (arXiv) · Evaluation best practices (OpenAI)
Which deterministic checks should run before and after a tool-using agent calls a tool?
Before a tool runs, deterministic checks confirm that the caller is allowed to perform the requested action and validate the arguments against the tool’s contract. After the call, they validate any returned data against the schema the next step expects.
- Before execution: a permission or allowlist rule rejects an action the caller may not take. Argument validation rejects missing, malformed or out-of-range inputs. These checks must come first because later validation cannot undo a side effect: an agent can call
delete_record, receive a success result and still return perfectly valid JSON.- After execution: result validation stops malformed or unexpected data from reaching the next step.
A strict function schema constrains the shape of arguments. Schema validity neither authorizes the action nor shows that it was the right action. A schema-valid request can still choose the wrong permitted action, such as
searchwhen the case requiredescalate. The product’s risks determine which further checks are needed and whether a failure is rejected outright or routed to review.Sources: OpenAI: Function calling · OWASP: LLM06 Excessive Agency · JSON Schema Core
When does an LLM judge fit better than a deterministic check?
An LLM judge fits when acceptance depends on meaning or quality that cannot be written as an exact rule, such as whether an answer is correct for the case, grounded in its sources, relevant or clear. A deterministic check fits any property that can be stated exactly, such as schema validity, an allowed action or a length limit. For those properties it remains the better tool even when a judge is available.
Aspect Deterministic check LLM judge What it can decide Exactly stated properties such as format, structure, allowed actions and hard policy rules Open-ended qualities that a rubric can describe but code cannot fully specify Repeatability The same rule and input always give the same result Verdicts can change with the judge model, prompt and sampling settings Cost Local parsing or scanning A model call for every judgment How it fails Exact for schemas and allowlists; heuristic scanners such as PII detectors can misclassify It can misread the rubric or the answer, or favor irrelevant features such as length The two approaches combine. Deterministic checks run first, so a malformed output or a forbidden action fails immediately without the cost of a judge call. A good semantic score cannot rescue a failed hard rule. The judge then evaluates the remaining semantic criteria, and its verdicts are trusted only after calibration against human labels.
Sources: OpenAI: Evals guide · Zheng et al.: Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena · JSON Schema Core
What evidence supports trusting an LLM judge's verdicts?
An LLM judge earns trust through measured agreement with human judgment on representative cases. It also needs tests showing that its verdicts do not depend on irrelevant features such as answer order. This evidence supports the judge’s reliability; it does not prove that the rubric measures the right thing.
- Agreement with human labels: the judge scores a fixed set of cases that humans have already labeled, and its agreement with those labels is measured. Disagreements are inspected because they reveal whether the rubric is ambiguous, the reference is incomplete or the judge is biased.
- Order-swap consistency: in pairwise comparison, each pair is judged twice, once in each order. A consistent judge prefers the same underlying answer both times, so its A/B label switches along with the answers. A judge that keeps choosing the same position instead follows position rather than content, which is position bias. A pair whose preferred answer changes with the order is treated as a tie or sent to human review.
- Bias probes: calibration cases set concise correct answers beside padded or repetitive ones to expose verbosity bias. They also include outputs from the judge’s own model family to expose self-preference.
- Recalibration after changes: the judge model, prompt, rubric and sampling settings together define the measurement. A change to any of them can make scores stricter or looser, so the calibration set is rerun before new scores are compared with old ones.
High agreement shows only that the judge matches the labelers on those cases. A judge that consistently applies a flawed rubric is stable but still measures the wrong thing.
Sources: Zheng et al.: Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena · OpenAI: Evals guide
How do context, harness, and loop engineering differ in what they control?
The three disciplines own different decisions about the same agent run. Context engineering decides what the model sees. Harness engineering decides what the model’s proposed actions can reach and do. Loop engineering decides whether and how the work continues across turns.
Discipline Decision it owns Typical controls Context engineering Which information the model receives on each call Selecting and compacting instructions, retrieved evidence, history and tool results Harness engineering What the runtime allows a proposed action to do once the model emits it Which tools exist, their permissions and credentials, call validation, sandboxing and approval gates Loop engineering Whether another turn happens and when the run stops Progress and verification checks, iteration and cost budgets, and termination The boundaries touch. Tool schemas take up context and tool results feed later turns, so context engineering decides how much of them the model sees. Whether a tool such as
delete_branchexists at all, and which policy approves its use, remains a harness decision.Sources: Anthropic: Effective context engineering for AI agents
When should a tool remain an in-process function instead of being exposed through an MCP server?
A tool should stay an in-process function when only one application uses it, especially when it works directly on that application’s private state or business logic. An MCP server is worth its extra moving parts when several compatible clients need the same capability, managed and secured separately from any one application.
The choice concerns where the tool lives, not how the model calls it. With function calling, the model returns a structured request and the application executes it. MCP is a client/server protocol through which a server advertises tools and other capabilities to any compatible client. A host can discover an MCP tool and present it to its model as a function definition. That translation is done by the host or its SDK, not by function calling itself.
Aspect In-process function tool Tool behind an MCP server Reuse Belongs to one application; other applications reuse it only through shared code or services Any compatible MCP client can discover and call it Application state Reads the application’s in-memory state directly Cannot see the host application’s memory; needed context arrives in each request, for example as an explicit handle that the server must authorize Runtime cost A local method call A separate process or network hop, plus deployment and lifecycle for the server Trust Runs inside the application’s own security context Runs as a separate server that each host must configure and trust Sources: OpenAI: Function calling · MCP specification 2026-07-28: Tools · MCP 2026-07-28 update
Which security checks must an MCP server still perform after HTTP authorization admits a caller?
HTTP authorization admits a caller only as far as its access token allows: the token must be issued for this MCP server and carry the scopes the request requires. The server still decides, in its own code, whether that caller may perform the operation on the specific object and tenant involved. It also validates every tool input before using it.
- Covered by MCP authorization for HTTP-based transports: the server accepts only an OAuth access token issued for itself as the intended audience, and it can require particular scopes, such as a write scope for a tool that changes data. Audience and scopes decide who may reach the server and which kinds of operation the token permits.
- Left to application-defined authorization: whether this caller may act on this particular object or tenant, such as deleting a specific record or reading another tenant’s data. A write scope permits writing in general but does not say which records. The server also validates tool arguments, rate-limits tool calls and sanitizes tool output.
Tool names, descriptions and results are text placed in the model’s context. They can influence what the model requests, but they enforce nothing, so permission checks belong in server code.
Sources: MCP specification 2026-07-28: Authorization · MCP specification 2026-07-28: Tools
Why use path-specific instruction files for coding agents, and what must be checked before relying on them?
Path-specific instruction files apply guidance only to files whose path matches a pattern. An agent working on those files receives the local constraints it needs, while guidance for unrelated parts of the repository stays out of its context. Before relying on them, the selected agent’s documented rules for loading, precedence and conflict handling must be checked, because those rules are tool-specific.
- Example: in GitHub Copilot CLI, a path-specific file declares an
applyTopattern. When the agent works on a matching file, those instructions are combined with the repository-wide instructions.- Precedence: GitHub’s documentation describes this combination but does not define a general precedence order for conflicting instructions. If two files name different test commands, the outcome depends on runtime behavior rather than on a documented rule.
- Maintainability rule: local files written as refinements of the root guidance, rather than reversals, do not depend on how a runtime resolves conflicts. This is a maintenance discipline, not a portable precedence guarantee.
What controls reduce risk when a coding agent can change a repository?
The main controls are a bounded task and budget, restricted permissions, runtime-enforced approval or denial of consequential actions, and CI plus review of the resulting change. They work because the runtime and delivery process enforce them; instructions alone depend on the model complying.
Control What it does Risk it reduces Narrow task, limited tools, step or iteration budget Restricts what the agent attempts and how long it can keep retrying Broad diffs that are hard to review, and costly retry loops that repeat the same failure Sandbox and permissions Limits filesystem, network and command access, where the platform provides these controls Actions outside the intended workspace or network reach Runtime approval or denial Requires approval for consequential actions or blocks them before execution; in GitHub Copilot, a hook that runs before a tool call can deny that call Dangerous actions that the model was only told to avoid CI and code review Check the resulting change before it merges Defects and boundary violations in the produced code Each control covers specific paths and failure modes. Passing tests and review reduce risk but do not prove that every tool call or behavior was safe. A change can pass the suite and still violate an architectural boundary that the tests do not encode.
Source: GitHub Copilot hooks reference
When should repeated agent failures be fixed in a tool contract instead of the prompt?
A failure belongs in the tool contract when it recurs across different requests that use the same tool: the wrong tool is selected, arguments are malformed, or errors are not recovered from. A pattern that repeats across requests points to the interface, not to one request’s wording. The contract is part of every call to that tool, so a change there reaches all later calls. A prompt workaround works only when the model remembers and follows the exception.
Reaching every call does not guarantee that the failure stops. Rerunning representative tool tasks and comparing selection, argument and recovery failures before and after the change shows whether it actually helps.
Contract-level fixes include:
- Names and descriptions that state what the tool does, what it returns and when a different tool fits.
- Parameters with explicit formats, enums and required fields, leaving fewer ways to produce a call that parses but means nothing.
- Errors that say what failed and how to correct the call, so the next attempt has a concrete repair.
- Results that contain the fields the next step needs, including identifiers a later call may use.
For example, Anthropic reported that its SWE-bench agent made mistakes with relative file paths after moving out of the root directory. It also reported that those mistakes stopped once the tool was changed to always require absolute paths.
The prompt remains the place for task intent and for instructions specific to one request.
Sources: Writing effective tools for AI agents (Anthropic) · Building effective agents (Anthropic)
What determines whether related operations should use one broad tool or several narrower tools?
The deciding factor is whether the operations share one contract. Operations that need different permissions, input schemas or descriptions belong in separate tools. Closely related actions that share context, permissions and arguments can stay in one tool.
Signal Points toward Reason Different permission checks Separate tools Each tool can be authorized and exposed on its own. One schema collecting unrelated optional fields Separate tools Fewer argument combinations remain that are valid JSON but meaningless as a request. Actions sharing context, permissions and arguments One tool A single description and schema cover them without unrelated options. Many similar or overlapping tools already exposed Fewer, clearer tools Each additional tool is another option the model must tell apart, so selection errors can rise. Narrower tools usually simplify each call’s arguments but enlarge the set the model chooses from. The net effect depends on the real task mix and the whole exposed catalog, so it is measured rather than assumed. Tracking tool-selection accuracy and argument correctness separately on representative tasks shows which failure a change actually reduces.
What does ReAct add to language-model reasoning when a task needs external information or computation?
ReAct adds acting to reasoning. The model interleaves reasoning steps with actions such as a search, an API call or a calculation, and each action’s result returns as an observation that informs the next reasoning step.
- Reason: the model determines what is missing and which action could supply it.
- Act: the model requests a tool call, and the runtime executes it.
- Observe: the result is added to the model’s context.
- Continue or answer: the model reasons over the new evidence, then requests another action or gives a final answer.
Reasoning alone draws only on the model’s internal knowledge and the initial prompt, so an invented fact early in a reasoning chain can carry through to the answer. Observations ground each next step in external evidence, while reasoning decides which action is worth taking and how to interpret its result.
ReAct does not guarantee a correct answer: a tool can return poor evidence, and the model can misread it. Each sequential cycle also adds a model call and more context, although a runtime can batch or parallelize independent actions.
Sources: ReAct: Synergizing Reasoning and Acting in Language Models · Claude: How tool use works
In a tool-using agent loop, which safeguards must be enforced outside the model, and what failure does each contain?
Hard limits on iterations, time and spending, plus validation of every tool call before it executes, must be enforced by the runtime. The model cannot be relied on to stop itself or to request only valid actions, so these checks live in code the model cannot override.
Safeguard Failure it contains Iteration cap per request A loop that keeps calling tools and never reaches a final answer Time limit A slow or hanging call that never finishes its iteration, so an iteration cap alone would never trigger Token and cost budget Context growth that overflows the window, or spending that exceeds its limit Repetition detection The same call repeated, or two states alternating, without progress Tool-call validation Unknown tools, schema-invalid arguments or unauthorized requests reaching a side-effecting operation; a precise error lets the next iteration correct the call Frameworks expose some of these limits directly; the OpenAI Agents SDK, for example, bounds a run with
max_turns. Prompt instructions can reinforce the limits but cannot replace them, because none may depend on the model obeying.Sources: OpenAI Agents SDK: Running agents · Claude: How tool use works
What does a tool-using model need from a runtime to operate as a loop?
The runtime executes requested tools, returns observations to the model, and decides whether another turn runs. It owns the completion check and operational bounds; the model’s final text alone does not prove the task succeeded. A one-call tool request becomes a loop only when the runtime sends its result back for another decision. Claude tool-use loop
What should a runtime report when its loop budget expires before the task is complete?
The runtime reports that the run stopped incomplete, not that it succeeded. The report names the limit that was reached, the completion condition that is still unmet, and the partial result or state produced so far.
- Why it matters: budget exhaustion is not success. If partial output is presented as a final answer or silently truncated, callers treat unfinished work as done and the failure stays hidden.
- What follows: where the application configures it, the partial state and the unresolved work pass to a continuation run or to a person. Otherwise the run ends with an explicit error.
Platform behavior differs. The OpenAI Agents SDK treats reaching
max_turnsas an error condition and supports a handler for that case. With Anthropic task budgets, a task that exceeds its budget can end with a partial result.Sources: OpenAI Agents SDK: Running agents · Anthropic task budgets
In a long-running agent, how is compaction work split between deciding what context to keep and deciding when to compact?
Compaction needs two separate decisions. The content decision, owned by context engineering, determines what the model still needs and what a summary must preserve. The timing decision, owned by loop engineering, determines when to compact, offload an artifact or stop. Compaction typically runs at a token threshold, before the next request would overflow the window.
Decision What it settles Failure when it is neglected Content (context engineering) Which goals, decisions, open issues and references a summary keeps The summary drops a constraint, and later turns proceed without it Timing (loop engineering) When to summarize, when to move large artifacts to external storage behind lightweight references, and when to stop The window overflows, and truncation removes old messages that may still matter Durable state sits outside both decisions. A plan or progress log stored by the application can be reread after compaction, which keeps a long run tied to its original goal. OpenAI Responses API compaction, for example, carries prior state forward as smaller, opaque compaction items. Plans and artifacts that the application must rely on remain application-managed.
This split is a convention for assigning responsibility, not a boundary defined by an API.
Source: OpenAI: Compaction
When is multi-agent coordination worth its added complexity?
Multi-agent coordination is worth its cost when at least one of these pressures exists and evaluation shows an improvement in a required outcome over a single agent:
- Context isolation: a large subtask would fill the main context with material the coordinator does not need. A separate agent explores it and returns a condensed result.
- Parallel work: independent paths can run at the same time, and latency matters.
- Specialization: part of the task needs a different tool set, or instructions that conflict with the rest of the task.
The cost is real. Each split adds handoffs, duplicated context, more traces and new coordination failures. Agents can also pass work back and forth, so the whole run needs turn and token limits, not just each agent. Anthropic reported that its multi-agent research system used about 15 times as many tokens as ordinary chat interactions. That multiplier describes one workload, but the direction of the cost is general.
A useful split gives each agent a bounded body of context that it can summarize back as an artifact or decision. Closely related work, such as implementing a feature and testing it, stays with one agent, which can reuse its evidence and decisions instead of rebuilding them after a lossy handoff. Without one of the pressures above, a single agent with well-designed tools is usually cheaper and easier to debug.
Source: Anthropic: How we built our multi-agent research system
When one agent hands work to another, how much prior context should the receiving agent get?
The receiving agent should get the context its assignment needs, not automatically everything that came before. More history preserves detail but dilutes focus and exposes more data. Less history keeps focus but can drop an edge case the next agent needed.
Handoff content Benefit Risk Full conversation history Nothing is lost, and it is the simplest option to wire up Irrelevant turns compete with the assignment, and all prior data is exposed to the receiver Scoped assignment or summary The receiver sees only its task, which keeps it focused and limits data exposure A summary can omit or distort a detail; in a chain of handoffs each summary summarizes the previous one, so loss compounds Small assignment plus references to durable artifacts Focus is kept while exact source material stays retrievable, and structured outputs can be validated before use Requires shared storage and an agreed artifact format In the OpenAI Agents SDK, a handoff gives the receiving agent the previous conversation history by default. An
input_filtercan change what is passed, for example by removing earlier tool calls. Reducing that history is a deliberate decision about focus and data exposure.Source: OpenAI Agents SDK: Handoffs
What evidence makes failures across multi-agent handoffs traceable?
Failures become traceable when a run records who produced each piece of work, what crossed each handoff, and why routing and tool decisions were made, all linked to the same run.
- Run-level correlation: every agent’s activity in one workflow belongs to a single trace, so a failure in a later agent can be followed back to earlier steps.
- Artifact provenance: each artifact records the agent that produced it and the evidence behind its key claims, so an unsupported premise can be found where it entered.
- Handoff contents: the assignment, summary or references passed to the receiving agent show whether a detail was lost or distorted at the transfer.
- Decisions: routing choices and tool calls are recorded with their arguments and results.
Forwarding every raw message to every agent is not required. Structured artifacts with provenance preserve the chain of evidence with less context and less data exposure. The retained trace is kept under the system’s access and retention policy. The OpenAI Agents SDK, for example, records agent runs, model generations, tool calls and handoffs as spans within a trace.
Source: OpenAI Agents SDK: Tracing
When should a prompt move from zero-shot to one-shot or few-shot?
A prompt moves beyond zero-shot only when evaluation shows a gap that examples can close. Each added example stays only if it improves results on a representative evaluation set.
- Zero-shot: instructions without examples. This is enough when the instruction and allowed labels already define the task, and it is the cheapest baseline that later prompts must beat.
- One-shot: one complete input-output pair. This fits when the task is understood but the output shape is not, such as JSON keys or field order.
- Few-shot: several diverse examples. These fit when they show decision boundaries better than prose can, such as classes that are easy to confuse. Near-duplicate easy examples add tokens without adding information.
The evaluation gate matters because examples are not free. Each one consumes context on every request. Example choice, order and formatting can also change behavior, so a model upgrade or a reordering can require reevaluation. When inconsistent output comes from an ambiguous instruction or conflicting example formats, more examples can amplify the problem, so fixing the instruction and format comes first.
Examples make a format likely, not guaranteed. A strict output shape still needs structured outputs or schema validation.
Sources: Min et al., Rethinking the Role of Demonstrations · OpenAI Structured Outputs
How does in-context learning work, and how does it differ from fine-tuning and retrieval?
In-context learning adapts a pretrained model to a task through instructions and examples placed in the prompt. The model still predicts the next token, and the examples shift that prediction by showing the task structure, the label space and the output format. No weights change, so the effect applies only to requests that include the examples.
Aspect In-context learning Fine-tuning Retrieval What changes The prompt of one request. The model’s weights, through a training run. The context, filled at request time with passages from an external store. What it mainly supplies How to perform the task: format, labels and boundary cases. Stable learned behavior that no longer has to fit in every prompt. Evidence the model lacks, such as current or private documents. How long the effect lasts Only while the examples are in the prompt. Until the model is retrained or replaced. It reflects whatever the index contains when the request runs. Main cost Context tokens on every request, plus sensitivity to example choice and order. Training data, training runs and a model lifecycle to maintain. Indexing and retrieval quality; poor retrieval caps answer quality. The mechanisms combine: one prompt can contain retrieved passages and a few examples and be sent to a base or fine-tuned model. Examples are still not a reliable substitute for current or private source material, because they demonstrate behavior rather than serving as a dependable store of facts.
Sources: Brown et al., Language Models are Few-Shot Learners · Min et al., Rethinking the Role of Demonstrations · OpenAI prompt engineering
What can few-shot examples clarify, and what problems do they not solve?
Few-shot examples clarify how a task should be performed. They show the output format, the allowed labels, conventions such as casing and field order, and the boundary between easily confused cases. They are not a dependable source of facts and cannot enforce permissions or runtime limits. Their own effect also depends on how they are chosen.
What they do not solve:
- Missing or changing facts: examples condition the response to the current prompt without updating model weights. They demonstrate behavior, so they are not a reliable store for facts absent from the model and the supplied context. Current or private evidence comes from trusted context supplied with the request, such as provided documents or retrieved passages.
- Permissions and runtime limits: prompt text cannot enforce authorization, budgets or loop termination in a multi-step workflow. Those controls run in the runtime, outside the model.
Their effect is also sensitive and costly. Example choice, format and order, as well as the model itself, can change the result, so a change to any of them needs reevaluation. A long example block also uses context needed for the request and its source material.
Sources: Min et al., Rethinking the Role of Demonstrations · OpenAI prompt engineering
When is prompt chaining a better fit than one large prompt?
Prompt chaining fits better when a task has real stages and an intermediate result must be checked before later work can trust it, or when that result determines what happens next. One prompt fits better when no intermediate result needs separate treatment. Each extra call adds latency, token cost and another boundary where errors can propagate.
A chain earns its cost only when its boundaries are real:
- Each stage has one narrow job and an explicit output contract, such as entities as JSON or an intent label from a closed set.
- A deterministic check runs before the next stage uses that output. A missing required field or an unknown label stops the chain instead of corrupting later steps.
- Failures can be localized, because each stage’s input and output can be inspected separately.
For example, a support assistant extracts entities, validates them against a schema, classifies intent into an allowed set and only then writes the reply. If the error code
ERR-42comes back aserror_code: null, a required-field check stops the chain. Without that check, the classifier may route the request as a general inquiry.Splitting one vague prompt into several vague prompts adds failure points without adding control. For that reason, the complete chain is evaluated end to end against a simpler single-prompt baseline.
Sources: Anthropic, Building effective agents · OpenAI evals
What is a practical meta-prompting workflow for improving a weak prompt?
A practical workflow treats a model-drafted prompt revision as a candidate that must beat the current prompt on held-out cases before it ships.
- Collect failures: the starting evidence is observed failures from real outputs, such as incident summaries that omit the timeline and root cause.
- Group by cause: failures are grouped by likely cause, such as a missing output requirement or an ambiguous instruction. One revision then targets a cause instead of patching single cases.
- Draft a candidate: a model receives the current prompt, the grouped failures and the required output contract, and proposes a revision. In the incident example, the revision can require
timeline,root_causeandactionsas mandatory JSON keys.- Compare on held-out cases: the candidate and the current prompt are both scored on the known failures and on representative cases that were not used to write the revision.
- Version and gate: the prompt is versioned together with its evaluator, and a rollback threshold is set before deployment.
The held-out comparison is the check for meta-prompt overfitting, where a revision fixes the remembered failures but silently breaks ordinary inputs. It can reveal that regression before release, which reduces the risk without ruling it out. Inputs that the evaluation set does not represent can still regress, so the rollback threshold limits the damage after deployment. A model-drafted revision is a hypothesis to test, not a reliability guarantee.
Sources: OpenAI model optimization · OpenAI evals
Why must prompt instructions and generation settings be evaluated together?
The prompt and the generation settings control different parts of the result, and neither can compensate for a defect in the other. The prompt defines the task, context, constraints and requested output. Where the selected model and endpoint support them, generation settings control sampling and output limits.
- A low-randomness setting, such as a low temperature, makes output more repeatable but cannot resolve an ambiguous instruction. It can just produce the same misreading more consistently.
- A clear prompt cannot make an unsupported setting take effect. Parameter support differs across models and endpoints, so values tuned on one model may not carry over to another.
Because the result depends on the combination, prompt and settings are evaluated together on a task-specific set for the selected model and endpoint. A change to either one, or a switch to another model, calls for reevaluation.
Sources: OpenAI prompt engineering · OpenAI reasoning best practices
How can an output-token cap limit cost without truncating valid answers?
An output-token cap limits cost as a hard ceiling against runaway generation, not as a way to shorten normal answers. Normal length and cost are reduced through the prompt. The cap is sized from the measured total of generated tokens plus headroom, within the room the context window leaves after the input.
- Prompt first: repeated instructions, irrelevant context and redundant examples are removed, and the desired answer shape is stated directly, such as a label, a fixed set of fields or a length limit. This lowers cost without risking a cut-off answer.
- Cap as a ceiling: generation stops when the cap is reached, whether or not the answer is complete. A cap is not a reliable request for concise wording, so a cap set near typical length truncates longer valid answers. A truncated JSON object, for example, no longer parses.
- Total generated tokens: the cap is sized from the generated-token counts of valid responses on the evaluation set, plus headroom. Where a model’s reasoning or other non-visible tokens count toward the output limit, they belong in that total. Hidden reasoning can then use up the cap before a short visible answer is finished, or before any visible output appears.
- Context capacity: input and generated tokens share the model’s context window. A long input leaves less room for output, so generation can stop at the context limit before it reaches the cap.
- Detecting truncation: a response that stopped at a limit is identified from the stop or incomplete reason that the API returns. It is handled as truncated rather than complete.
Parameter names and limit reporting are endpoint- and model-specific. In OpenAI’s APIs, Responses uses
max_output_tokensand Chat Completions usesmax_completion_tokens. On OpenAI reasoning models, reasoning tokens count toward that limit and are billed as output tokens. A Responses call that hits the limit returns anincompletestatus with the reasonmax_output_tokens.Sources: OpenAI token counting · OpenAI reasoning models
When can Chain-of-Thought prompting help, and when can it add risk or waste?
Chain-of-Thought prompting can help when a task carries intermediate state across several steps, such as multi-step arithmetic, and the model is large enough to use worked reasoning. It can waste tokens when the task has little intermediate state. It can also waste them on models that already reason internally and whose provider advises against explicit step-by-step prompts. It adds risk when the visible trace is trusted as proof.
Where it helped: in Wei et al. (2022), chain-of-thought demonstrations improved some tested arithmetic, commonsense and symbolic benchmarks, and the gains depended on model scale. Writing out intermediate steps can turn one hard generation into a sequence of smaller continuations. These are results for the tested benchmarks and models, not a universal rule.
Where it can add risk or waste:
- Cost: longer outputs add tokens and latency. That cost is waste whenever the reasoning does not improve the answer on the selected model and task.
- Error propagation: a longer trace has more steps in which an early wrong assumption can carry forward, and a fluent explanation does not show that the answer is right.
- Little intermediate state: simple extraction or lookup has few steps to make explicit, so extra reasoning tokens are unlikely to improve the result.
- Internally reasoning models: OpenAI’s reasoning best practices advise against explicit “think step by step” instructions for its reasoning models, because those models already reason internally. Other providers and models need their own guidance and evaluation.
Whether CoT helps is settled by comparing it with a direct prompt on the selected model and task, with the final answer checked independently of the trace.
Sources: Wei et al., Chain-of-Thought Prompting · OpenAI reasoning best practices
When does Tree of Thoughts justify its search cost over a single reasoning path?
Tree of Thoughts justifies its cost when a problem has a real search space. That means meaningful alternatives at intermediate steps, consequences that appear only later, and partial states that can be evaluated usefully. Without those, a single reasoning path or a direct call is cheaper.
How it works: ToT wraps the model in a search procedure.
- The model proposes several candidate next states, or thoughts, from the current state.
- An evaluator judges how promising each state is. The evaluator can be the model itself.
- A search strategy such as breadth-first or depth-first search keeps and expands the promising branches.
- When a branch is judged a dead end, it is abandoned and the search backtracks to an earlier state. This replaces continuing one irreversible completion.
Costs and limits:
- Work grows with branching and depth, because each expanded state needs generation and evaluation calls.
- The evaluator is the weak point. A poor evaluator can keep a bad branch or prune the one that leads to the solution.
- Straightforward extraction has no useful search tree, and plain arithmetic is cheaper with a calculator.
- The reported gains come from the tasks tested in the original paper and are not a general guarantee.
Self-consistency sits between a single path and ToT. It samples complete paths independently and votes on their final answers, without evaluating intermediate states or backtracking from them.
Sources: Yao et al., Tree of Thoughts · Wang et al., Self-Consistency
When does self-consistency help, and when can majority aggregation fail?
Self-consistency helps when independent samples tend to make different mistakes and their final answers can be normalized and compared, so the correct answer is likely to be the most common one. Majority aggregation fails when errors are correlated, so that most samples share the same mistake. It also fails when correct answers cannot be matched reliably.
How it works:
- The same prompt is sampled several times, producing different reasoning paths instead of one greedy path.
- Each path’s final answer is extracted and normalized, so
67,67.0and “sixty-seven” count as the same answer.- Equivalent answers are grouped, and the most frequent one is selected.
For example, three sampled paths for an age puzzle answer
67,67and35, and the vote returns67.Where it fails or costs too much:
- Correlated errors: if most samples repeat the same misconception, the wrong answer wins the vote.
- Open-ended outputs: two correct summaries rarely match word for word, so literal string voting splits the correct answers. A meaningful comparison rule is needed, and that rule must itself be tested.
- Cost: calls and tokens grow roughly with the number of samples. Parallel calls reduce elapsed time but still consume capacity.
A comparison against a single-path baseline on the target task shows whether the extra calls pay off.
Sources: Wang et al., Self-Consistency Improves Chain of Thought Reasoning
Which guardrails keep unsafe model output from becoming an unsafe action?
The controls that stop unsafe output from becoming an unsafe action sit between the model and the action. They are least-privilege access to tools and data, authorization enforced in application code, strict validation of structured output before dispatch, and bounds on requests and loops. They assume that a bad output or a successful injection can occur, and they limit what it can do.
- Least-privilege tools and data: only the operations and data the current task needs are exposed. An injected instruction cannot call a tool the dispatcher never exposed. Data is contained only if no model-visible path can reach it, including the prompt, conversation history, retrieved context and every exposed tool. Leaving a document out of one retrieval result is not, by itself, a confidentiality guarantee.
- Authorization in application code: the executor checks whether this caller may perform the requested operation on this resource, regardless of what the model asked for.
- Validation before dispatch: output is parsed against a strict schema of allowed fields and values, then mapped to fixed operations. If the model emits the action
refund, the schema only confirms thatrefundis an allowed action name. The permission check still decides whether this refund may happen.- Budgets and bounds: rate limits, token budgets and loop-iteration caps limit runaway or abused loops.
Classifiers, monitoring and human approval for high-impact actions such as payments or account changes are added where the potential harm justifies their cost. Prompt-injection screening can flag familiar attack wording but can miss novel, obfuscated and indirect instructions, so it cannot replace least privilege. Together these controls form a defense-in-depth baseline, not a security or compliance guarantee.
Sources: OWASP LLM Prompt Injection Prevention Cheat Sheet · OpenAI safety best practices
What is an LLM hallucination, and why does it happen?
An LLM hallucination is generated content that the available evidence does not support. It happens because a language model produces plausible continuations of text rather than checking each claim against reality, so fluent, confident output can carry unsupported claims.
Type What happens How it can be detected Intrinsic The answer contradicts the source, such as naming Dickens when the passage names Austen. Comparing the answer with the supplied context can expose the contradiction. Extrinsic The answer adds a claim the source never states, such as a plausible date or citation. The claim may even be true, but nothing supports it. The context alone cannot settle it. Another source or an abstention policy is needed. Several causes can produce the same symptom:
- Weak evidence: sparse or outdated training data gives the model little to draw on.
- Preference tuning: it can reward confident, agreeable answers over accurate ones.
- Sampling: generation can select an invented detail from several plausible continuations.
Polished prose does not reveal which cause applies. Claims are therefore checked against evidence rather than judged by how convincing they sound.
Sources: NIST AI 600-1, Generative AI Profile · Ji et al., Survey of Hallucination in Natural Language Generation · Anthropic sycophancy research
Why can retrieval-augmented generation still produce a hallucinated answer?
Retrieval supplies evidence, but it does not guarantee that the evidence is sufficient or that the generated answer stays within it. The hallucination itself is an unsupported claim in the generated answer. A retrieval defect is a separate failure, although it can leave the gap that the model then fills.
- Retrieval defect: the corpus may lack the fact, or retrieval may miss it or return weak or irrelevant passages. This is a retrieval failure, not yet a hallucination. It leads to one when the answer goes on to state claims the passages do not support instead of abstaining.
- Unsupported generation: even with good passages, the model can contradict them or add a plausible date, number or citation that they never mention.
Retrieval quality and answer faithfulness are measured separately so that a wrong answer can be traced to the stage that caused it. Separate measurement does not mean the failures are independent: a retrieval gap can be the reason for an unsupported claim, and both can affect the same answer. A faithfulness score checks only whether the answer’s claims are supported by the supplied context. It does not show that the context is true, current or complete, and a faithful answer cannot recover evidence that retrieval never supplied. Automated faithfulness judges need calibration against reviewed examples.
Sources: RAGAS faithfulness metric · RAGAS paper
How can preference tuning make an answer more agreeable without making it more factual?
Preference tuning optimizes a model toward answers that human raters or preference models score highly. When those scores reward agreeing with the user or sounding convincing, optimization can make answers more agreeable without making them more correct. This includes accepting a false premise in the question.
The mechanism:
- Raters, human or model, compare candidate responses and mark the preferred one.
- The model is optimized to produce responses that earn higher preference scores.
- When raters sometimes prefer a convincingly written agreeable answer over a correct one, that preference is reinforced along with the useful ones.
Research on sycophancy found that in the studied RLHF settings, human and preference-model judgments sometimes favored sycophantic responses over correct ones. This is a risk, not an inevitable result of every preference-tuning setup.
A higher preference score does not imply higher factual accuracy. Factual accuracy and calibration are therefore evaluated separately from preference scores, with test prompts that contain false premises so that agreement is not mistaken for quality.
Sources: Anthropic, Towards understanding sycophancy in language models
What is prompt injection, and how does indirect prompt injection differ from direct prompt injection?
Prompt injection is input that changes a language model’s behavior or output in a way the application did not intend. Direct and indirect injection differ in where that input enters the model’s context.
Aspect Direct injection Indirect injection Entry point The user’s own message to the model. External content that the model processes, such as web pages, files, retrieved documents or tool results. Who can supply it The person interacting with the system. Anyone who can influence that content, without access to the conversation. Example A chat message telling the model to ignore its instructions and reveal hidden rules. A web page with hidden instructions that make a summarizing assistant leak the conversation through a link. The input can be deliberate or accidental, because ordinary content can also conflict with instructions. Injected text does not have to be visible to people as long as the model parses it, and multimodal systems can receive it through images or other media. Its impact depends largely on what the manipulated model can reach. Access to protected context, or to tools that act, is what turns a manipulated answer into a data leak or a damaging action.
Why is prompt injection harder to prevent than SQL injection?
Prompt injection is harder to prevent because SQL has a code/data boundary that the database enforces, and language models have no established equivalent. A model receives its instructions and untrusted text through the same natural-language input, so untrusted text can still act as instructions.
Aspect SQL injection Prompt injection How code and data reach the interpreter A parameterized query defines the SQL code first and passes values separately. The database treats each bound value as data, whatever it contains. System instructions, user messages, retrieved documents and tool results all enter the same model context. No mechanism forces the model to treat any part of that context only as data. Effect of marking the data Binding a value stops it from changing the query’s structure. Delimiters, labels on untrusted content and input filters lower the risk. The model may still follow instructions inside the marked content, and new phrasings can slip past filters. Resulting defense Prevention: a correctly bound value cannot become SQL code. Containment: bypass is assumed possible, so controls outside the model limit what a manipulated model can do. These controls include least-privilege tools, validation of model output, authorization of each consequential action and human approval for high-risk actions. OWASP’s 2025 guidance states that it is unclear whether foolproof prevention methods for prompt injection exist. That statement describes current techniques. It does not prove that a stronger boundary can never be built.
OWASP SQL Injection Prevention Cheat Sheet · OWASP LLM01:2025 Prompt Injection
Why should a system prompt not be treated as a secret or authorization boundary?
A system prompt can be disclosed, and the model follows its instructions only probabilistically. It therefore cannot keep secrets or reliably decide what a caller is allowed to do.
- Not a secret: adversarial queries can reveal the prompt. Any API keys, database credentials or other sensitive data placed in it can leak with it. Revealed internal rules, filtering criteria or role descriptions can also help an attacker plan further attacks.
- Not an authorization boundary: an instruction such as “only administrators may delete records” does not stop a manipulated or mistaken model from requesting a deletion. Enforcement belongs in deterministic code or the downstream service, which checks each requested action against the caller’s actual permissions.
Secure designs keep credentials and authorization decisions outside the prompt and assume that the prompt may be exposed. That assumption does not make every prompt harmless to publish. It means that the system stays protected if the prompt leaks.
OWASP LLM07:2025 System Prompt Leakage · OWASP LLM06:2025 Excessive Agency
How do security failures and reliability failures differ, and why does the distinction matter?
A security failure breaks a protection boundary: data is disclosed, or an access or action happens that should not be allowed. An attacker can cause it, and so can an ordinary model mistake. A reliability failure is incorrect or unsupported output, and it can occur with no attacker at all. The distinction matters because each type of failure is limited by different controls, and neither set of controls replaces the other.
Aspect Security failure Reliability failure What goes wrong Unauthorized disclosure, access or action. An incorrect, unsupported or confabulated answer. Typical cause Prompt injection or other hostile input, or a model error in a system that gives the model too much authority. Generation itself; no adversary is needed. Controls Least privilege, authorization enforced outside the model, and isolation. They limit the damage whatever produced the bad request. Grounding in evidence, checking claims against sources, and evaluating answer quality. What those controls do not fix A tightly permissioned system can still give a wrong answer. Grounding and claim checks do not stop a manipulated or mistaken request from running a privileged action. One incident can be both. A hallucinated tool call is a reliability failure. It also becomes a security incident when an overprivileged agent executes it and the result is unauthorized disclosure or access, or a damaging action. If the agent lacked permission for that action, enforcement outside the model would refuse the call, and the error would stay a reliability failure.
Why are prompt-level guardrails insufficient on their own?
Prompt-level guardrails are insufficient on their own because a prompt only influences what the model generates. The prompt cannot enforce anything at the point where that output is used. The model also follows instructions probabilistically, so prompt injection or an ordinary model error can make it ignore a rule such as “never call
delete”.Enforcement therefore happens in deterministic code at the points where model output takes effect:
- Output handling: the code that consumes model output treats it as untrusted input. It validates or encodes the output for its destination, for example with parameterized SQL, context-aware HTML encoding or schema checks on tool arguments.
- Authorization: application code or the downstream service checks each requested action against the caller’s permissions. The model does not make that decision.
- Impact limits: least-privilege tools and isolation such as sandboxing limit the damage when a bad request passes the other checks.
- Human approval: reserved for actions whose consequences justify the delay, such as irreversible operations.
Prompt instructions still help by guiding normal behavior and reducing mistakes. They are a useful layer, not an enforcement mechanism.
OWASP LLM05:2025 Improper Output Handling · OWASP LLM06:2025 Excessive Agency
What is the difference between data drift and concept drift?
Data drift, in its narrow sense, is a change in the input distribution P(X). Concept drift is a change in the relationship between inputs and outcomes, P(Y|X). After concept drift, the same input tends to have a different correct answer.
- What changes: with input (covariate) drift, the mix of feature values shifts, for example when a new user segment with different demographics arrives. With concept drift, the mapping from inputs to outcomes changes, for example when spam patterns change as spammers adapt.
- Effect on the model: input drift may or may not reduce quality. It matters when the shift reaches features the model relies on or regions where the model generalizes poorly. Concept drift means the learned mapping is out of date, so predictions can be wrong even for inputs that look familiar.
- Evidence needed: input drift can be measured from unlabeled production inputs by comparing feature distributions with a baseline. Confirming concept drift needs outcome labels, because unlabeled inputs cannot show whether the relationship between inputs and outcomes has changed.
The term “data drift” is used inconsistently, sometimes for any change in distribution. Stating which distribution is monitored, such as P(X) or P(Y|X), avoids that ambiguity. An input-distribution alert neither proves that model quality fell nor rules out concept drift. Recent labeled outcomes, which often arrive late, are the stronger check.
What does ROC-AUC show about a classifier?
ROC-AUC shows how well a binary classifier’s scores rank positive cases above negative cases, summarized across all decision thresholds.
The ROC curve comes from sweeping the threshold from strict to permissive. Each threshold gives one point on the curve. The point plots the true-positive rate (the share of positives flagged) against the false-positive rate (the share of negatives wrongly flagged).
The area under this curve has a direct ranking meaning. An AUC of 0.8 means that a randomly chosen positive gets a higher score than a randomly chosen negative about 80% of the time, with ties counted as half. Random ranking gives about 0.5, and perfect ranking gives 1.0.
Because it summarizes ranking over every threshold, ROC-AUC does not:
- choose an operating threshold: that choice still depends on error costs, review capacity or a required precision or recall;
- show calibration: a model with a high AUC can give a case a score of 0.9 without that score meaning a 90% probability of the event.
scikit-learn: ROC and model evaluation · Hanley and McNeil: The meaning and use of the area under a ROC curve
When is PR-AUC especially useful alongside ROC-AUC?
PR-AUC is especially useful when positives are rare and the system acts on positive predictions, as with fraud alerts or review queues. ROC-AUC still describes overall ranking. PR-AUC adds whether the predicted positives are mostly correct, which ROC-AUC can hide.
- Why ROC-AUC can look strong: the false-positive rate divides false positives by all negatives. With many negatives, even a large number of false alarms gives a small rate. For example, with 100 positives and 100,000 negatives, catching 80 positives while flagging 1,000 negatives gives a false-positive rate of 1%, yet only 80 of the 1,080 alerts, about 7%, are correct.
- What PR shows: precision is the share of alerts that are correct, and recall is the share of positives found. Both measures focus on the positive class, so false alarms that consume review capacity stay visible.
Two qualifications apply when reading a PR-AUC value:
- The baseline depends on prevalence: a random ranker’s expected precision equals the positive rate. A PR-AUC value is therefore meaningful only next to that rate. Comparing values across datasets with different positive rates mixes model quality with dataset composition. The evaluation set needs to match the production population and workload.
- Implementations differ: average precision, trapezoidal areas and interpolated areas under the PR curve can give different values. A reported PR-AUC is comparable only when the calculation method is known.
Davis and Goadrich: The Relationship Between Precision-Recall and ROC Curves · scikit-learn: average_precision_score
What is the difference between supervised and unsupervised learning?
Supervised learning trains on inputs paired with known targets and learns to predict those targets. Unsupervised learning trains on inputs without targets. It finds structure defined by a chosen objective, such as groups of similar examples.
Aspect Supervised learning Unsupervised learning Training data Examples with labels, such as tickets paired with their owning team or houses paired with their sale price. Examples without labels, such as merchant transaction histories. What training optimizes Error between predictions and the known targets. An objective that defines structure, for example how close examples are to their cluster centers in k-means, or how much variance PCA keeps. Typical tasks Classification, regression and ranking. Clustering, dimensionality reduction and anomaly detection. How results are evaluated Directly, by comparing predictions with held-out labels the model did not see during training. Training gives no answers to compare against, and a good objective score is only a proxy. Usefulness needs external evidence, such as whether the structure explains different behavior or improves a real decision. That evidence can be label-free or use labels collected only for evaluation. Common failure Label errors, held-out data that does not represent production, or distribution shift. Structure that is mathematically tidy but useless. Feature choice, scaling and random initialization can change the result substantially. A cluster is therefore not a discovered truth. It is the output of a chosen objective and feature space.
How do supervised, self-supervised, and reinforcement learning differ in their learning signal, even when all three use neural networks?
The three paradigms differ in where the training signal comes from. Supervised learning uses a target supplied for each example. Self-supervised learning derives targets automatically from the input data. Reinforcement learning uses rewards that an environment returns in response to actions, and it optimizes expected cumulative reward.
Paradigm Learning signal Example What training optimizes Supervised A label for each example, provided by people or by another system. Support tickets paired with the team that owns them. Agreement between predictions and the given labels. Self-supervised A proxy target derived from the data itself, with no human annotation. Predicting the next token, or a hidden one, in text. Performance on the proxy task, usually to learn representations that are reused for other tasks. Reinforcement A reward from the environment after an action. The reward may arrive immediately or only after later actions. A routing policy rewarded when tickets get resolved. A policy with high expected cumulative reward (return) over time. The architecture does not determine the paradigm: similar neural-network components can be trained with any of these signals. The signal determines what training can optimize and how the result is evaluated. Taxonomies differ on where self-supervised learning belongs. It is sometimes described as turning an unsupervised problem into a supervised one by creating surrogate labels from unlabeled data.
Google Machine Learning Glossary · OpenAI Spinning Up: Key Concepts in RL
How are symmetric and asymmetric cryptography used together in a real system?
Public-key (asymmetric) cryptography handles the small jobs that establish trust: authenticating a party with a signature, and establishing or protecting a shared key. Symmetric authenticated encryption then protects the bulk data with that key, because it is efficient and detects tampering as well as hiding content. Real systems combine the two instead of choosing one family for everything.
TLS 1.3 with certificate authentication:
- Client and server run an ephemeral (EC)DHE key agreement. Both sides derive fresh shared secrets without ever sending the key itself.
- The server signs the handshake transcript with its certificate’s private key. The client validates the certificate and checks the signature with the certificate’s public key, which authenticates the exchange as coming from that server.
- Keys derived from the shared secrets drive a symmetric AEAD cipher, such as AES-GCM, that protects all application data.
In this handshake, the certificate key only signs. It never encrypts a session key.
Envelope encryption for stored data:
- A random data-encryption key (DEK) encrypts the payload with symmetric authenticated encryption.
- A key-encryption key held in a key service, or a recipient’s public key, encrypts (wraps) the DEK.
- The wrapped DEK is stored with the ciphertext. Decryption first unwraps the DEK, then decrypts the payload.
Rotating the key-encryption key or adding a recipient then rewraps only the small DEK instead of re-encrypting the bulk data.
Public-key encryption suits only small payloads such as keys, so it is not applied to bulk data directly. Combining the families also does not remove key management. Public keys still need authentication, for example through certificates, and private keys and key-encryption keys still need protection.
Sources: NIST SP 800-57 Part 1 Rev. 5: Recommendation for Key Management · RFC 8446: TLS 1.3
What is a cryptographic hash function, and what security properties does it provide?
A cryptographic hash function maps input of any length to a fixed-length digest, such as 256 bits for SHA-256. It uses no key and has no decryption operation. It is designed so that the digest can represent the data in comparisons and integrity checks:
- Deterministic: the same bytes always produce the same digest, so anyone can recompute and compare it.
- Preimage resistance: given a digest, finding any input that produces it is computationally infeasible.
- Second-preimage resistance: given one input, finding a different input with the same digest is infeasible, so a known file cannot be replaced by a different file with a matching digest.
- Collision resistance: finding any two different inputs with the same digest is infeasible. Signatures and attacker-facing file identities rely on this property. An attacker who could produce such a pair could get one input approved and then substitute the other.
These properties do not hide a value drawn from a small or predictable set. Passwords, phone numbers or IDs can be recovered by hashing likely candidates and comparing digests. Once practical collisions are found, as they have been for MD5 and SHA-1, the function no longer suits uses that depend on collision resistance, such as signatures.
Why is SHA-256 alone unsuitable for password storage?
SHA-256 is unsuitable because it is designed to be fast. An attacker who steals the stored digests can test password guesses offline at very high speed, and nothing in SHA-256 makes each guess costly. Without a salt, identical passwords also produce identical digests. Precomputed tables, and every digest the attacker cracks, therefore apply to every account with the same password.
Password storage instead uses a purpose-built password-hashing function:
- Deliberately expensive, adjustable cost: Argon2id, scrypt or PBKDF2 make each guess slow. Argon2id and scrypt also need substantial memory per guess, which raises the cost of cracking on massively parallel hardware. The cost parameters are tuned to current production hardware and raised over time.
- A unique random salt for each stored password: the salt is stored with the hash, so equal passwords produce different hashes and precomputation cannot be reused across accounts.
These measures raise the cost of every guess. They do not make a weak or common password safe, because it can still be guessed, only more slowly.
Sources: OWASP Password Storage Cheat Sheet
How do a hash, an HMAC, and a digital signature differ?
All three produce a value that detects changes to a message. They differ in their keys, and therefore in who can create a valid value and who can check it.
Aspect Hash, such as SHA-256 HMAC Digital signature Key None. One secret key shared by everyone who creates or verifies tags. A private signing key and a matching public verification key. Who can create a valid value Anyone, including an attacker who changes the message and recomputes the digest. Any holder of the shared key. Only the holder of the private key. Who can verify Anyone who has a trusted expected digest. Only holders of the shared key, who could also have created the tag. Anyone who has the public key. What a successful check shows The data matches the expected digest. This stops deliberate tampering only if the expected digest arrived through an authenticated channel. The message was not altered and came from some holder of the key, but not from one identifiable holder. The message was not altered and was signed with that private key. Linking the key to a person or organization depends on a trusted key binding, such as a certificate. Typical use Fingerprints, download checks against accidental corruption, and a building block for HMAC and signatures. Webhook signatures, HS256 JWTs and messages between parties that already share a secret. Software and certificate signing, signed tokens and messages that many parties must verify. A signature does not prove non-repudiation by itself either. That depends on how the private key is held, how a compromise is handled and what audit evidence connects the key to an act.
Sources: NIST FIPS 180-4: Secure Hash Standard · NIST FIPS 198-1: The Keyed-Hash Message Authentication Code (HMAC) · NIST FIPS 186-5: Digital Signature Standard
What does a salt protect against, and why must it be unique per user?
A salt protects against attacks that reuse one computation across many stored password hashes. With a unique random salt for each stored password, the same password produces different hashes. As a result, precomputed lookup or rainbow tables do not apply, cracking one hash does not reveal every account with that password, and the stored hashes do not show which accounts share a password.
That protection depends on each salt being unique:
- Shared salt: equal passwords again produce equal hashes. Each guess, hashed once with the shared salt, can be checked against every account at once, and a table built for that salt covers the whole database.
- Unique salt: each guess must be hashed separately for each account, so attacking many accounts requires many separate efforts.
The salt is not secret and is stored next to the hash. It does not slow down guessing against one stolen hash, so it does not protect a weak password by itself. That per-guess cost comes from a deliberately slow password-hashing function such as Argon2id.
Sources: OWASP Password Storage Cheat Sheet
What is the difference between hashing, encryption, and encoding?
Encoding changes how data is represented, encryption hides data from anyone without the key, and hashing produces a fixed-size fingerprint that cannot be turned back into the data. Only encryption both protects a secret and allows authorized recovery.
Aspect Encoding, such as Base64 Encryption, such as AES-GCM Hashing, such as SHA-256 Purpose Represents bytes in a form a channel or syntax accepts, such as an HTTP header. Keeps data confidential. Authenticated encryption also detects tampering. Produces a fingerprint for comparison and integrity checks, and serves as a building block for HMAC, signatures and password hashing. Reversal Anyone who knows the encoding can decode the value. Only a holder of the decryption key can recover the plaintext. There is no decryption operation. A guessable input can only be found by hashing candidates and comparing digests. Secret involved None. A key. Whoever can obtain the key can read the data. None for a plain hash. Security provided None. The encoded value is the original data in another form. Confidentiality against anyone without the key. Detects changes when the expected digest is trusted. It does not hide inputs from a small or predictable set. Choosing the wrong operation causes predictable failures. A Base64-encoded password, as in HTTP Basic authentication, is readable by anyone who sees it. Data that must be read later is lost if it is only hashed.
Sources: NIST FIPS 180-4: Secure Hash Standard · NIST SP 800-57 Part 1 Rev. 5: Recommendation for Key Management · RFC 7617: The ‘Basic’ HTTP Authentication Scheme
How should a .NET application prevent SQL injection?
A .NET application prevents SQL injection by keeping untrusted input out of the SQL text. Values are bound as parameters, and any dynamic part of the query structure is chosen from a fixed allowlist.
Why parameters work: when input is concatenated into a query, the database parses it as SQL. An email value such as
x' OR '1'='1then closes the string literal and adds a condition that is always true. A parameterized query contains only a placeholder, and the value is bound to it as data. Whatever characters the value contains, it cannot close a literal or add a clause.// Unsafe: the input becomes part of the SQL text context.Users.FromSqlRaw("SELECT * FROM Users WHERE Email = '" + email + "'"); // Safe (EF Core 7.0+): the interpolated value becomes a DbParameter context.Users.FromSql($"SELECT * FROM Users WHERE Email = {email}");
- Library calls: an ORM or data-access helper protects only when it actually creates parameters. The two calls above look similar, but only
FromSqlturnsFromSqlRawsends the concatenated text as written.- Identifiers and structure: parameters represent values, not table names, column names or sort direction. A dynamic sort column is therefore mapped from user input to known SQL through a closed allowlist, such as
"name"→Name, and any other input is rejected.- Least privilege: a database account with narrow permissions limits what injected SQL can do, but the injected SQL still runs. It reduces impact and does not replace parameterization.
OWASP: SQL Injection Prevention Cheat Sheet · Microsoft: SQL queries in EF Core
How should a service store and verify user passwords?
A service stores each password as a verifier produced by a slow, salted password-hashing function. It never stores plaintext, reversibly encrypted text or a fast general-purpose hash. The service only needs to check a password, not recover it, and the function’s deliberate cost makes each guess against a stolen table expensive.
Storing: OWASP currently recommends Argon2id with at least 19 MiB of memory, 2 iterations and a parallelism of 1. That configuration is a floor, and the cost is raised as far as the authentication servers can sustain. When Argon2id is unavailable, scrypt is the next option, and PBKDF2 fits systems that require FIPS-140 compliance. A maintained library generates a unique random salt for every password. It encodes the algorithm, its cost parameters, the salt and the derived output in one record:
$argon2id$v=19$m=19456,t=2,p=1$<salt>$<output>Verifying a login:
- The algorithm, parameters and salt are read from the stored record.
- The same function runs on the submitted password with those stored settings.
- The library’s constant-time verification routine compares the result with the stored output.
- After a successful match, the password is rehashed with current settings if the record uses an older algorithm or a lower cost.
Because each record carries its own settings, old and new records can be verified side by side, and accounts move to stronger settings as their users sign in. Accounts that never sign in again keep the old verifier, so an expiry or forced-reset policy is needed for them.
In password storage, what protection does a pepper add, and what does it cost?
A pepper is a secret key that is applied in addition to each password’s salt. It is shared by all password records and stored outside the database. It protects against theft of the database alone: without the key, a stolen verifier cannot be tested against guessed passwords. In exchange, the system must protect and keep available one more critical secret, and rotation becomes harder. The pepper also does not replace a slow password-hashing function.
How it protects: a salt is stored next to its verifier, so a thief who copies the table has everything needed to hash a guess and compare the result. A pepper is kept elsewhere, such as in a secrets vault or an HSM, and serves as the key for an extra step after password hashing. OWASP describes hashing the password as usual and then applying HMAC to the result, with the pepper as the key. NIST describes that extra secret-key step as either a keyed hash or an encryption operation. Without the key, a thief cannot compute the stored value for any guess.
What it costs:
- A critical secret: every login needs the pepper. If it leaks along with the table, for example when the application host is compromised, the extra protection is gone. If it is lost, no existing verifier can be checked, and every user must reset their password.
- Harder rotation: how existing records move to a new pepper depends on the construction. With a one-way keyed hash such as HMAC, a record can be rebuilt only from the password: at the user’s next successful login or through a reset. With reversible encryption, stored values can instead be decrypted with the old key, while it is still available, and re-encrypted under the new key. Re-encryption changes only the records currently stored. A copy of the table already stolen together with the compromised key remains open to offline guessing.
- No substitute for the hashing function: once the pepper leaks, only the cost of the salted password-hashing function slows offline guessing. A fast hash combined with a pepper therefore remains weak.
OWASP: Password Storage Cheat Sheet · NIST SP 800-63B-4: Password verifiers
What should happen when a secret is accidentally committed and pushed?
A pushed secret is treated as compromised. The response replaces and revokes the credential at its issuer, confirms that the old value no longer works, and checks whether it was used. Cleaning the repository comes afterward and is not a fix by itself, because copies that already exist cannot be recalled.
- Assume exposure: once pushed, the value may already exist in clones, forks, caches, CI logs and notifications. Deleting it from Git does not make it secret again.
- Rotate and revoke: a replacement credential is issued, every service that uses the secret moves to it, and the old value is revoked where it is accepted. Until revocation, anyone holding a copy can still use the old value. Revoking before consumers move stops misuse sooner but breaks those consumers, so the window is kept as short as the rollout allows.
- Verify: an attempt with the old value is rejected, and the dependent services work with the new one.
- Investigate: the issuer’s access or audit logs are reviewed for use of the credential while it was exposed.
- Clean up the repository: removing the value from current files, and optionally rewriting history, reduces future discovery. Neither step revokes the credential or recalls copies already made.
The common failure is treating deletion as remediation. A later commit that deletes the line leaves the value in history, and the credential stays valid wherever it was copied.
How do authentication and authorization work together when handling a request?
Authentication runs first and establishes who is calling. It validates a credential, such as a cookie or bearer token, and produces a principal with claims. Authorization then uses that principal to decide whether this caller may perform this action on this resource in the current context. A valid identity is an input to the authorization decision, not permission for every operation.
In ASP.NET Core, one request flows as follows:
- Authenticate: the handler for the configured scheme validates the credential and sets
HttpContext.Userto the resulting principal. This step does not reject a request that lacks a valid credential. Such a request continues without an authenticated identity.- Authorize the endpoint: policies on the endpoint are evaluated against the principal before the action runs. Examples are requiring an authenticated user, a role or a claim.
- Authorize the resource: an endpoint policy cannot see a record that the action has not loaded yet. When the decision depends on the record, such as whether the caller owns invoice 42, the action loads it and calls
IAuthorizationService.AuthorizeAsync(User, invoice, "CanReadInvoice").- Handle a denial: an unauthenticated caller receives a challenge, which asks for credentials. An authenticated caller without permission receives a forbid. The scheme decides the actual response:
- JWT bearer authentication returns 401 for a challenge and 403 for a forbid.
- Cookie authentication returns 401 or 403 to XHR requests and, from ASP.NET Core 10, to recognized API endpoints by default.
- Other cookie-authenticated requests are redirected to a sign-in page or an access-denied page.
Microsoft: ASP.NET Core authentication · Microsoft: Resource-based authorization · Microsoft: .NET 10 cookie authentication change for API endpoints
Why is Basic Auth unsafe over HTTP?
Basic Auth sends a reusable username/password value encoded with Base64; Base64 is reversible and provides no confidentiality. Without TLS, an observer on the path can recover credentials and replay them. TLS protects only up to its termination point, so logs and trusted intermediaries remain separate exposure boundaries. RFC 7617: Basic authentication
When is Basic Auth acceptable in production?
Basic Auth is acceptable in production only for a narrow, controlled integration that needs it. A typical case is a legacy or machine-to-machine interface whose protocol requires Basic. Its coarse credential lifecycle must be acceptable, and the credential must be protected on every hop.
Basic itself provides none of this protection, so the deployment must supply it:
- TLS on every hop: without it, anyone on the path can read the Base64 credential.
- A distinct credential per client: one client can be disabled without rotating every other client.
- Narrow authorization: the credential can perform only the operations its integration needs.
- Rotation and revocation: the password is long-lived and reusable. It is therefore kept in a secret store and replaced on a schedule and after any suspected exposure.
- Redaction and abuse controls: the
Authorizationheader stays out of logs and traces, and repeated failed attempts are rate-limited.Basic is a poor default for interactive login and broad external APIs. Every request presents the primary secret, and revocation means disabling the account or changing its password. Workload identity, mutual TLS or OAuth client authentication usually give a better credential lifecycle. Being on an internal network does not make Basic safe either.
What is the difference between role-based and resource-based authorization?
Role-based authorization decides whether a principal holds a broad capability, such as being an editor. Resource-based authorization decides whether the principal may perform an operation on one specific resource, using facts from that resource such as its owner, tenant or current state.
Aspect Role-based Resource-based Question answered Whether the principal may perform this kind of operation at all. Whether the principal may perform the operation on this particular instance. Facts evaluated The principal’s assigned roles. A conventional role check does not need the target object. Data from the loaded resource, such as owner, tenant, project or classification, compared with the principal’s identity and claims. When the check can run Before the action executes, for example declaratively with [Authorize(Roles = "Editor")].Only after the resource is loaded. The code passes the resource to IAuthorizationService.AuthorizeAsync(User, document, "DocumentOwner")before returning data or applying a change.Limit A global role cannot express ownership or tenancy of one object. An Editorrole alone lets every editor edit every document.The resource must be loaded before the decision, and every code path that reads or changes the instance needs the check. The two work together. A role check can act as a coarse early gate, and the resource check makes the final decision for the instance. Resource-based authorization is an evaluation shape rather than a separate model: the handler that receives the resource can apply role, attribute or relationship rules to it. When permission depends on the instance but only a role check runs, an authenticated user can change an identifier and reach another owner’s or tenant’s object. This flaw is called an insecure direct object reference.
Sources: Microsoft Learn: Resource-based authorization in ASP.NET Core · Microsoft Learn: Role-based authorization in ASP.NET Core
No questions in this topic yet.
What separates continuous delivery from continuous deployment?
They differ only in the final step to production. Continuous delivery keeps every change that passes the pipeline releasable, but a separate authorized decision releases it. Continuous deployment has no routine approval step: each change that passes all required gates is released to production automatically.
Both build on continuous integration, in which small changes are merged frequently and each one is built and verified automatically.
Aspect Continuous delivery Continuous deployment Production release decision A person or business process approves the release of an already verified artifact. The pipeline releases the artifact as soon as the required gates pass. Release timing Verified changes can wait, so a release can follow business timing such as a planned launch. Each qualifying change reaches production shortly after it is verified, without waiting for an approver. Role of automated gates They show that an artifact is ready to release; the approval adds the decision to release it. They make the whole release decision, so their evidence must be trustworthy enough to stand in for an approver. The gates are broader than tests. They can include security and policy checks, environment checks, and health checks during a progressive rollout.
AWS: Continuous integration and continuous delivery/deployment
What is the difference between a Pod, Deployment, and Service?
They separate three concerns. A Pod runs containers. A Deployment declares the desired set of Pods, drives the cluster toward it and rolls out changes to it. A Service gives clients a stable way to reach the Pods behind it while those Pods change.
Object Responsibility Identity and lifetime Pod The smallest unit Kubernetes schedules onto a node: one or more containers that share networking and can share storage. Replaceable. A replacement is a new Pod, usually with a different IP, so Pod addresses are not durable identity. Deployment Declares how many Pods should run from a Pod template and reconciles actual state toward that count through a ReplicaSet. A template change rolls out through a new ReplicaSet. A long-lived declaration; the individual Pods it manages come and go. Service (typical selector-based ClusterIP) Selects Pods by label, puts a stable name and virtual IP in front of them and distributes traffic among them. Stays the same while its set of backend Pods changes. How they work together: a Deployment with
replicas: 3and the Pod labelapp: myapideclares three desired replicas. If one Pod is deleted or its node fails, the ReplicaSet creates a replacement Pod, usually with a new IP, to converge back toward three. The count is a target, not a guarantee: unschedulable Pods, insufficient quota, image pull errors or failing readiness checks can leave fewer Pods ready to serve. A Service that selectsapp: myapikeeps its name and address throughout. Normally only Pods that pass their readiness check receive Service traffic, so the replacement starts receiving requests once it becomes ready. Clients never have to track Pod IPs.A Deployment never routes traffic, and a Service never creates Pods. In this selector-based setup, they are connected only through the labels on the Pods. A Service without a selector relies on separately managed endpoints, and a headless Service has no virtual IP.
Kubernetes: Pods · Kubernetes: Deployments · Kubernetes: Service API
How do metrics, logs, and traces answer different observability questions?
Each signal answers a different question. Metrics show whether the system is healthy and how large a problem is. Logs record exactly what happened at a point in time. Traces show where one operation spent its time or failed as it crossed services.
Signal What it records Question it answers Main limit Metrics Numeric measurements aggregated over time, such as request rate, error count or a latency distribution. Is something wrong, since when, and how widespread is it? Aggregation discards per-request detail, and label values must stay bounded. Logs Discrete, timestamped events, ideally with structured fields that describe state or errors. What exact error or state change occurred? High volume, sensitive data and inconsistent fields make them costly and hard to query. Traces A tree of timed spans for one operation, linked by a trace ID that is propagated across process boundaries. Which step or dependency made this request slow or fail? Every boundary must propagate the trace context, and sampling can drop the trace that matters. Correlation turns a symptom into an explanation. A metric alert identifies the affected time window and scale. A trace from that window shows the failing or slow dependency. Logs that carry the same trace ID supply the application state that the spans do not contain. Each signal still needs its own retention and access policy.
Prometheus: Instrumentation · OpenTelemetry Logs data model · OpenTelemetry traces · W3C Trace Context
Why should request or user IDs not be metric labels?
Each distinct combination of label values creates a separate time series. Request IDs take a new value for every request and user IDs one for every user, so the number of series grows with traffic instead of staying bounded. Memory, storage and query cost grow with it.
For example, an
http_requests_totalcounter labeled byrouteandstatushas one series per route and status pair, such as one that counts all failed/checkoutrequests. Adding arequest_idlabel creates a new series for every request, each used once. Even a simple total then has to be computed across an ever-growing number of series.Metric labels suit bounded dimensions such as service, route template, method and status class, where each value is worth aggregating. Per-request identifiers belong in logs or traces, which record individual events. Trace IDs connect those records to each other, so an investigation can move from a metric spike in a time window to the matching traces and logs. That link does not grant access: the logs and traces still need their own access controls.
The exact cost of high cardinality depends on the metrics backend, but an unbounded label sets no upper limit on the number of series.
Prometheus: Do not overuse labels · OpenTelemetry Logs data model · W3C Trace Context
What are the main deployment strategies, and how do they differ?
Deployment strategies differ in how the new version replaces the old one and how quickly users are exposed to it. Recreate stops the old version before starting the new one. Rolling replaces instances in batches. Blue-green switches traffic to a fully prepared copy. Canary sends a small share of traffic to the new version first. They trade spare capacity and rollout complexity against downtime, the number of users a defect can reach, and how fast a release can be reversed.
Strategy Mechanism User exposure Main cost Reversal Recreate All old instances stop, then the new ones start. Downtime, then every user gets the new version at once. An outage during each release. Redeploying the old artifact the same way. Rolling Instances are replaced in batches while the rest keep serving. Grows batch by batch; old and new versions usually serve side by side during the rollout. Compatibility between adjacent versions while they overlap, and capacity that dips or surges during the rollout. Rolling the previous version back through the fleet. Blue-green A complete new environment runs and is tested beside the live one, then traffic switches to it. All traffic moves at the switch. Near-duplicate capacity while the decision is open. Switching traffic back while the old environment still runs. Canary A small representative share of traffic goes to the new version, and exposure expands as health evidence allows. Starts with a small cohort and grows in steps. Fine-grained traffic routing and telemetry separated by version. Removing the canary from traffic. Kubernetes Deployments offer the first two directly as the
RecreateandRollingUpdatestrategies. For rolling updates,maxUnavailablelimits how many desired Pods may be unavailable, andmaxSurgeallows temporary extra Pods. These settings and the replica count shape how much the old and new versions overlap.Rolling, blue-green and canary commonly involve a period in which old and new versions run at the same time. Whenever they coexist, shared schemas, messages and data must work for both. If the new version writes data the old one cannot read, moving traffic back does not restore the previous system.
When does canary fit better than blue-green deployment, and what does each approach cost?
Canary fits better when a small, representative share of production traffic can be routed to the new version and its health measured before exposure grows. Blue-green fits better when a complete candidate environment can be tested before traffic is switched to it. Canary costs traffic control and reliable per-cohort telemetry; blue-green costs near-duplicate capacity plus a safe cutover and switch-back path.
Aspect Canary Blue-green How the release proceeds A small cohort gets the new version first, and exposure expands only while its health evidence stays acceptable. The candidate runs and is tested beside the live environment, then traffic moves to it. Direct exposure to the new version Only the current cohort’s requests run the new version until the next expansion step, so a missed defect directly reaches only that cohort. With a single switch, all requests are routed to the new version at once, so a missed defect can affect any request that exercises the defective code path. Main cost Routing a controlled share of traffic, telemetry separated by version, and a phase in which both versions serve live traffic. Close to a second full environment while the decision is open, plus a reliable traffic switch. Reversal Removing the canary from traffic. Switching traffic back, which works only while the old environment is still running and compatible. Routing limits direct exposure, not total impact. The canary and the old version can share databases, caches, queues and downstream services. If the canary corrupts shared data or overloads a shared dependency, requests still served by the old version can fail too. A small cohort therefore contains a defect only as far as its failure domain is isolated.
A canary is only as useful as its evidence. Expanding exposure needs a metric, a threshold and an observation window long enough to show the failure that matters. A fixed pause is not evidence by itself, and very low traffic can leave a small cohort inconclusive.
Neither strategy removes compatibility requirements. While both versions exist, shared schemas, messages and external side effects must work for both. If the new version writes data the old one cannot read, moving traffic back does not restore the previous system.
AWS: Deployment methods · Google SRE Workbook: Canarying releases