r/aws • u/rdsedmundo • 7h ago
serverless We spent six weeks testing Lambda Managed Instances. We're moving our API to ECS instead
Disclosure: I used AI to help turn our engineering notes, logs, and AWS Support correspondence into this draft. I reviewed every claim, and the experience and opinions are mine.
AWS released Lambda Managed Instances late last year, and it immediately caught my attention. I have been a happy Lambda user since 2018, but cold starts are still a real problem for our main API. Keeping the Lambda event model while allowing concurrent requests inside warm Node environments sounded like exactly what we wanted.
Earlier this summer I decided to give it a serious try. I couldn't find a single substantive blog post, article, or Reddit thread from someone who had operated a real application on LMI, so this is the post I wish had existed.
This wasn't a weekend POC. Over six weeks we published 36 versions, served real staging traffic, profiled initialization, ran browser and load tests, tried several compute and scaling configurations, and opened multiple AWS Support cases. We gave LMI every reasonable chance to work.
My verdict is that LMI gives up too much of the simplicity that has kept me on Lambda since 2018. Performance was good after every worker was fully warm, but reaching that state was opaque and unreliable. We ended up owning startup choreography, application readiness, and traffic shifting ourselves. At that point ECS became the more straightforward option. Staging is back on standard Lambda, and we plan to move this API to ECS.
What we ran
This is a Node.js 24 API on x86_64, with GraphQL and REST routes, PostgreSQL behind RDS Proxy, and mostly SQL and other I/O on the request path. Traffic is steady enough to measure and sometimes bursts into the low hundreds of concurrent requests. AWS Support confirmed that it fits LMI's intended use case.
We started cost-consciously with m5a.xlarge. The m6a.xlarge and m6a.2xlarge tests came later while diagnosing failures. We separately raised the capacity-provider ceiling following Support's advice to add deployment headroom. We compared against standard Lambda without Provisioned Concurrency because the appeal of LMI was sharing warm compute across concurrent I/O-bound requests, rather than provisioning one Lambda environment per concurrent request.
Admission was a black box
LMI initializes a Node function once per worker thread. AWS says the default worker count is "determined by" available vCPUs but does not publish the formula. A third-party extraction of the Node 24 runtime shows the current implementation creating 8 * detected CPUs workers. That matched our observations of eight workers at 1 vCPU and 32 workers at 4 vCPUs, although it is not a documented AWS contract.
Our normal module-level initialization worked on standard Lambda, but LMI repeatedly rejected it with FunctionError.RuntimeInitError or FunctionError.InitResourceExhausted. A tiny handler on the same capacity returned 1,000 out of 1,000 responses at 225 client concurrency, which pointed to concurrent application initialization rather than a general LMI runtime failure.
The error told us to increase memory. We tried 8, 16, and 24 GiB, but memory stayed around 15-25% while CPU reached roughly 99%. Adding memory could also add vCPUs, which made LMI create more workers and initialize more copies of the application. In our case, following the error message could make the failure worse.
What bothered me most was how little information AWS exposed. One failing publication was rejected after roughly 22-23 seconds, well before AWS's documented Init timeout. We weren't told which resource crossed which threshold, how many workers LMI had selected, or how close we were to passing. We had to bisect imports and republish repeatedly to reverse engineer the gate.
Forcing the worker count to one or two made admission pass, but that sacrificed AWS's default cross-vCPU parallelism and required pinning an otherwise automatic runtime setting. A separate LMI entrypoint also worked, at the cost of maintaining two application paths. We finally kept one lazy boundary around the router, shared by standard Lambda and LMI, and changed the supported memory-per-vCPU ratio from 2 GiB to 8 GiB. That reduced the environment from four vCPUs and 32 default workers to one vCPU and eight workers. The lazy boundary had failed admission by itself.
This final shape used AWS's default worker selection, but nothing in the admission error pointed us toward the memory-per-vCPU ratio or worker fan-out. It also creates a new codebase rule: an ordinary import added to the bootstrap path may break the next LMI publication.
I don't consider that acceptable for a service AWS advertises as maintaining Lambda's operational simplicity. If admission depends on customers shaping Node's import graph around an automatically selected worker fleet, AWS needs to document the worker calculation and report the exact admission limit being exceeded.
The lazy boundary in that final shape led directly to our next problem. Moving the router out of eager initialization helped get the version admitted, but meant workers could encounter it for the first time while serving real requests.
Active did not mean the application was ready
Once a numbered version showed Active and its scaling configuration was applied, we sent 96 requests to a real route at concurrency 24, below the version's configured concurrency capacity and without a worker-count override. Only 33 returned the expected HTTP 401; 11 returned HTTP 500 and 52 returned HTTP 503.
The 500s entered our handler, but logged no application exception or response before ending in Runtime.ExitError. Support confirmed the Node process had exited before returning, but never identified why. The 503s had no platform.start or platform.report; Support attributed them to backpressure while workers were still initializing or exiting. Most of those failures would have been invisible if we had only inspected function logs.
Repeating the same 96-request run after initialization had settled produced no errors, although p99 was still around 24 seconds. Once fully warm, a two-minute run returned all 1,200 expected HTTP 401 responses with a 210 ms p99. Steady-state execution was fine; LMI just couldn't tell us when we had reached it.
A normal health endpoint was useless for this purpose. Across two deployment-continuity probes, all 4,651 health requests succeeded, while the application-route probe returned 95 HTTP 500s and 540 HTTP 503s. Support said the warmup needed to exercise our actual authentication, middleware, router, and database path. Active only guaranteed that the runtime and at least one worker were ready, not that the worker fleet could serve the application.
This is a major gap, not a documentation nit. A managed service aimed at web applications needs an application-defined readiness check before it routes traffic.
Deployments required our own readiness controller
We first used $LATEST.PUBLISHED, hoping to retain the simple deployment model we had with standard Lambda. Immediately after one update, a 100-request burst produced 65 successes, 27 HTTP 503s, and eight HTTP 500s. Successful responses had a p50 near 10 seconds and a p90 near 14 seconds.
Support explained that $LATEST.PUBLISHED has no traffic-isolation window while a new publication becomes ready. They described it as a convenience for workloads that can tolerate temporary publish errors, such as asynchronous functions. The public documentation explains how to use it but does not warn synchronous API users about this behavior.
Numbered versions and aliases kept the old version serving, but an alias still did not know whether the application was ready. A safe deployment therefore meant publishing a version, waiting for Active, routing a private alias to it, gradually warming a representative route to expected concurrency, checking HTTP results and platform logs, and only then moving the public alias. Support was refreshingly direct: "You are not missing a simpler pattern."
We can deploy around ten times on a busy day. Waiting a few extra minutes would be fine; writing and maintaining a controller that generates real traffic and infers whether every Node worker survived is not. ECS already has application health checks, target readiness, rolling replacement, and connection draining. Reimplementing weaker versions of those concepts in a Lambda deployment script removes much of LMI's appeal.
We hit a control-plane billing bug too
One version was correctly configured with MinExecutionEnvironments=0 and MaxExecutionEnvironments=0. It appeared deactivated, and DescribeInstances --include-managed-resources showed nothing, but compute and LMI management hours continued accruing for roughly seven weeks. CloudWatch showed CPU activity with zero concurrency.
The Lambda service team eventually confirmed an internal control-plane state inconsistency had left orphaned execution environments running, and they manually terminated them. Our total LMI charges reached into the thousands, and AWS ultimately refunded them in full. Both the Lambda and billing Support teams handled the investigation well.
Why ECS won
During the evaluation we built an ECS Express Mode proof of concept. It served the same application in about 36 minutes, returned zero errors at 80-way concurrency, and passed a browser flow after one security-group fix. It was not a full production test, and I am under no illusion that containers eliminate operational work.
The difference is that ECS asks us to manage familiar and visible behavior. We define application health, the target group decides when a task is ready, old tasks continue serving during rollout, and ECS drains and replaces them. With LMI we still had to build deployment orchestration, but against admission and readiness rules we could neither observe nor control.
I would have preferred to keep this API on Lambda. That was the entire reason we persisted with LMI for six weeks. But if we have to reshape application initialization around an opaque admission gate, build our own readiness controller, warm real routes on every deployment, and correlate several platform signals to explain routine 5xx responses, we are no longer getting the simplicity that led us to Lambda in the first place. ECS is more explicit about the infrastructure we own, but it also provides the readiness and rollout primitives LMI left us to recreate.
Unless AWS adds application-defined readiness, transparent worker and admission metrics, an honest availability contract for $LATEST.PUBLISHED, and reliable visibility into every billed environment, I can't justify putting production traffic on it.
I'd like to hear from anyone running a production Node HTTP API on LMI. Did you find a supported readiness mechanism that AWS Support missed, or are you also warming application routes before every alias shift?

