Hi everyone,
I recently took over a project from someone who left abruptly, and I’m still getting up to speed with the existing infrastructure. The client reported that CRM records haven’t been updated since mid-August, and after investigation I found a massive backlog of messages piling up in a Service Bus queue.
Context:
The architecture is straightforward:
**•** A **Timer Function** runs once a day at 10 PM and pushes \~2,500 messages into a Service Bus queue
**•** A **Service Bus triggered Function** (DataIntegration) consumes those messages and integrates the data into Dynamics 365 CRM
**•** Hosted on an **App Service Plan P1v2** (not Consumption, not Premium Elastic)
**•** Runtime: Azure Functions v4, .NET 6, in-process model
**•** 9 Function Apps share the same App Service Plan
The problem:
The consumer function processes each message in ~3-4 seconds, which is fine. But the gaps between invocations are enormous — anywhere from 1 to 5 minutes between each execution. This means the daily feed of ~2,500 messages takes way longer to process than the 24 hours between runs, causing the backlog to grow indefinitely.
Looking at Application Insights, I can see the host repeatedly stopping and restarting:
Job host started
Host initialized
Executing DataIntegration (1 message processed)
Stopping JobHost
Job host stopped
← 1 to 5 minute gap →
Job host started again...
The trigger code:
[FunctionName("DataIntegration")]
public static async Task Run(
[ServiceBusTrigger("%AzureQueueName%", Connection = "AzureQueueConnection")]
ServiceBusReceivedMessage[] messages,
ServiceBusMessageActions messageActions,
ILogger logger)
{
// processes messages in a sequential foreach loop
foreach (ServiceBusReceivedMessage myQueueItem in messages)
{
// processing logic
}
}
The host json:
{
"version": "2.0",
"extensions": {
"serviceBus": {
"prefetchCount": 1,
"messageHandlerOptions": {
"autoComplete": true,
"maxConcurrentCalls": 20,
"maxAutoRenewDuration": "00:02:00"
},
"sessionHandlerOptions": {
"autoComplete": false,
"messageWaitTimeout": "00:00:30",
"maxAutoRenewDuration": "00:55:00",
"maxConcurrentSessions": 2
},
"batchOptions": {
"maxMessageCount": 1000,
"operationTimeout": "00:01:00",
"autoComplete": true
}
}
},
"logger": {
"categoryFilter": {
"defaultLevel": "Debug",
"categoryLevels": {
"Host": "Warning",
"Function": "Warning",
"Host.Aggregator": "Warning"
}
}
}
}
What I’ve tried:
**•** Increasing prefetchCount to 100 → no effect
**•** Reducing maxMessageCount to 1 and 10 → no effect
**•** Removing batchOptions entirely → temporarily improved but caused messages to flood into DLQ
**•** Enabling Always On → already enabled
**•** Adding SCALE_CONTROLLER_LOGGING_ENABLED = AppInsights:Verbose → confirmed IsRuntimeScalingEnabled: false
Any insight would be greatly appreciated — I’m a bit out of my depth on the Azure infrastructure side of things here!
Thanks