Lab 12 — Azure App Service Operations
azure administrator series
Lab 12 — Azure App Service Operations. Neon Azure-and-coding cover illustration, not validation evidence.
ZIP SHA-256: 525c751467e14a18c803a33a08ccbda742fa9ad4e8d2ca74104012ab20172bcb
Production returns HTTP 200 with release version 1. Staging returns HTTP 503 with release version 2. A release must not be promoted until staging is healthy.
Identify the cause using response evidence, deployed file hashes, configuration comparisons, and platform state. Recover staging without rebuilding the plan, replacing code, weakening access restrictions, or modifying production. Demonstrate that recovery is safe to repeat.
Submit the actual failing and recovered responses, the smallest justified configuration difference, proof that production stayed healthy, unchanged file hashes, and the result of a second recovery invocation. Keep identifiers and addresses private.
Production is healthy, but staging returns HTTP 503. In this workshop you compare application settings, code integrity, and platform configuration before changing anything. After recovery, practise controlled promotion and rollback, scale the hosting plan, and restore a backup into a separate disposable slot.
This lab follows the AZ-104 App Service objectives. The application deliberately produces a synthetic readiness failure; it is not an Azure infrastructure outage.
Architecture and time
The coding package contains a dependency-free Node.js 24 HTTP server, a synthetic marker, and release metadata. Production runs version 1 and staging runs version 2. Both return the release, slot label, readiness, and SHA-256 hashes. Generated cover and inline artwork are conceptual illustrations; only genuine captures and recorded responses establish validation.
Conceptual Azure-and-coding illustration: deploying a Node.js application into App Service. Not validation evidence.
One Linux S1 plan initially hosts both slots on one worker. A separate restore slot is created later on the same plan. Each application and SCM endpoint accepts only the recorded workstation /32 and denies unmatched requests. VNet integration supplies the route used for storage backup/restore; integration is outbound connectivity, not a replacement for inbound restrictions.
Suggested timing: preparation 25 minutes, deployment/coding 35, diagnosis/recovery 30, swaps 30, scaling 25, backup/restore 45, verification/cleanup 20. Azure provisioning and propagation can extend these timings.
1 Prepare and inspect
Read the README, sign in separately using Azure CLI and PowerShell, and confirm that both contexts identify the intended tenant, subscription, and operator. Run the Preflight stage. No Azure object is created in this stage.
Actual authored Bicep source: restrictions, TLS, runtime, and VNet backup routing. Source capture, not an execution result.
Review the private governance capture if the script asks for -ReviewedGovernance. That switch records a review; it does not disable a policy. Inspect the quota's returned units: Azure may return a regional instance counter rather than the documentation's core-count example. Unknown counters stop the script.
Preflight also checks dependencies, registered providers, runtime/SKU availability, permissions, address conflicts, source IPv4 stability, names, pricing, and current cost inventory. Failures do not authorize registration, grants, exemptions, a broader address rule, or a different region.
Portal checkpoint: confirm the intended subscription. Do not capture its identifier or your signed-in account for publication.
2 Build the protected foundation
Read templates/foundation.bicep. Parameters supply unique names and the recorded /32; resources declare a VNet, delegated integration subnet, S1 plan, storage, app, and protected staging module. Read the compiled ARM representation privately.
Genuine Portal baseline: Linux Standard S1 plan with one worker; identifier masked.
Genuine Portal security configuration: HTTPS, TLS, Always On, and disabled publishing credentials.
Genuine application access restriction and deny default; source address masked.
Genuine SCM restriction configuration.
Genuine Portal network configuration: restricted inbound access and separate outbound VNet integration; identifiers and addresses masked.
Run Foundation without -ReviewedWhatIf. The empty lab-owned group is created so Azure can validate and preview the group-scoped template. Review every change before rerunning with the review switch. Keep deployments incremental.
The initial application configuration includes HTTPS only, TLS 1.2 for app and SCM, Always On, disabled FTP/basic publishing, explicit deny defaults, and the same allow rule for SCM. The subnet uses 10.112.0.0/26, delegation to Microsoft.Web/serverFarms, and a Microsoft.Storage service endpoint. Storage uses LRS, TLS 1.2, HTTPS, a private container, no anonymous blob access, and only the workstation and integration subnet in its firewall.
Portal captures: plan S1/one worker, application runtime, HTTPS/TLS, access restrictions, disabled publishing credentials, VNet integration, subnet delegation, and storage network rules. Keep raw screenshots private; crop or redact identifiers deterministically without altering settings or statuses.
3 Read the code and deploy two releases
Open app/server.js. Node's standard http, fs, and crypto modules serve and hash the files. No package install or remote build is required. The startup command is node server.js; the server listens on the platform-supplied PORT.
Actual dependency-free Node.js source, including readiness and SHA-256 response logic.
const ready = process.env.LAB_RELEASE_READY === 'true';
res.writeHead(ready ? 200 : 503, { 'Content-Type': 'application/json' });Applications creates two bounded ZIP packages and deploys extracted content with Azure CLI's Microsoft Entra authentication. Run-from-package and remote build remain disabled. This matters later because custom App Service backups do not include content mounted from a ZIP package. See ZIP deployment guidance and backup limitations.
Set LAB_SLOT_NAME and LAB_RELEASE_READY as slot-specific settings. A staging release must continue identifying itself as staging after promotion changes its code placement.
Run Baseline validation. Expect production version 1 and staging version 2, HTTP 200, exact file hashes, trusted HTTPS, unchanged restrictions, and disabled basic publishing. Independent SCM inspection checks deployed files rather than trusting only the app's reported hashes. A fresh request originating from Azure SCM must receive denial at the workstation-restricted production endpoint.
Coding captures show the actual Bicep declarations, Node readiness branch, Azure CLI fault command, and PowerShell recovery code in a read-only source viewer. Execution responses are recorded separately. A source screenshot illustrates implementation; it does not prove that a command ran successfully.
4 Observe the staging discrepancy
Fault changes only staging's LAB_RELEASE_READY to false through Azure CLI. Capture the actual staging response and compare production. The intended observations are staging HTTP 503, production HTTP 200, unchanged version/file hashes, and one changed application setting.
Actual Azure CLI fault source. Its execution responses are recorded separately.
Genuine Portal setting: staging LAB_RELEASE_READY is false and slot-specific.
Do not treat any generic 503 as proof. A platform startup failure, blocked route, wrong package, or wrong hostname could produce a different symptom. Check the synthetic readiness message, file hashes, settings, and controls together.
Portal captures show staging's actual setting value. Compare the separately recorded staging HTTP 503 and production HTTP 200 responses. If browser navigation is blocked, do not label that browser error as the application fault. Never modify an image to manufacture a response or status.
5 Restore the captured setting
Run Restore-Lab12. PowerShell compares the current dictionary against the captured baseline and the single permitted fault difference. Unexpected changes block recovery. It restores the exact saved value while preserving unrelated keys.
Actual PowerShell recovery source preserves unrelated application settings.
Genuine Portal recovery setting: LAB_RELEASE_READY restored to true.
Run Recovery validation, then run recovery again. The second invocation should verify state without writing a configuration update. Both slots must retain their original code and hashes. Do not redeploy code or rebuild the plan to correct a setting discrepancy.
6 Preview promotion and roll it back
Only proceed after staging checks pass. Swap warm-up uses /health and accepts HTTP 200. The Swap stage requests preview, verifies staged content under target settings, completes the swap, verifies version 2 in production and version 1 in staging, then swaps back.
Genuine Azure Activity Log: slot configuration and completed swap succeeded. Application versions were independently verified.
Verify slot labels and readiness settings stayed with their slots. Recheck restrictions, TLS, and VNet integration. Some settings are swapped while others are attached to the slot; do not assume everything moves with the files. Consult deployment-slot behavior.
If interrupted, reconcile the exact correlated operation and both versions before any second swap. A repeat is not inherently idempotent. The bounded response checks wait for both the expected release and sticky slot label, because worker warm-up can briefly expose the previous settings. Record successful operation history separately from verified application content. Use -PauseForEvidence to hold verified checkpoints for genuine Portal captures.
7 Scale capacity and worker count
Demonstrate S1 to S2 and back, then one worker to two and back. Scale-up changes worker resources; manual scale-out changes instance count. Record the plan's returned SKU/capacity and the actual application instance inventory, then verify responses and unchanged files after each transition.
Genuine Portal scale-up checkpoint: Standard S2 with one worker.
Genuine Portal scale-out checkpoint: Standard S1 with two workers.
Worker labels are sanitized hashes, not public infrastructure identifiers. Multiple responses need not divide evenly between workers. Metric/scheduled Azure Monitor autoscale and App Service Premium automatic scaling are different mechanisms; this workshop does not claim to trigger either. See scale-up guidance and automatic scaling.
8 Back up production and restore elsewhere
Review the protected restore-slot template and its separate what-if. The target is never production. Backup/restore routing through VNet integration is configured for participating slots, and the integration subnet is allowed by the storage firewall. No trusted-service bypass is enabled.
Actual PowerShell source for backup artifact verification and the separate restore target.
Genuine Portal successful on-demand backup checkpoint.
Genuine Portal separate restore slot; production was not overwritten.
PowerShell creates an HTTPS-only container-scoped service SAS with rwdl, expiring within the run's six-hour window. Shared Key is retained solely for this lab exercise. Never print the SAS, put it in CLI arguments, or publish backup artifacts. Necessary cross-process storage uses Windows DPAPI under the private ACL.
Create one on-demand production backup and wait for full success. Privately inspect its ZIP/XML artifacts and compare the original file hashes. Restore that exact backup into the protected restore slot, reconcile its label/readiness settings, and verify version 1 and original hashes. Production and staging must remain unchanged.
The restore uses Restore-AzWebAppBackup with the explicit disposable slot. It derives the target slot and existing plan name; the source application must not be supplied as the destination. A polling error is not a reason to resubmit. Reconcile the exact terminal Activity Log operation first, retaining failed attempts as failed evidence.
The observed Linux backup ZIP used backslash-separated entry names. The verifier normalizes separators and reads only the exact fs/site/wwwroot entries in memory. A format mismatch must stop inspection, not justify skipping hash verification or creating a replacement backup. A recorded successful backup can be checked again without starting a second one.
A backup success status alone does not prove an application's recovery. The restored content and application response supply that proof. Scheduled backup setup is explanatory only. Review custom backup prerequisites before adapting this process to a real application.
9 Check TLS and understand custom domains
The Azure-provided hostname uses a platform certificate. Verify normal certificate-chain and hostname validation, a TLS 1.2 handshake, configured minimum TLS, the HTTPS-only setting, and a same-host HTTP-to-HTTPS redirect. Do not bypass certificate checks.
Guided custom-domain workflow: choose a domain you control; review tier and hostname/certificate prerequisites; create the required ownership-validation TXT and A/CNAME records; validate and add the hostname binding; obtain an eligible managed certificate or securely supply an appropriate certificate; bind it; test trust/hostname/HTTPS; monitor renewal; remove obsolete bindings and DNS records in an approved order. Ownership validation is not the same as certificate issuance. This lab changes no registrar, existing DNS, hostname binding, or certificate material.
Use Microsoft's custom-domain guide and TLS certificate guide for the selected hostname and certificate type. No live custom-domain binding is claimed.
10 Collect evidence and clean up
Export only allowlisted sanitized results. Keep raw identifiers, source addresses, tokens, keys, signed URLs, backups, and full Portal captures private. Publication requires genuine screenshots across all core stages, not just attractive conceptual art.
Genuine Portal cleanup checkpoint, supplemented by two exact-ID absence checks.
Remove recorded children before parents, delete only recorded deployment histories, then remove the empty owned group. Unexpected contents or ambiguous pending operations block deletion. Run cleanup twice and confirm absence twice. Temporary encrypted SAS material is removed; private evidence remains available for audit.
Complete package extraction and SHA-256 checks locally and again against a fresh public download after publication. Verify the visible article title, native rich content, links, ordered Workshopz card, desktop layout, and Wix mobile preview. Mobile preview is not an actual-phone test.
Run the stage scripts
$state = Join-Path $PWD 'private-lab12/state.json'
./Start-Lab12.ps1 -StatePath $state -Stage Preflight -Execute
# If the policy review gate stops, inspect the captured private governance file.
# Only after reviewing it, repeat preflight with -ReviewedGovernance.
./Start-Lab12.ps1 -StatePath $state -Stage Foundation -Execute
# The first Foundation invocation creates the empty owned group and saves what-if.
# Review the private template, parameters, validation, and what-if before continuing.
./Start-Lab12.ps1 -StatePath $state -Stage Foundation -Execute -ReviewedWhatIf
./Start-Lab12.ps1 -StatePath $state -Stage Applications -Execute
./Test-Lab12.ps1 -StatePath $state -Phase Baseline -Execute
./Invoke-Lab12Fault.ps1 -StatePath $state -Execute
./Test-Lab12.ps1 -StatePath $state -Phase Fault -Execute
./Restore-Lab12.ps1 -StatePath $state -Execute
./Test-Lab12.ps1 -StatePath $state -Phase Recovery -Execute
./Restore-Lab12.ps1 -StatePath $state -Execute # verified no-op
./Start-Lab12.ps1 -StatePath $state -Stage Swap -Execute
./Start-Lab12.ps1 -StatePath $state -Stage Scaling -Execute
./Start-Lab12.ps1 -StatePath $state -Stage BackupRestore -Execute
# Review the separate restore-slot what-if before using -ReviewedWhatIf.
./Start-Lab12.ps1 -StatePath $state -Stage BackupRestore -Execute -ReviewedWhatIf
./Remove-Lab12.ps1 -StatePath $state -Execute
./Remove-Lab12.ps1 -StatePath $state -Execute
./Export-Lab12Evidence.ps1 -StatePath $state -Destination ./sanitized-evidence.json -ExecuteProgressive hints and separate solution
The download includes HINTS.md with progressive help and SOLUTION.md as a separate solution. Try CHALLENGE.md before opening the solution. README.md explains private-state, locking, interrupted operations, costs, and the explicit execution switches.
Hint 1
Compare the response body as well as the status. Does the running application describe its own readiness? Is staging still reporting the expected release and hashes?
Hint 2
Compare production and staging with their own saved baselines. Different release numbers are expected; unexplained changes are not.
Hint 3
Inspect slot-specific application settings. A correct deployment can run with an incorrect environment value.
Hint 4
Restore only the captured readiness setting. Preserve unrelated keys, then recheck hashes and production health before considering a swap.
Eight knowledge checks
1. What differs between scale-up and scale-out?
Scale-up changes the plan SKU and resources per worker. Scale-out changes the number of workers.
2. Why use a staging slot instead of deploying directly into production?
A slot supports validation before promotion and a controlled swap/rollback while separating the staging configuration and endpoint.
3. What does making LAB_SLOT_NAME slot-specific accomplish?
The slot label remains with its slot when code is swapped, so promoted code still identifies the endpoint correctly.
4. Why does HTTP 503 alone not establish this lab's root cause?
Different application and platform failures can produce 503. The readiness message, setting comparison, unchanged code, and healthy production identify the controlled cause.
5. Does VNet integration make the application's inbound endpoint private?
No. It supplies outbound VNet connectivity. Inbound access is governed separately by the configured endpoint and access restrictions.
6. Why disable run-from-package for this backup exercise?
The custom backup exercise needs extracted content; ZIP-mounted run-from-package content is not included in these App Service backups.
7. Why restore into a separate slot and verify file hashes?
It avoids overwriting production and proves that recovered content matches the original, rather than relying only on a backup status.
8. Does stopping a web app stop billing for its dedicated App Service plan?
No. Dedicated plan workers remain allocated and chargeable until the plan is removed or its capacity/tier is changed appropriately.
Live Azure results
The App Service exercises passed in West Europe on 10 October 2026. Production retained version 1 while a controlled staging setting produced HTTP 503. Restoring the captured setting returned staging to HTTP 200 without replacing code or changing access restrictions.
Baseline: production version 1 and staging version 2 returned HTTP 200. Three deployed file hashes matched their packages, including independent SCM inspection.
Networking and TLS: application and SCM endpoints retained the single workstation /32 allow rule and deny defaults. An Azure SCM request to production received HTTP 403. Trusted TLS 1.2 and same-host HTTP-to-HTTPS redirects passed. No certificate warning was bypassed.
Fault: staging returned the synthetic readiness HTTP 503; production remained HTTP 200. Original file hashes and unrelated controls were unchanged.
Recovery: PowerShell restored the captured value. Both slots passed validation; a second recovery was a verified no-op.
Slots: preview, promotion, and rollback passed. Release placement, sticky slot labels, restrictions, TLS, and integration were checked after convergence.
Scaling: S1 to S2 to S1 and one to two to one worker passed. Azure instance inventory confirmed two workers; 48 fresh requests observed both worker labels across the two slots with matching content. This does not establish equal traffic distribution.
Backup: the on-demand production backup succeeded. Its 68,039-byte ZIP and one XML artifact were inspected privately; all three original file hashes matched.
Restore: the supported PowerShell restore completed into the separate disposable slot. Version 1, HTTP 200, the corrected restore label, and all three hashes passed, including independent SCM file checks. Production and staging retained their original versions and hashes.
Cleanup: all recorded resources, deployment histories, and the empty owned group were removed. Two exact-ID absence checks passed; encrypted temporary SAS material was removed. Private audit evidence was retained.
Restore reconciliation
The first direct REST restore failed with an unauthorized-operation diagnostic. Its failed outcome remains in the evidence; the error alone did not establish the cause. After exact Activity Log reconciliation, a separate supported PowerShell attempt succeeded. No pending restore was resubmitted.
Restoration cleared the disposable target's ownership tags. After user approval, only its previously recorded tags were reapplied following exact-ID, provenance, and security checks. A temporary old slot label after the setting update was handled by waiting for the expected label rather than repeating the restore. The source scripts now include that bounded convergence check and a verified continuation path.
Local safety and package checks
Both Bicep templates built, Azure template validation and reviewed incremental what-if passed, and PowerShell static analysis reported no errors. Eighteen local checks passed, including parser checks, preview behavior, context/identity/lock/interruption simulations, and local Node readiness/hash responses. These offline guard simulations are not destructive live Azure fault tests.
Only allowlisted sanitized evidence and reviewed genuine captures are packaged. Raw addresses, resource identifiers, credentials, signed URLs, backup contents, and private state are excluded. The release process separately verifies ZIP extraction, every indexed file hash, and a fresh public-download SHA-256 before posting the article.
Scope of screenshots and guided topics
Portal captures show real configuration and operation checkpoints. Coding captures display actual source files, not command execution results. Application HTTP results come from fresh authenticated-network checks; a browser navigation block was not presented as the staging HTTP 503. The neon cover and inline illustration are conceptual artwork, not evidence.
Custom-domain mapping, certificate binding, metric/scheduled autoscale, Premium automatic scaling, and scheduled backups remain guided. The lab changes no existing domain or certificate. Wix mobile preview is distinct from actual-phone testing.
Verified download
A fresh public download matched the ZIP SHA-256 and passed extraction plus 52 individual file-hash checks. The downloaded solution is separate from the symptom-only challenge.
Comments