Platform Architecture and Capabilities

1. Executive decision

Levy Core On-Prem is a supported product edition, not a per-customer fork or a copy of Levy's hosted application.

The edition packages the rider experience, operator dashboard, core APIs, database, object storage, job workers, and IoT gateway for installation on customer-owned infrastructure. Payments and messaging are provider-neutral interfaces implemented by a customer's local gateways. Customer-support AI is separately gated, disabled by default, and permitted outbound access only through an explicit support egress policy.

The production edition provides all of these properties:

1. One customer installation is one isolated deployment and one data boundary.
2. It can operate without Supabase Cloud, Vercel, Stripe, Twilio, Sentry Cloud, Expo Update, or Levy CRM.
3. It has no runtime dependency on Levy-operated infrastructure.
4. It has a documented, default-deny outbound network policy.
5. It is installed, upgraded, backed up, restored, monitored, and rolled back through versioned artifacts and runbooks.
6. It preserves core rental safety and money invariants even when a local gateway, node, or network link fails.
7. It does not require a unique source-code branch per customer.

The product supports two certified local data-platform profiles:

• Pilot compatibility profile: all services run locally, but open-source Supabase-compatible APIs may be used internally to reach a working pilot sooner. This does not use Supabase's hosted platform or transmit telemetry to it.
• GA native-local profile: core clients call Levy APIs, authentication is local OIDC/OTP, storage is S3-compatible, and core server data access no longer requires Supabase client semantics. This is the supported long-term boundary.

If a customer prohibits even locally operated Supabase-derived components, skip the compatibility profile and begin with the native-local profile. That is a schedule choice, not an architectural dead end.

2. Outcomes and success measures

2.1 Product outcomes

• A rider can register, add an approved payment method, find a vehicle, scan it, unlock it, ride, pause/resume when supported, end the ride in a valid zone, receive a receipt, and view ride history.
• An operator can configure the fleet, vehicles, prices, zones, operating hours, local gateways, users, and rental policies; observe live fleet state; intervene safely; and export operational data.
• Supported IoT devices connect directly to the customer's in-country gateway, and GPS/command processing stays inside the approved boundary.
• Core scheduled behavior—ride metering, stranded-vehicle relock, reservation expiry, stale-device handling, payment reconciliation, and retention—runs locally and is observable.
• A trained customer infrastructure team can install a certified release without Levy production access.
• Levy can support the installation through customer-controlled, audited access without acquiring an undeclared copy of operational data.

2.2 Measurable acceptance targets

The exact service-level objectives are agreed per customer, but the reference release must demonstrate:

• no known critical core-flow parity gaps against the agreed feature matrix;
• zero unauthorized outbound DNS or network connections during the residency test suite;
• successful restore from encrypted backup into an empty recovery environment;
• lossless processing of the gateway's payment and messaging webhook conformance suite;
• no double capture, double refund, duplicate ride, or duplicate vehicle command under retry tests;
• continued safe ride completion and command reconciliation across worker restarts;
• documented recovery from app-node failure, worker failure, database failover, storage interruption, and IoT-gateway restart;
• a clean install, upgrade, and rollback rehearsal performed by someone who did not build the release;
• rider and operator mobile binaries that contain no production Levy-cloud fallback endpoint.

2.3 Non-goals for version 1

• Reproducing Levy's CRM, marketing site, lead workflows, collections, US tax filing, toll recovery, Turo/E-ZPass tooling, P2P marketplace, CaaS, insurance marketplace, dispute automation, Mercury/Airwallex payouts, fleet-growth AI, or internal support automation.
• Providing a generic payments product. The edition provides a gateway contract and certified adapter mechanism; the customer supplies or selects a processor.
• Guaranteeing operation on every Linux distribution, hypervisor, database service, or Kubernetes distribution. Levy certifies a bounded platform matrix.
• Supporting a single physical server as a highly available production architecture. A single-node bundle is for development, demonstration, or an explicitly accepted low-availability pilot.
• Creating an air-gapped public-rental system. Rider phones and IoT devices still need network access. An offline artifact-delivery option is separate from an offline runtime.
• Proving a country's legal interpretation. The customer and counsel define the approved data boundary, processors, retention, and disclosure obligations; the product enforces the resulting technical policy.

3. Scope matrix

The product uses an explicit allowlist. Only registered on-prem routes and modules are included in the release profile.

3.1 Required core modules

3.2 Optional locally operated modules

These are off by default and enabled only when the customer accepts their data and operational implications:

• advance reservations;
• booking portal and unattended kiosk clients;
• subscriptions or ride passes;
• parking/end-ride photo evidence;
• condition reports and digital agreements;
• local push notifications;
• local map-tile hosting and routing;
• customer-support AI through an explicit egress proxy;
• customer SSO for operators;
• additional IoT protocol packs;
• Arabic and right-to-left presentation packs;
• customer-specific analytics exports.

3.3 Excluded cloud/supplemental modules

The on-prem build manifest must exclude or hard-disable:

• Levy CRM/contact widget and marketing pixels;
• Frappe or any other customer-support/CRM application as part of the core rental bundle; if retained separately, it may use only the bounded support API/bridge;
• Stripe, Twilio, Resend, Sentry Cloud, Vercel analytics and domain APIs;
• Levy Gmail, WhatsApp, support, and internal alert destinations;
• OpenAI, Anthropic, Gemini, or other model access outside the support-AI adapter;
• US credit reporting, skip trace, disputes, sales tax, tolls, insurance, and payout integrations;
• external connected-car services unless separately certified for the customer's residency policy;
• Expo-hosted OTA updates;
• automatic calls to current fleets.levyelectric.com, the current Supabase project, iot.levyelectric.com, or any Vercel hostname.

3.4 Feature-manifest enforcement

Add a build-time DeploymentProfile manifest with cloud and core_onprem profiles. It controls route registration, navigation, background jobs, provider factories, CSP/allowed hosts, bundled assets, environment validation, and mobile configuration.

Rules:

• No on-prem feature is activated solely by a missing environment variable falling back to Levy Cloud.
• A module absent from the profile returns a stable feature_not_available response and is not scheduled.
• The build fails when a required local provider is not bound.
• CI scans compiled server and mobile artifacts for prohibited domains and cloud SDK initialization.
• CI scans source and generated artifacts for provider/Levy fallback literals, including ENV || 'host-or-key' patterns.
• The on-prem build runs a dedicated tsc --noEmit, import-boundary lint, and runtime configuration-schema check; it may not inherit the current web build's ignored TypeScript or lint failures.
• Selecting an optional module automatically selects its required cleanup, retry, expiry, reconciliation, and recovery jobs. Configuration validation rejects half-wired module/job combinations.
• Customer variations are signed configuration overlays, not source forks.

4. Portability and hardening guarantees

The on-prem edition is packaged independently from Levy's hosted environment. Direct cloud SDK coupling, hosted fallbacks, provider-specific payment and messaging behavior, deployment-specific schedules, and non-local telemetry are excluded from the on-prem profile at build and release time.

The signed deployment profile controls route registration, jobs, provider adapters, allowed destinations, mobile endpoints, configuration, and packaged assets. Release qualification verifies that each enabled feature has its local data, identity, storage, scheduling, recovery, and observability dependencies. This makes the installed system a bounded local product rather than a repointed cloud container.

5. Target architecture

5.1 Logical topology

 Rider app ───────────────┐
 Operator app/dashboard ──┼──► In-country load balancer / WAF / TLS
 Booking or kiosk client ─┘                    │
                                               ▼
                                  Levy API + web application
                                     │       │       │
                                     │       │       └──► local object storage
                                     │       └──────────► local auth/OIDC
                                     ▼
                              local PostgreSQL cluster
                                     ▲
                                     │
      IoT devices ─► TCP/MQTT gateway┼◄──► durable command/job worker
                                     │
                         ┌───────────┴────────────┐
                         ▼                        ▼
               local payment adapter    local messaging adapter

 Optional support namespace ─► audited egress proxy ─► approved AI provider

5.2 Deployment units

These OCI images are independently versioned and released as one compatibility-tested bundle:

1. levy-core-web-api — Next.js operator UI, rider APIs, administrative APIs.
2. levy-core-worker — PostgreSQL-backed jobs, reconciliation, retention, outbox delivery.
3. levy-core-iot — selected device protocol listeners and HTTP operations endpoint.
4. levy-core-migrations — baseline/install/forward migration job; never a long-running service.
5. levy-core-support-bridge — optional, isolated support-AI boundary.
6. levy-core-tools — diagnostics, synthetic checks, backup verification, support-bundle creation.

Customer-provided or reference infrastructure services:

• PostgreSQL with PostGIS; TimescaleDB only when required by the certified telemetry profile;
• S3-compatible object storage such as the customer's existing platform or the reference local object-store chart;
• local OIDC identity provider for operators and a Levy rider-token service backed by the messaging adapter;
• local container registry or offline OCI artifact import;
• customer DNS, certificates, load balancer, firewall, monitoring, and secrets system.

5.3 Certified runtime profiles

Kubernetes is the reference production orchestrator. Docker Compose remains a developer and pilot convenience, not the production abstraction. Customer-managed PostgreSQL, object storage, OIDC, ingress, and observability are supported only after conformance tests pass.

6. Data platform and removal of hosted Supabase dependency

6.1 Architectural boundary

Introduce provider-neutral server interfaces:

AuthService
Database / repositories
ObjectStore
EventPublisher
JobQueue
SecretsProvider

Core domain services depend on these interfaces. They must not import hosted-client constructors or read vendor environment variables directly.

6.2 Compatibility-local profile

For the first pilot, Levy may deploy locally operated open-source PostgREST/Auth/Storage-compatible components on customer infrastructure. This profile exists to reduce simultaneous change in authentication, storage, RLS, and core ride flows.

Constraints:

• all images are pinned, scanned, and deployed inside the customer boundary;
• telemetry and phone-home behavior are disabled and verified;
• public clients use the customer's domain, keys, and certificates and call only versioned Levy APIs; they never call the compatibility Auth, PostgREST, or Storage components directly;
• the compatibility services are an internal implementation detail, not a managed dependency;
• no new core feature may deepen direct client coupling;
• every core endpoint migrated to native-local is covered by parity tests before the compatibility path is retired.

6.3 Native-local profile

The GA target removes direct Supabase semantics from core clients and server modules:

• Rider and operator clients call Levy APIs, not PostgREST tables or Storage directly.
• Operator authentication uses customer OIDC where available; local accounts remain an explicitly configured fallback.
• Rider OTP verification creates a local Levy session after the messaging adapter proves possession.
• The API verifies signed access tokens against local keys and maps them to local customer/operator identities.
• Server data access uses explicit repositories and stored procedures through PostgreSQL connections.
• S3-compatible signed URLs replace vendor storage calls.
• Live updates use Levy WebSocket/SSE endpoints backed by PostgreSQL notifications or the selected local event transport.

6.4 Database baseline strategy

Do not initialize a new installation by replaying the complete SaaS migration history.

Create a canonical core_onprem_1_0 baseline:

1. Capture an observed schema census from a production clone or approved production introspection; the live schema is evidence, not automatically the desired definition.
2. Inventory all tables, functions, triggers, types, policies, extensions, grants, and application queries reached by the core allowlist.
3. Machine-diff the observed live schema, repository definitions, migration ledger, and application expectations. Classify every difference as intended, accidental drift to repair, compatibility-only, or deliberately omitted; zero unclassified differences is the workstream exit criterion.
4. Generate a dependency graph from retained objects and identify shared financial/identity primitives that core flows require.
5. Produce the intended definitions from that reconciliation, not merely from the production migration ledger or a raw schema dump.
6. Normalize extension placement and every dependent function/search path, especially the current extensions.* PostGIS convention, against the certified PostgreSQL image.
7. Remove excluded product tables, seed data, cloud URLs, foreign webhooks, provider secrets, and Levy-specific recipients.
8. Replace auth.uid() and service-role assumptions with local identity/session context.
9. Convert storage policies to object-store/application authorization.
10. Replace pg_net/external HTTP database calls and cloud scheduler functions with the worker/outbox.
11. Decide whether TimescaleDB is required for each supported telemetry tier; do not make it a hidden install dependency.
12. Apply the baseline to an empty certified PostgreSQL image and run schema-contract tests.
13. Generate a deterministic schema manifest containing object names, normalized definitions, extension versions, grants, and checksums.
14. Record the baseline version in an on-prem migration ledger.
15. Permit only reviewed forward migrations after the baseline; never edit an already released on-prem migration.

Baseline deliverables:

• deploy/onprem/database/baseline.sql or a generated equivalent;
• deploy/onprem/database/schema-manifest.json;
• synthetic reference data with no production PII;
• a clean-install integration test;
• a baseline regeneration tool whose output must be reviewed and versioned;
• a machine-readable live-versus-ledger-versus-baseline reconciliation report with no unclassified drift;
• a cloud-to-on-prem mapping document for each retained object.

6.5 Identity baseline and compatibility-to-native cutover

Create a provider-neutral identity model before the compatibility pilot:

• identities owns the stable local subject ID used by core foreign keys and audit records;
• identity_credentials/identity_sessions own local credential metadata, refresh-token families, revocation, and issuer state;
• external_identity_refs maps compatibility Auth subjects and customer-IdP subjects to the stable local subject;
• rider, operator, and service realms remain distinct and cannot reuse identifiers accidentally;
• compatibility-local authentication creates or resolves the stable local subject on first use and records the external mapping; core tables never start a new dependency on auth.users;
• inventory and rewrite every auth.users foreign key, auth.uid() policy/RPC, service-role assumption, and auth_uid lookup reached by the core scope;
• backfill and reconcile existing auth_uid references before cutover, including orphan, duplicate, merged, deleted, and disabled identities;
• native-local GA changes the token issuer and removes the compatibility mapping from the request path; it does not re-key riders, operators, ownership, or audit history.

The identity cutover plan defines access/refresh token formats, issuer/audience, signing-key rotation and overlap, refresh reuse detection, logout-all, session migration or forced reauthentication, emergency revocation, and rollback. A compatibility profile is acceptable only if this local-subject foundation and API-only client boundary are present from the first pilot release.

6.6 Tenant and authorization model

Each deployment is single-customer but may contain multiple operational subaccounts or brands. Preserve subaccount_id scoping so roles, data exports, and future acquisitions remain isolated.

• Keep database row-level security where it adds defense in depth.
• Set request identity and allowed subaccounts through transaction-local PostgreSQL settings; reset them on every pooled connection.
• Do not expose a database service-role equivalent to clients.
• Separate application, migration, read-only reporting, backup, and monitoring database roles.
• Put operator permissions and rider ownership checks in both the domain service and database policy for safety-critical writes.
• Test horizontal and vertical authorization with generated cross-subaccount fixtures.

6.7 Data model normalization

Core tables must stop treating provider-specific identifiers as canonical business state.

Introduce or normalize:

• payment_customers, payment_methods, payment_transactions, payment_refunds, payment_webhook_inbox, and payment_reconciliation_runs;
• wallet_accounts, wallet_transactions, wallet_holds, wallet_topup_operations, and wallet_reconciliation_runs with immutable, idempotent entries and derived balances;
• message_deliveries, message_templates, otp_challenges, and message_webhook_inbox;
• external_provider_refs where a generic link is appropriate;
• job_runs, outbox_events, and inbox_events for durable asynchronous work;
• provider-neutral receipt fields on rides and reservations;
• provider-neutral ride financial fields that preserve gross fare, adjustments, refunds, wallet effects, processor effects, debt, and a locally defined equivalent of current net-deposited accounting;
• local identity subject mappings separate from legacy auth UIDs.

Do not rename historical Stripe-shaped SaaS columns in place during the first extraction. Add the neutral model, dual-read only where necessary for migration, prove parity, and remove compatibility fields in a later major on-prem baseline.

6.8 Retention and deletion

Every high-volume or sensitive class receives an explicit configurable policy:

• raw GPS telemetry;
• normalized ride route points;
• IoT protocol payloads and debug logs;
• parking/condition images;
• authentication challenges and device-attestation evidence;
• payment and refund records;
• message bodies and provider receipts;
• audit logs;
• support-AI prompts and responses;
• backups and exported reports.

GPS retention is tiered rather than a single deletion clock: short-lived raw device payloads, longer-lived normalized route points needed for ride evidence and support, and minimized/downsampled operational aggregates. Each tier has a fixed configured duration, table/partition boundary, downsampling rule, legal-hold behavior, storage budget, and deletion proof.

Deletion jobs must be resumable, auditable, scoped, and covered by legal-hold configuration. Backup expiration is part of deletion policy; deleting only the live row is insufficient.

7. Identity, authentication, and access control

7.1 Identity split

Use distinct identity realms:

• Riders: application-managed identity, normally phone OTP through the local messaging adapter; optional email or customer-approved national identity integration.
• Operators: customer OIDC/SAML brokered through a local identity provider; optional local emergency accounts.
• Devices: per-device credentials or certificates, never rider/operator tokens.
• Services: short-lived workload identities or rotated service credentials.

7.2 Rider authentication flow

1. App requests an OTP challenge from Levy API.
2. API applies device attestation when permitted, phone normalization, rate limits, abuse controls, and customer country rules.
3. Messaging adapter sends the OTP without receiving unrelated rider data.
4. API stores only a salted/peppered challenge hash, expiry, attempts, delivery reference, and audit metadata.
5. Verification consumes the challenge atomically and creates or resolves the rider.
6. Local token service issues short-lived access and rotating refresh tokens.
7. App stores tokens in platform secure storage.
8. Revocation, logout-all, lost-device, and account-deletion flows operate locally.

The adapter must never be trusted to decide application authorization. It proves delivery/possession only.

7.3 Operator access

• Map customer IdP groups/claims to Levy roles through signed configuration.
• Support fleet admin, fleet manager, technician/operations, finance/reporting, support-read-only, and platform break-glass roles.
• Require MFA from the IdP for privileged roles.
• Make local break-glass accounts disabled by default, stored in the customer's secrets system, time-limited when activated, and audit-alerted.
• Record sign-in, permission changes, data exports, vehicle commands, price changes, refunds, and support access.

7.4 Mobile attestation

Mobile attestation requires an explicit processor/network decision. Google Play Integrity verification uses Google's API; every enabled attestation implementation is allowlisted with its documented and tested verification path and egress. Support three customer-selectable modes:

• required — normal public-store deployment;
• risk_based — attestation failures receive constrained behavior plus local fraud controls;
• disabled_by_policy — only when the customer accepts the abuse risk and compensating controls.

disabled_by_policy requires explicit local compensating controls: per-IP/ASN/device/installation quotas, first-send velocity caps, provider spend ceilings, challenge/proof-of-work where feasible, anomaly alerts, and an operator-visible circuit breaker. Attestation must not silently fall back to Levy infrastructure.

8. Local payment gateway architecture

8.1 Design principles

• Levy never stores raw PAN, CVV, or equivalent payment credentials.
• Tokenization happens in the customer's selected gateway UI/SDK or hosted field.
• The Levy ledger is provider-neutral and authoritative for application intent; the gateway is authoritative for movement status.
• Every mutation has a Levy operation ID and a provider idempotency key.
• Webhooks are authenticated, stored before processing, deduplicated, and replayable.
• No ride is treated as paid solely because a synchronous client response says so.
• Refund and void semantics are capability-driven, not inferred from Stripe behavior.

8.2 Gateway contract

Define a versioned PaymentGateway interface and an HTTP adapter protocol for out-of-process customer connectors. Out-of-process, mutually authenticated HTTP is the default trust and failure boundary for customer/vendor adapters; in-process adapters are limited to Levy-authored implementations shipped, scanned, and supported as part of the core release.

Required operations:

interface PaymentGateway {
  capabilities(): Promise<PaymentCapabilities>
  createOrLinkCustomer(input: CustomerRefInput): Promise<GatewayCustomerRef>
  createTokenizationSession(input: TokenizationInput): Promise<TokenizationSession>
  attachPaymentMethod(input: AttachMethodInput): Promise<PaymentMethodRef>
  authorize(input: AuthorizationInput): Promise<PaymentResult>
  capture(input: CaptureInput): Promise<PaymentResult>
  charge(input: ChargeInput): Promise<PaymentResult>
  voidAuthorization(input: VoidInput): Promise<PaymentResult>
  refund(input: RefundInput): Promise<RefundResult>
  getPayment(input: GetPaymentInput): Promise<PaymentResult>
  listSettlementActivity(input: ReconciliationWindow): Promise<SettlementPage>
  verifyAndParseWebhook(input: RawWebhook): Promise<VerifiedPaymentEvent>
  health(): Promise<ProviderHealth>
}

Capabilities include:

• tokenization model and supported mobile/web SDKs;
• authorize/capture versus immediate sale;
• partial/multiple capture;
• void window;
• full/partial/multiple refunds;
• 3DS or local step-up flow;
• asynchronous payment states;
• supported currencies and minor-unit rules;
• card/network wallet types;
• webhook ordering and retry behavior;
• reconciliation API/export availability;
• provider idempotency limits.

8.3 Application payment state machine

Use explicit states:

created → pending_customer_action → authorized → capture_pending → captured → partially_refunded → refunded

Terminal/exception states:

declined, authorization_expired, voided, failed, unknown_requires_reconciliation, chargeback_reported.

State transitions are monotonic unless a documented provider event reverses settlement. Unknown or conflicting provider state fails closed for new money movement but does not strand a rider or disable safe ride termination.

8.4 Ride interaction

• Pre-ride eligibility is a policy: stored token, preauthorization, wallet balance, deposit, or postpaid account.
• Record the policy decision and quoted currency/amount before unlock.
• Never hold an IoT unlock indefinitely while waiting for a slow payment callback.
• Fare accumulation remains local and currency-safe.
• End-of-ride finalization writes an immutable fare snapshot before capture/sale.
• Fare adjustments and refunds are always recorded against the originating ride first; any wallet credit/debit is a downstream ledger effect. A direct wallet credit may never replace or obscure ride-level gross fare, refund, processor, debt, or net-deposited-equivalent accounting.
• Wallet top-ups, auto-top-ups, holds, releases, debt applications, and refunds use immutable idempotent operations and reconcile independently against the payment provider and ride ledger.
• Payment failure enters a recoverable debt state and may restrict future starts, but never blocks a safety-required ride end or vehicle lock.
• Retry workers use the same immutable operation key and amount.
• Reconciliation compares Levy intent, provider status, and ledger effects.

8.5 Adapter certification suite

Every local payment adapter must pass:

• contract/schema/version negotiation;
• authentication and certificate rotation;
• repeated request/idempotency;
• duplicate/out-of-order webhook delivery;
• timeout after provider success;
• asynchronous success and decline;
• partial capture/refund if advertised;
• provider outage and recovery;
• settlement pagination and timezone boundaries;
• currency/minor-unit vectors;
• reconciliation with intentionally missing webhooks;
• secret redaction and residency logging;
• load and rate-limit behavior.

The first customer's adapter is production code owned under the same release discipline as core. A vendor-provided adapter is not trusted until it passes the suite.

9. Local messaging gateway architecture

9.1 Gateway contract

interface MessagingGateway {
  capabilities(): Promise<MessageCapabilities>
  sendOtp(input: OtpDeliveryInput): Promise<MessageAccepted>
  sendTransactional(input: TransactionalMessageInput): Promise<MessageAccepted>
  getDelivery(input: DeliveryLookup): Promise<DeliveryStatus>
  verifyAndParseWebhook(input: RawWebhook): Promise<VerifiedMessageEvent>
  health(): Promise<ProviderHealth>
}

The contract supports SMS first and allows WhatsApp or a local channel only when configured. OTP codes are generated and verified by Levy unless the approved gateway offers a formally certified provider-managed OTP flow. As with payments, customer/vendor messaging adapters run out of process over a versioned mutually authenticated HTTP contract by default; only Levy-authored adapters may be packaged in process.

9.2 Messaging invariants

• Normalize and validate destinations before adapter invocation.
• Rate-limit by destination, device, IP, installation, and provider budget.
• Template IDs and localized bodies are customer configuration.
• Store provider message ID, template/version, status, timestamps, and redacted failure details.
• Do not log OTP values or full message bodies containing sensitive data.
• Delivery callbacks use inbox deduplication and tolerate reordering.
• Transactional messages are emitted through an outbox after the business transaction commits.
• Push notifications contain minimal text/data; the app fetches sensitive details from the local API after authentication.

9.3 Email and push

Email is optional through a local SMTP/API adapter. Push may go through APNs/FCM or a customer-approved intermediary because public mobile operating systems control delivery. The residency policy must explicitly approve this path. When approved, send only an opaque event identifier and generic text for sensitive events.

10. Core application extraction

10.1 One codebase, explicit composition

Keep one source tree. Add boundaries rather than copying applications:

• src/lib/platform/auth/
• src/lib/platform/database/
• src/lib/platform/storage/
• src/lib/platform/jobs/
• src/lib/platform/events/
• src/lib/gateways/payments/
• src/lib/gateways/messaging/
• src/config/deployment-profiles/
• deploy/onprem/

Provider factories receive validated configuration at process start. Domain code receives interfaces through request/application context. Tests can bind in-memory or deterministic simulators.

10.2 API migration order

Migrate vertically by executable rider journey rather than replacing all Supabase calls horizontally:

1. health/config/branding and session verification;
2. OTP enrollment and rider profile;
3. nearby vehicles, vehicle details, zones, and pricing;
4. payment-method setup and pre-ride eligibility;
5. reserve/hold, unlock, and ride start;
6. location, metering, pause/resume, and active ride;
7. end ride, parking validation, fare snapshot, capture, and receipt;
8. history, refund/support-safe views, and account deletion;
9. operator vehicles/zones/pricing/users/live fleet;
10. reports, exports, and optional reservations.

For each slice:

• define the stable API schema;
• add repository and provider-neutral domain service;
• move web/mobile callers to that API;
• run parity tests against the existing cloud implementation with synthetic fixtures;
• block direct table/storage access from the migrated client feature;
• add it to the core feature manifest.

10.3 Mobile applications

Create a customer build pipeline that generates signed rider and operator variants from configuration:

• bundle/package IDs and display names;
• icons, colors, support/legal URLs, privacy strings;
• API/Auth/Update domains with no fallback;
• associated/universal-link domains;
• map provider and tile endpoint;
• gateway-specific tokenization SDK configuration;
• allowed locales and RTL behavior;
• attestation mode and identifiers;
• push provider configuration;
• customer support and emergency contacts.

Extend the rider application's existing brand configuration rather than replacing it, but remove every hosted Supabase, Stripe, Expo/Google, Levy-domain, key, and project-number fallback from the on-prem build path. Build the corresponding deployment-profile/brand layer for the operator application, whose current API, Supabase, map, monitoring, OTA, bundle, and project settings are more directly embedded. Rider and operator variants have separate owners, tests, manifests, and store/distribution checklists.

Prefer gateway-hosted fields or browser/redirect tokenization because they avoid compiling customer payment code into the app. If a gateway requires a native SDK or Expo config plugin, define a manifest-selected native module set. Each distinct module set creates a separately built and certified native binary/runtime pair, store-review/distribution lead time, SBOM, adapter compatibility entry, and upgrade obligation; it is configuration-driven, but not an OTA-switchable runtime option and not a source fork.

Configuration is validated and frozen into a signed build manifest. The application displays build profile, API host, release, and configuration digest in an operator-accessible diagnostics screen.

Host OTA updates on customer infrastructure using a protocol-compatible service, or disable OTA and require store/native releases. A binary must never switch from a customer endpoint to Levy's hosted endpoint because a configuration value is missing. An OTA release requires non-zero installed-base evidence for the exact platform, runtime version, customer profile, and native module set before promotion; publication success alone is not proof that a compatible device can receive it.

10.4 Web/operator application

• Remove marketing layouts, pixels, CRM widget, public lead forms, and unrelated admin navigation from the on-prem bundle.
• Replace Vercel geo headers/domain management with customer ingress headers and configuration.
• Generate CSP/connect-src from the signed allowed-host manifest.
• Serve all required static assets locally unless an approved external asset host is declared.
• Support customer OIDC logout, session expiry, and role mapping.
• Expose local health, job, gateway, storage, and IoT status without leaking secrets.

10.5 Localization

Externalize locale, currency, taxes/fees, timezone, phone normalization, address format, number/date display, and legal copy.

For an Egypt deployment, Arabic/RTL is a first-client workstream unless the customer formally accepts English only. RTL acceptance includes navigation, maps/controls, forms, receipts, PDF agreements, push/SMS templates, and operator tables—not merely translated strings.

11. IoT and GPS data plane

11.1 Required changes

• Remove hardcoded Levy app, Supabase storage, and proxy URLs.
• Bind APP_URL, database, firmware, and public proxy identity from validated local configuration.
• Package only the protocol listeners enabled for the customer.
• Store raw and normalized telemetry locally under explicit retention rules.
• Move command intent and acknowledgement state to a durable local store.
• Replace in-memory-only offline command queues for supported commands.
• Preserve per-device ordering, TTL, acknowledgement requirements, and safety rules.
• Separate telemetry receipt from downstream processing so temporary database slowdown does not drop frames without evidence.
• Add backpressure, bounded disk spool, replay, poison-frame quarantine, and operator-visible lag.
• Produce a protocol-specific command safety matrix for every command: durability, ordering scope, supersession/coalescing rule, TTL, acknowledgement definition, retry budget, reconnect behavior, and safe drop/expiry outcome. The generic queue cannot ship until each enabled protocol/command row is approved and tested.

11.2 Connection ownership and failover

Do not place two unrestricted IoT gateways behind a generic TCP load balancer.

Reference design:

• active gateway instances acquire a renewable ownership lease per protocol/device shard;
• connection registration writes the owning instance and fencing token;
• commands route to the current owner through the durable command bus;
• stale owners are fenced before another instance accepts command authority;
• graceful deployment drains connections and stops accepting new work before termination;
• queued commands survive process restart but expire according to command-specific safety TTLs;
• split-brain tests prove that at most one owner can issue a command for a device at a time.

For protocols/devices that cannot reconnect predictably, certify active/passive failover first. Active/active is earned per protocol through testing.

11.3 Device and network onboarding

The customer supplies:

• public/static in-country endpoints and DNS;
• firewall/NAT rules for enabled TCP/MQTT ports;
• carrier/APN routing evidence when network path residency matters;
• device credentials/password rotation process;
• SIM/provider escalation contacts;
• certificate model for TLS-capable protocols.

The installer supplies a device connectivity test, protocol handshake simulator, and per-device migration checklist. A production cutover uses canary devices before fleet-wide endpoint changes.

11.4 Maps

GPS processing and map rendering are separate residency decisions. Support:

• customer-approved external map provider;
• local vector/raster tile service generated from licensed map data;
• customer GIS service through a map adapter.

Local maps must include attribution/license compliance, update cadence, geocoding/routing requirements, disk sizing, and failure behavior. Core ride start/end and zone enforcement cannot depend on a third-party map UI being available; they use local geometric data and PostGIS.

12. Job and event architecture

12.1 Replace deployment-specific cron

Inventory every current cron/LaunchAgent and classify it as:

• required core safety/revenue;
• optional core operations;
• cloud-only/supplemental;
• obsolete or duplicate.

Move retained jobs to a PostgreSQL-backed worker with:

• named schedules in version-controlled configuration;
• leader election/advisory locking;
• durable jobs and retry state;
• idempotency/operation keys;
• bounded concurrency per job and subaccount;
• deadlines and resumable pagination;
• poison-job handling;
• run history, metrics, manual replay, and pause controls;
• dependency health checks;
• graceful shutdown and deploy drain.

HTTP cron endpoints may remain as authenticated manual triggers, but they are not the scheduler of record.

PostgreSQL is the default durable job/event transport for version 1. Add a dedicated broker only when a 2×-peak qualification run shows any of these after database tuning: queue load degrades core API p95 by more than 5%, consumes more than 15% of primary database CPU or IO budget, pushes safety/money dispatch p99 over one second, or cannot drain a one-hour peak backlog within 15 minutes. Customer SLOs may set stricter values; crossing a threshold triggers an architecture review and migration plan rather than an ad hoc second queue.

12.2 Initial required job catalog

Conditional modules bring mandatory companion jobs: parking photos require stuck-photo-end recovery; automatic or low-rating refunds require the matching policy/reconciliation jobs; asynchronous identity verification requires its retry/reaper; booking/channels require reserve-expiry; kiosk/rental sessions require timeout and ride-recovery jobs; subscriptions/top-ups require renewal, dunning, and reconciliation. The manifest cannot enable the user-facing module without these jobs, dashboards, and runbooks.

Before GA, each retained job receives an owner, schedule, maximum runtime, concurrency, idempotency proof, retry policy, alert threshold, recovery command, and disable/rollback method.

12.3 Transactional outbox/inbox

• Business transactions write domain state and outbox event atomically.
• Workers claim outbox rows with leases and idempotent delivery keys.
• Provider webhooks are authenticated and stored in an inbox before domain processing.
• Duplicate events return success after confirming the stored result.
• Events contain opaque IDs whenever the consumer can retrieve protected detail locally.
• Retention and replay windows are explicit.

13. Security and data-residency controls

13.1 Network policy

Restricted-Egress starts with no outbound internet access from application namespaces.

Separate allowlists by workload:

• core API/worker: local infrastructure and explicitly approved gateways only;
• IoT gateway: local database/event transport plus device networks;
• migration/tooling: local registry/database only at runtime;
• support bridge: approved AI endpoints through an audited egress proxy;
• mobile-platform attestation verifier: only the exact server-side provider endpoint required by the selected profile (Google Play Integrity in the current Android implementation), with no access to unrelated application data;
• mobile clients: customer API/map/update endpoints plus approved OS services.

Do not use a broad *.googleapis.com, *.supabase.co, or https: CSP/network allowance in the on-prem profile.

13.2 Automated residency verification

Add a release gate that:

1. scans source, generated config, container layers, JS bundles, native manifests, and database baseline for prohibited domains;
2. runs core E2E tests behind a DNS sink and default-deny firewall;
3. captures attempted DNS and network destinations;
4. fails on undeclared egress, including error reporting and asset loading;
5. verifies support AI cannot be reached from the core namespace;
6. verifies logs and support bundles redact GPS, payment tokens, OTPs, secrets, and unnecessary PII;
7. generates a signed machine-readable egress manifest for the release.
8. asserts that E2E OTP codes, debug attestation tokens, debug login routes, fixture users, and other test bypasses are compiled out or unconditionally rejected in every on-prem production profile.

13.3 Encryption and secrets

• TLS for all public and cross-node traffic; mutual TLS where supported internally.
• Customer-controlled encryption at rest for database, object storage, backups, and persistent queues.
• Secrets supplied through customer secrets management; never baked into images, charts, mobile config, or Git.
• Separate secrets per service and environment with documented rotation.
• Provider webhook secrets/certificates support overlap during rotation.
• Database field-level encryption for gateway tokens or sensitive identity artifacts when the threat model requires it.
• Encryption keys and backups remain in the approved country/boundary.

13.4 Software supply chain

Every release includes:

• pinned image digests;
• signed OCI images and offline archive manifest;
• SBOM for server and mobile artifacts;
• vulnerability scan receipt and exception ledger;
• open-source license inventory and required notices;
• provenance tying artifacts to source commit and build workflow;
• supported dependency and operating-system matrix;
• reproducible configuration checksums.

13.5 Support AI boundary

The optional support bridge is the only model-enabled component. OpenAI and any separately operated Frappe/customer-support system sit outside the core rental runtime and may access it only through this bounded support interface.

• Disabled and unscheduled by default.
• Separate namespace/service account/network policy.
• No direct database credentials; it calls a bounded support API.
• The support API returns allowlisted, minimized fields and redacts location precision and payment/identity secrets.
• Explicit operator action or approved workflow initiates a request.
• Provider, model, purpose, data categories, prompt hash, actor, and response disposition are audited locally.
• Customer can require local models or disable the feature without affecting rentals.
• Model failure never changes a ride, vehicle command, balance, refund, or access decision.

13.6 Security testing

• threat model covering riders, operators, devices, local admins, gateway providers, supply chain, and Levy support;
• SAST, dependency, secret, container, IaC, and mobile binary scanning;
• API authorization fuzzing and RLS tests;
• payment/messaging webhook spoofing and replay tests;
• IoT command authorization, protocol fuzzing, and split-brain testing;
• external penetration test before first production deployment;
• remediation SLA and signed exception process.

14. Observability and operations

14.1 Local observability

Emit vendor-neutral OpenTelemetry-compatible traces, Prometheus metrics, and structured logs. The reference stack may include local collectors, dashboards, and log storage, but customers can bind certified equivalents.

Required dashboards:

• API availability/latency/error rate;
• active rides and ride lifecycle lag;
• IoT connections, frames, command queue depth/age, acknowledgements, replay lag;
• job schedules, failures, retries, and overdue runs;
• payment authorization/capture/refund/reconciliation state;
• messaging acceptance/delivery/OTP conversion;
• database replication, connections, locks, disk, slow queries;
• object-store errors/capacity;
• backup age and restore-test status;
• unauthorized egress attempts;
• release/configuration versions.

Alerts route through the customer's local alert manager. Levy email, Slack, Gmail, or Sentry must not be a hidden fallback.

The operations package generates a monthly evidence pack from local systems: uptime and error budget, ride-start/end success, OTP acceptance/delivery/conversion, payment success and wallet/settlement reconciliation variance, incidents, overdue jobs, backup age and restore-test receipt, signed egress-manifest result and unauthorized attempts, security exceptions, capacity trend, installed versions, certificate/secret expiry, and unresolved risks. The customer owns receipt and retention; Levy receives it only through an approved support process.

14.2 Health model

Expose separate endpoints:

• liveness — process is responsive;
• readiness — instance can safely receive traffic;
• dependency health — database/storage/gateways/event transport;
• deep synthetic checks — explicit, authenticated, rate-limited operations;
• version/config — non-secret artifact and schema compatibility.

An unhealthy payment or messaging provider degrades only the dependent function. It does not make vehicle lock/end-ride APIs unavailable.

Always-available safety invariants are tested explicitly:

• an already authenticated rider can end a known active ride and the platform can issue/confirm the safe lock command during payment, messaging, push, map, operator-IdP, or attestation-provider outage;
• local access-token verification does not call the customer IdP on each request, and active-ride ownership proof remains locally available for the supported token/offline window;
• time-limited, customer-controlled operator break-glass access can execute audited critical fleet commands during IdP outage;
• dependency loss may block a new login, tokenization, or ride start when required proof is unavailable, but it cannot silently discard an end, lock, fare checkpoint, or financial intent;
• recovery workers reconcile every deferred end, lock acknowledgement, charge, message, and audit event after the dependency returns.

14.3 Support bundle

Provide a customer-run tool that creates an encrypted diagnostic bundle containing:

• versions and configuration digests;
• redacted health/metrics excerpts;
• job and migration status;
• recent error fingerprints;
• network/egress policy results;
• optional selected device diagnostics.

It excludes raw GPS, raw email/SMS bodies, OTPs, payment tokens, secrets, identity documents, and consumer PII by default. The customer reviews and transfers it through an approved channel.

14.4 Responsibility model

Before install, sign a RACI covering:

15. Availability, backup, and disaster recovery

15.1 Reference production topology

• redundant ingress/load balancer;
• at least two web/API replicas;
• at least two worker replicas with leader/lease controls;
• PostgreSQL HA appropriate to the customer's RPO/RTO;
• redundant object storage or customer storage SLA;
• IoT active/passive or certified sharded failover;
• monitoring/alerting independent enough to detect application outage;
• encrypted backups to a second failure domain that remains in-country.

Final node counts and sizing come from load tests and the customer's failure-domain design, not a generic minimum pasted into a contract.

15.2 Backup policy

Back up:

• database base backups plus WAL/PITR where supported;
• object storage with versioning/replication as required;
• identity-provider configuration;
• signed customer configuration overlays;
• encryption-key recovery material under customer control;
• release and migration manifests.

Backups are encrypted, access-audited, retention-controlled, and tested through automated restore rehearsals. A backup job succeeding is not evidence that restore works.

15.3 Recovery tiers

Define customer-specific targets for:

• node/service restart;
• database failover;
• site restore from backup;
• lost object-store node;
• corrupted deployment/configuration;
• compromised secret rotation;
• IoT endpoint/IP migration;
• mobile release rollback.

At least one full site-recovery exercise is a production launch gate.

16. Installation, configuration, and lifecycle

16.1 Installation bundle

deploy/onprem/ will contain:

• internal Helm chart and values schema, reconciled by the signed installation controller for source-confidential third-party production installs rather than handed to the installer as a generic Helm interface;
• developer/pilot Compose bundle;
• signed levyctl client, installation-controller image/API, immutable installation-request CRD, customer-owned site-policy object, signed bootstrap, exact admission policies, and installer authorization/denial contract;
• customer-scoped online/offline artifact-custody tooling, aggregate release index, offline entitlement, trust-anchor/revocation package, and source-exclusion receipt;
• bounded preflight, capacity, conformance, residency, support-bundle, evidence-signing, and acceptance tooling;
• immutable installation-fixture marking, reporting/payout exclusion invariants, and signed test-data disposition tooling;
• migration/baseline job;
• reference network policies;
• ingress/TLS examples;
• backup and restore jobs;
• observability dashboards/rules;
• gateway simulator and conformance harness;
• synthetic IoT/rider/operator tests;
• signed sample configuration with no secrets;
• install, upgrade, rollback, DR, and troubleshooting runbooks.

16.2 Preflight

Before writing data, the installer verifies:

• supported CPU architecture, OS/container runtime/orchestrator versions;
• DNS and time synchronization;
• storage class capacity/performance;
• database version/extensions/settings;
• certificate chain and expiry;
• internal and device-facing ports;
• registry or offline artifact availability and signatures;
• secrets and key permissions without printing values;
• payment/messaging sandbox reachability;
• object-store signed upload/download/delete;
• backup target reachability;
• outbound policy matches the declared manifest;
• customer configuration schema and unique identifiers.

Preflight produces a signed receipt and makes no irreversible external transaction.

16.3 Configuration hierarchy

1. signed release defaults;
2. deployment profile;
3. customer signed overlay;
4. environment-specific non-secret values;
5. secret references resolved at runtime.

Startup fails on unknown keys, invalid combinations, missing required providers, or a configuration digest not approved for the installed release. Avoid free-form environment-variable sprawl.

16.4 Upgrades

• Publish regular LTS releases with a declared support window and N/N-1 mobile/API compatibility.
• Use expand/migrate/contract database changes; never require app and schema to switch at the same instant.
• Run migration preflight against a restored recent backup for high-risk releases.
• Back up and verify recovery point before production migration.
• Deploy web/workers with rolling or blue/green strategy.
• Drain IoT gateways deliberately; never overlap incompatible command owners.
• Run post-upgrade synthetic rides, gateway tests, and job checks.
• Retain the previous signed application artifacts until the acceptance window closes.

Database rollback is not assumed. Reversible application rollout plus a reviewed forward repair or database restore is the strategy. Each migration declares rollback class: reversible, forward-fix-only, or restore-required.

16.5 Version compatibility

Maintain a release manifest covering:

• server images;
• database baseline/migration range;
• rider/operator/kiosk binaries, package/store identities, native module sets, signing provenance, and OTA runtime;
• gateway protocol versions;
• IoT parser bundle and firmware compatibility;
• customer configuration schema;
• supported platform versions.

The API rejects incompatible adapter major versions and warns on clients outside the supported window without blocking safety-required ride completion.

17. Migration and first-client cutover

17.1 Greenfield customer

1. Complete discovery and residency matrix.
2. Certify infrastructure and local gateways in sandbox.
3. Install empty environment and run baseline tests.
4. Configure operator IdP, branding, locale, pricing, zones, and policies.
5. Import vehicle/device inventory without live endpoint changes.
6. Issue customer-specific mobile builds and complete store/private distribution.
7. Point canary devices/SIMs to local IoT endpoint.
8. Run staff-only payment and full ride scenarios.
9. Run backup/restore, failure, security, and egress gates.
10. Expand device cohort, then open controlled rider cohort.
11. Observe through the agreed stability window.
12. Approve production launch and begin LTS support.

17.2 Existing Levy-cloud customer

Treat export/import as a separate, reviewed migration project.

• Freeze scope and define the authoritative cutover timestamp.
• Export only the customer's rows and authorized historical period.
• Map auth subjects, payment tokens (usually re-tokenized through the local gateway), device credentials, objects, and provider references.
• Run a rider re-enrollment campaign for payment credentials that cannot be transferred: defined in-app/SMS notices, grace and debt policy, support script, and activation dashboard. Measure re-tokenization among active riders and affected wallet/subscription/auto-top-up cohorts; Phase 0 sets the go/no-go percentage and treatment of riders below it.
• Encrypt transfer artifacts and keep them inside approved routes.
• Reconcile row counts, financial ledgers, active rides, devices, objects, and audit hashes.
• Do not migrate live payment secrets or cloud provider credentials.
• Plan mobile forced-update/endpoint transition and universal links.
• Quiesce new starts before final delta; safely finish or explicitly transfer active rides.
• Change IoT endpoints in canary waves.
• Keep the source read-only for the approved reconciliation period, then delete according to the signed data-migration agreement.

17.3 Cutover rollback

Rollback conditions and owners are agreed before starting. Examples:

• payment gateway cannot authorize/capture within thresholds;
• OTP delivery falls below the launch floor;
• IoT command acknowledgements or telemetry lag breach limits;
• reconciliation finds financial divergence;
• unauthorized egress occurs;
• backup or restore gate fails;
• critical rider/operator authorization defect;
• fleet cannot safely end rides or lock vehicles.

For a greenfield deployment, rollback normally means close new rentals and return to staff-only operation. For a migrated fleet, dual-write is avoided; rollback requires a predesigned source reactivation and delta reconciliation plan, not an improvised DNS change.

18. Verification strategy

18.1 Test layers

18.2 Core golden journeys

At minimum automate:

1. new rider OTP enrollment and consent;
2. returning rider token refresh/revocation;
3. valid and declined payment-method setup;
4. vehicle discovery and zone/pricing display;
5. reserve/hold expiry;
6. unlock with authorization and acknowledged IoT command;
7. ride metering across worker restart;
8. pause/resume when supported;
9. normal end ride with parking validation and lock;
10. end ride during payment-provider outage;
11. end ride during messaging outage;
12. lost webhook followed by reconciliation;
13. duplicate webhook/request/command retry;
14. refund and local receipt;
15. operator vehicle disable/enable and audit;
16. device disconnect/reconnect with safe queue behavior;
17. backup restore followed by synthetic ride-read validation;
18. prohibited-domain attempt causing residency gate failure;
19. active-ride end and lock during customer-IdP and attestation-provider outage;
20. wallet top-up/hold/ride/refund/reconciliation without direct-credit accounting drift;
21. conditional-module recovery job activation and config rejection when its mandatory companion job is absent.

18.3 Performance qualification

Create workload models by vehicles, connected devices, telemetry frequency, concurrent riders, active rides, operators, stored route points, photo volume, and retention.

Qualify at 2× the contracted expected peak and measure:

• telemetry frames/sec and end-to-end freshness;
• command dispatch/ack latency;
• API p50/p95/p99;
• ride-metering and completion lag;
• database CPU/IO/locks/storage growth;
• worker queue age;
• live-map update latency;
• gateway rate limits;
• backup duration and restore time.

Published sizing guidance is based on release-specific benchmark results; the signed customer capacity profile supplies the contractual limits for the installed environment.

19. Release assurance

Every production release is delivered as a signed, digest-pinned compatibility set covering the core services, installation controller, command-line client, database baseline and migrations, mobile applications, gateway contracts, runbooks, conformance catalogs, and evidence schemas.

Release qualification includes clean installation, upgrade, rollback, isolated restore, provider and IoT conformance, financial idempotency and reconciliation, safety behavior under dependency loss, residency capture, security testing, capacity testing, and independent cold-install certification. Customer-specific configuration is applied through a signed site package rather than a source fork.

The Engineering and Release Reference, Release Package Contract, Third-Party Installation Guide, and Installation Certification define the complete delivery and acceptance controls.