Resource-Aware Distributed Systems Techniques for Edge AI Environments

Loading...
Thumbnail Image

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

As artificial intelligence expands into distributed edge environments, new systems challenges emerge around coordination and inference under resource constraints. Traditional approaches assume reliable connectivity to centralized cloud infrastructure, but edge deployments often operate with limited bandwidth, intermittent connections, and heterogeneous hardware. These conditions demand fundamentally different design choices for both how nodes agree on system state and how inference workloads get allocated across available resources. This thesis presents two complementary contributions addressing these challenges. First, Stake-Weighted Leader Vote is a lightweight consensus protocol that reinterprets stake as committed compute budget rather than monetary value. In conventional blockchain systems, stake represents financial collateral used to incentivize honest behavior. SWLV adapts this concept for edge AI by treating each node's dedicated processing capacity as its stake, allowing nodes with greater compute commitments to carry proportionally more influence in leader election and block validation. This design reduces communication overhead and aligns consensus participation with actual resource contribution, making it well suited for environments where nodes vary significantly in capability. Second, DIT (Distributed Inference Tables) is a framework for routing inference queries across domain-specialized open-source models. Rather than relying on a single general-purpose model hosted remotely, DIT maintains a routing table that maps query characteristics to the most appropriate local or nearby model. Each model in the network can specialize in a particular domain, and the routing layer directs requests based on factors like model expertise, current load, and latency requirements. This allows a collection of smaller, focused models to collectively cover a broad range of tasks while keeping inference fast and local. Together, SWLV and DIT demonstrate that lightweight coordination and resource-aware inference can make AI practical in edge environments without dependence on centralized infrastructure. By rethinking consensus around compute rather than capital and distributing inference across specialized models, these systems offer a viable path toward resilient and scalable AI at the edge.

Description

Keywords

Distributed Systems, Edge AI, Distributed Inference, Small Language Models, LLMs, Consensus Algorithms

Citation

Endorsement

Review

Supplemented By

Referenced By