Embedded AI for IoT

Placement decisions

Run inference locally when low latency, offline operation, privacy, or bandwidth reduction matters and the model fits the device’s compute, memory, and energy budget. Keep training, very large models, or infrequent/complex processing in the cloud when edge hardware cannot support the model or when centralized data is needed.

A hybrid design is common:

• a low-power local keyword detector wakes a voice assistant;
• a smart camera performs cheap local motion/object filtering and sends selected frames or features to a cloud model for expensive classification.

The trade-off is that edge devices have limited compute, memory, flash, and energy; compressed models may lose accuracy and are harder to update/deploy.

Case studies and project ideas

The placement decision should follow the consequence of delay, connectivity, privacy, bandwidth, and model size—not simply whether a device is called “smart.” A useful rule is: local inference for immediate decisions; cloud processing for large-scale learning and comparison; hybrid operation when both matter.

Local or edge-first projects

Case/project	Local AI task	Why local is better	Possible cloud role and limitation
Wearable fall detector using an accelerometer and gyroscope	Classify a short motion window as walking, sitting, or falling	An alarm must work when the phone/network is unavailable; sending raw motion continuously wastes energy and exposes personal data	Send only fall events and summaries to caregivers. False positives and battery use must be evaluated.
Industrial vibration monitor on a motor	Detect an abnormal vibration pattern and trigger a local warning or shutdown	Millisecond-level response and continued operation during an outage are more important than a global view	Upload features and periodic data for fleet-level predictive maintenance. A safety action still needs validated fail-safe logic, not only an ML prediction.
Privacy-preserving occupancy camera	Detect people or count occupancy locally; discard the raw image	Video bandwidth and privacy risks are reduced, and a local HVAC/light decision has low latency	Send counts or selected events to a dashboard. A large cloud model could improve recognition but would require sending images or embeddings.
Embedded keyword detector in a voice device	Detect a wake word continuously	A tiny model can remain active at low power without streaming all audio	After activation, send a short audio segment to the cloud for speech recognition. The local model may miss accents or noisy speech.
Greenhouse controller	Predict irrigation need from soil moisture, temperature, and light	Watering must continue during an Internet outage and local control avoids unnecessary round trips	The cloud can compare multiple greenhouses and update a model. Local fallback rules should remain available if the model or connection fails.

Cloud-first projects

Case/project	Cloud AI task	Why cloud is better	Important limitation
Fleet predictive maintenance	Train and run a model using vibration, temperature, and service history from thousands of machines	The cloud can combine data across devices and run expensive retraining; an individual machine may not have enough history	The device should still perform a simple local threshold or emergency check because cloud inference has network latency.
City-wide energy forecasting	Forecast demand across buildings and weather zones	The model benefits from large historical datasets, cross-building correlations, and centralized updates	Send aggregates rather than every raw sample when bandwidth or privacy is a concern.
Multi-site traffic analytics	Combine camera counts, weather, incidents, and road topology to predict congestion	The problem spans many locations and requires a global view that one roadside device cannot maintain	Local devices can still detect immediate hazards; cloud results may arrive too late for emergency control.
Large-model medical or industrial image analysis	Analyze high-resolution images with a large classifier	The model may exceed the device’s memory and compute budget, and centralized model versioning is useful	Images may be sensitive or expensive to upload; local pre-filtering and encryption may be required.

Hybrid project pattern

A strong project separates the fast local loop from the slower cloud loop:

Sensor → local filtering/inference → immediate action or event
                    ↓
             selected features/events
                    ↓
Cloud storage → fleet analytics → model training/update

Example: a motor controller detects a possible bearing fault locally, keeps the machine within a safe operating mode, and sends a short feature vector to the cloud. The cloud compares the vector with failures across the fleet and periodically returns an updated quantized model.

Project-selection checklist

For each proposed project, state:

1. Input: sensor stream, image, audio, or device state.
2. Local decision: what must happen without the cloud?
3. Cloud task: what benefits from aggregation, storage, or a larger model?
4. Constraint: latency, energy, privacy, connectivity, memory, or bandwidth.
5. Model strategy: FP32 baseline, INT8 quantization, pruning, or binarization.
6. Evaluation: accuracy, false alarms, latency, energy per inference, memory, and bytes transmitted.

Pruning vs quantization

Technique	Operation	Network shape	Main benefit
Magnitude-based pruning	Set small-magnitude weights to zero, usually with a mask.	Sparse; may require sparse storage/hardware.	Fewer effective connections and operations.
Quantization	Reduce precision, e.g. FP32 to INT8, for weights and activations.	Usually still dense; same connections.	Smaller memory and cheaper integer arithmetic.

Magnitude pruning usually trains a dense network, ranks weights by |w|, chooses a threshold or percentage, masks the smallest weights, and fine-tunes. It may be iterated. The assumption that small weights are unimportant can fail: importance depends on activations, paths, and interactions, so removing a small but critical connection or an entire neuron’s inputs can cause a sharp accuracy drop.

Binarization is extreme quantization: weights/activations are represented with two values such as +1 and -1. Dot products can use bitwise XNOR plus population count instead of floating-point multiplication, which is attractive for specialized hardware. It can, however, introduce a larger accuracy/optimization challenge.

Sources and scope note

• master-cs/raw/iot/4.6-IoT-Connected Data - Machine Learning.md
• master-cs/raw/iot/Question Sheet Week 6.md