# Latency

> Measures inference latency of binarized, 8-bit, and 32-bit convolutional layers on edge devices to evaluate the efficiency and speedup of the Larq Compute Engine framework compared to standard implementations.

- Skill: `qhjqhj00/latency` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/latency`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/latency/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML, Model Training & Fine-tuning
- Tags: Benchmarking, Binarized Neural Networks, Convolutional Layers, Edge Devices, Larq Compute Engine, Latency
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-08-22
- Page: https://skillmd.com/skills/qhjqhj00/latency

---


# latency

> Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art Binarized Neural Networks — Bannink et al. (2020) (arXiv:2011.09398, 2020)

## What this evaluates

Measures the inference latency of binarized, 8-bit, and 32-bit convolutional layers on edge devices to evaluate the efficiency and speedup of the Larq Compute Engine framework compared to standard implementations.

## Datasets

- **Synthetic-Conv-Dimensions** — total ?; splits: test (-1)

## Metrics

- `latency` **(primary)** — range: ms
  - Wall-clock execution time for a single forward pass of a convolutional layer on the target hardware.

## Input / output format

**Input**: Convolutional layer specifications: input height, input width, input channels, output channels, and kernel size (3x3 or 5x5).

**Output**: Execution latency in milliseconds (ms).

## Scoring recipe

```python
def compute_latency(conv_specs, framework, device):
    times = []
    for spec in conv_specs:
        model = build_conv_layer(spec)
        framework.load(model)
        framework.run(model, dummy_input)  # warmup
        start = time.perf_counter()
        framework.run(model, dummy_input)
        end = time.perf_counter()
        times.append((end - start) * 1000)  # convert to ms
    return sum(times) / len(times)
```

## Common pitfalls

- Latency is highly platform-dependent and varies significantly across different hardware architectures, instruction sets, and inference frameworks.
- Overheads from bitpacking, im2col, and memory reads can cause deviations from theoretical MAC-based speedup predictions, especially for smaller convolutions.
- Framework fallbacks (e.g., to slower full-precision code) can dominate measured latency if not detected or disabled.

## Evidence (verbatim from paper)

> The measurements were taken on a Pixel 1 phone as well as a Raspberry Pi Model 4B with a 64-bit OS (Ubuntu LTS 20.04). ... Binarization reduces latency by 12-17x compared to the floating-point implementation, with the largest performance gains being in the layers with the most channels.

## Citation

```bibtex
@misc{bannink2020larq,
  title={Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art Binarized Neural Networks},
  author={Bannink et al. (2020)},
  year={2020},
  note={arXiv:2011.09398}
}
```

- arXiv: 2011.09398