Source: https://github.com/aipoch/medical-research-skills
When to Use
Use this skill in any of the following situations:
- You need to load and standardize Neuropixels recordings from SpikeGLX (
.ap.bin/.lf.bin/.meta), Open Ephys (.continuous/.oebin), or NWB (.nwb) into a consistent analysis pipeline.
- You are preparing raw extracellular data for spike sorting, including high-pass filtering, phase shift correction (NP1.0), bad channel detection/removal, and common average referencing (CAR).
- You suspect probe drift or tissue motion and need to estimate and correct motion before sorting (especially when drift is > ~10 µm).
- You want to run spike sorting (Kilosort4 recommended; CPU alternatives supported) and then compute post-processing products (waveforms, templates, amplitudes, correlograms, unit locations).
- You need quality control and curation using Allen/IBL-style thresholds, plus optional AI-assisted visual review for borderline units, and exports to Phy/NWB.
Key Features
- Multi-format ingestion: SpikeGLX, Open Ephys, and NWB readers via SpikeInterface.
- Neuropixels-aware preprocessing:
- High-pass filtering for spike band
- Phase shift correction for Neuropixels 1.0
- Bad channel detection and removal
- Median CAR / referencing
- Motion/drift workflow:
- Motion estimation presets (e.g., “Kilosort-like”)
- Optional rigid/non-rigid correction presets
- Drift visualization outputs
- Spike sorting orchestration:
- Kilosort4 (GPU) recommended
- CPU alternatives (e.g., SpykingCircus2, Mountainsort5, Tridesclous2)
- Post-processing and QC:
- SortingAnalyzer-based computation of waveforms, templates, amplitudes, correlograms, unit locations, and quality metrics
- Curation:
- Allen/IBL-style automated labeling
- Optional AI-assisted visual analysis for uncertain units
- Reporting and export:
- HTML report generation
- Export to Phy and NWB
- Save metrics tables (CSV)
Reference guides (if present in the repository) can be used for deeper explanations:
reference/standard_workflow.md
reference/api_reference.md
reference/plotting_guide.md
reference/PREPROCESSING.md, reference/MOTION_CORRECTION.md, reference/SPIKE_SORTING.md
reference/QUALITY_METRICS.md, reference/AUTOMATED_CURATION.md, reference/AI_CURATION.md
Dependencies
Python dependencies (typical versions known to work; adjust to your environment):
python >= 3.9
spikeinterface[full] >= 0.99
probeinterface >= 0.2
neo >= 0.13
- Spike sorters (optional, depending on what you run):
kilosort >= 4.0 (Kilosort4; GPU required)
spykingcircus >= 1.1 (SpykingCircus2; CPU)
mountainsort5 >= 0.5 (CPU)
- Optional (AI-assisted curation):
- Optional (IBL tooling):
ibllib >= 2.0
ibl-neuropixel >= 1.0
Example Usage
The following example is designed to be a complete, runnable script (assuming dependencies and a valid dataset path). It loads SpikeGLX data, preprocesses, estimates/corrects motion, runs Kilosort4, computes metrics, curates units, generates a report, and exports to Phy and NWB.
import spikeinterface.full as si
import neuropixels_analysis as npa
def main():
# Parallelization / chunking settings used by SpikeInterface functions
job_kwargs = dict(n_jobs=-1, chunk_duration="1s", progress_bar=True)
# 1) Load data (SpikeGLX example)
# For Open Ephys: si.read_openephys("/path/to/Record_Node_101/")
# For NWB: si.read_nwb("/path/to/file.nwb")
recording = si.read_spikeglx("/path/to/spikeglx_folder", stream_id="imec0.ap")
# Optional: slice first 60 seconds for a quick test
fs = recording.get_sampling_frequency()
recording = recording.frame_slice(0, int(60 * fs))
# 2) Preprocess (recommended chain; wrapper may include the same steps)
# Note: phase_shift is mandatory for Neuropixels 1.0 and not needed for 2.0.
rec = npa.preprocess(recording)
# 3) Estimate drift/motion and correct if needed
motion_info = npa.estimate_motion(rec, preset="kilosort_like", **job_kwargs)
npa.plot_drift(rec, motion_info, output="drift_map.png")
# Example threshold: correct if max drift exceeds 10 µm
if float(motion_info["motion"].max()) > 10.0:
rec = npa.correct_motion(rec, preset="nonrigid_accurate", **job_kwargs)
# 4) Spike sorting (Kilosort4 recommended; requires GPU)
sorting = si.run_sorter("kilosort4", rec, folder="ks4_output", **job_kwargs)
# 5) Post-processing + metrics
analyzer = si.create_sorting_analyzer(sorting, rec, sparse=True)
analyzer.compute("random_spikes", max_spikes_per_unit=500, **job_kwargs)
analyzer.compute("waveforms", ms_before=1.0, ms_after=2.0, **job_kwargs)
analyzer.compute("templates", operators=["average", "std"], **job_kwargs)
analyzer.compute("spike_amplitudes", **job_kwargs)
analyzer.compute("correlograms", window_ms=50.0, bin_ms=1.0, **job_kwargs)
analyzer.compute("unit_locations", method="monopolar_triangulation", **job_kwargs)
analyzer.compute("quality_metrics", **job_kwargs)
metrics = analyzer.get_extension("quality_metrics").get_data()
metrics.to_csv("quality_metrics.csv")
# 6) Automated curation (Allen/IBL-style)
labels = npa.curate(metrics, method="allen") # e.g., "allen", "ibl", "strict"
# 7) Report
results = {"sorting": sorting, "metrics": metrics, "labels": labels, "analyzer": analyzer}
npa.generate_analysis_report(results, "output_report/")
npa.print_analysis_summary(results)
# 8) Export
si.export_to_phy(
analyzer,
output_folder="phy_export/",
compute_pc_features=True,
compute_amplitudes=True,
)
from spikeinterface.exporters import export_to_nwb
export_to_nwb(rec, sorting, "output.nwb")
if __name__ == "__main__":
main()
Implementation Details
1) Data I/O and supported formats
- SpikeGLX:
si.read_spikeglx(path, stream_id="imec0.ap")
- Open Ephys:
si.read_openephys(path)
- NWB:
si.read_nwb(path)
Neuropixels probe types commonly encountered:
- Neuropixels 1.0: requires phase shift correction to align channels.
- Neuropixels 2.0: denser geometries; phase shift correction typically not required.
2) Preprocessing chain (typical)
A standard spike-band preprocessing sequence is:
- High-pass filter (commonly 300–400 Hz) to isolate spikes.
- Phase shift correction (
si.phase_shift) for NP1.0.
- Bad channel detection (
si.detect_bad_channels) and removal.
- Common reference (often median CAR) to reduce shared noise.
Key parameters:
freq_min (high-pass cutoff): typical 300–400 Hz
- bad channel detection sensitivity (implementation-dependent; often exposed as thresholds/presets)
3) Motion estimation and correction
- Motion/drift can strongly degrade sorting quality; a practical rule is to inspect drift before sorting.
- Presets:
preset="kilosort_like": faster estimation aligned with common sorter assumptions
preset="nonrigid_accurate": more robust correction for severe drift
Operational threshold often used in practice:
- If estimated drift exceeds ~10 µm, apply correction before sorting.
4) Spike sorting
- Kilosort4 is recommended for Neuropixels due to speed and quality, but requires a GPU.
- CPU alternatives can be used when GPU is unavailable (at the cost of runtime and sometimes quality).
Sorter parameters to tune (Kilosort4 examples):
batch_size: samples per batch (often ~30000 by default)
nblocks: number of drift blocks (increase for long recordings)
Th_learned: detection threshold (lower → more spikes, potentially more false positives)
5) Post-processing and quality metrics
Using SortingAnalyzer, the pipeline typically computes:
- waveforms (window:
ms_before, ms_after)
- templates (average/std)
- spike amplitudes
- correlograms (e.g.,
window_ms=50, bin_ms=1)
- unit locations (e.g.,
monopolar_triangulation)
- quality metrics (e.g., SNR, ISI violations, presence ratio, amplitude cutoff)
Common QC thresholds (dataset-dependent; document your choices):
snr_threshold: often 3–5
isi_violations_ratio: often 0.01–0.5
presence_ratio: often 0.5–0.95
6) Curation logic (Allen/IBL-style)
A conservative “good unit” selection often combines:
- high presence ratio (stable across recording)
- low ISI violations (refractory period respected)
- low amplitude cutoff (less truncation / missed spikes)
Example rule (illustrative):
presence_ratio > 0.9
isi_violations_ratio < 0.5
amplitude_cutoff < 0.1
7) AI-assisted visual analysis (optional)
For borderline units (e.g., moderate SNR), AI-assisted review can be used to interpret:
- waveform shape consistency
- refractory period evidence in autocorrelograms
- amplitude stability and drift effects
- multi-unit contamination indicators
If your repository provides npa.analyze_unit_visually(...), it can be integrated with an API client (e.g., anthropic) to generate structured curation suggestions.
1---2name: neuropixels-analysis-23description: End-to-end Neuropixels extracellular electrophysiology analysis (SpikeGLX/Open Ephys/NWB) including preprocessing, motion correction, Kilosort4 spike sorting, QC metrics, and Allen/IBL-style curation; use when processing Neuropixels recordings or when users mention Neuropixels, SpikeGLX, Open Ephys, Kilosort, quality metrics, drift/motion correction, or unit curation.4license: MIT5author: AIPOCH6metadata:7 source: https://skillmd.com/skills/gabrielmoreira/neuropixels-analysis-28---910> **Source**: [https://github.com/aipoch/medical-research-skills](https://github.com/aipoch/medical-research-skills)1112## When to Use1314Use this skill in any of the following situations:15161. **You need to load and standardize Neuropixels recordings** from SpikeGLX (`.ap.bin/.lf.bin/.meta`), Open Ephys (`.continuous/.oebin`), or NWB (`.nwb`) into a consistent analysis pipeline.172. **You are preparing raw extracellular data for spike sorting**, including high-pass filtering, phase shift correction (NP1.0), bad channel detection/removal, and common average referencing (CAR).183. **You suspect probe drift or tissue motion** and need to estimate and correct motion before sorting (especially when drift is > ~10 µm).194. **You want to run spike sorting** (Kilosort4 recommended; CPU alternatives supported) and then compute post-processing products (waveforms, templates, amplitudes, correlograms, unit locations).205. **You need quality control and curation** using Allen/IBL-style thresholds, plus optional AI-assisted visual review for borderline units, and exports to Phy/NWB.2122## Key Features2324- **Multi-format ingestion**: SpikeGLX, Open Ephys, and NWB readers via SpikeInterface.25- **Neuropixels-aware preprocessing**:26 - High-pass filtering for spike band27 - **Phase shift correction for Neuropixels 1.0**28 - Bad channel detection and removal29 - Median CAR / referencing30- **Motion/drift workflow**:31 - Motion estimation presets (e.g., “Kilosort-like”)32 - Optional rigid/non-rigid correction presets33 - Drift visualization outputs34- **Spike sorting orchestration**:35 - Kilosort4 (GPU) recommended36 - CPU alternatives (e.g., SpykingCircus2, Mountainsort5, Tridesclous2)37- **Post-processing and QC**:38 - SortingAnalyzer-based computation of waveforms, templates, amplitudes, correlograms, unit locations, and quality metrics39- **Curation**:40 - Allen/IBL-style automated labeling41 - Optional AI-assisted visual analysis for uncertain units42- **Reporting and export**:43 - HTML report generation44 - Export to **Phy** and **NWB**45 - Save metrics tables (CSV)4647Reference guides (if present in the repository) can be used for deeper explanations:48- `reference/standard_workflow.md`49- `reference/api_reference.md`50- `reference/plotting_guide.md`51- `reference/PREPROCESSING.md`, `reference/MOTION_CORRECTION.md`, `reference/SPIKE_SORTING.md`52- `reference/QUALITY_METRICS.md`, `reference/AUTOMATED_CURATION.md`, `reference/AI_CURATION.md`5354## Dependencies5556Python dependencies (typical versions known to work; adjust to your environment):5758- `python >= 3.9`59- `spikeinterface[full] >= 0.99`60- `probeinterface >= 0.2`61- `neo >= 0.13`62- Spike sorters (optional, depending on what you run):63 - `kilosort >= 4.0` (Kilosort4; GPU required)64 - `spykingcircus >= 1.1` (SpykingCircus2; CPU)65 - `mountainsort5 >= 0.5` (CPU)66- Optional (AI-assisted curation):67 - `anthropic >= 0.20`68- Optional (IBL tooling):69 - `ibllib >= 2.0`70 - `ibl-neuropixel >= 1.0`7172## Example Usage7374The following example is designed to be a complete, runnable script (assuming dependencies and a valid dataset path). It loads SpikeGLX data, preprocesses, estimates/corrects motion, runs Kilosort4, computes metrics, curates units, generates a report, and exports to Phy and NWB.7576```python77import spikeinterface.full as si78import neuropixels_analysis as npa7980def main():81 # Parallelization / chunking settings used by SpikeInterface functions82 job_kwargs = dict(n_jobs=-1, chunk_duration="1s", progress_bar=True)8384 # 1) Load data (SpikeGLX example)85 # For Open Ephys: si.read_openephys("/path/to/Record_Node_101/")86 # For NWB: si.read_nwb("/path/to/file.nwb")87 recording = si.read_spikeglx("/path/to/spikeglx_folder", stream_id="imec0.ap")8889 # Optional: slice first 60 seconds for a quick test90 fs = recording.get_sampling_frequency()91 recording = recording.frame_slice(0, int(60 * fs))9293 # 2) Preprocess (recommended chain; wrapper may include the same steps)94 # Note: phase_shift is mandatory for Neuropixels 1.0 and not needed for 2.0.95 rec = npa.preprocess(recording)9697 # 3) Estimate drift/motion and correct if needed98 motion_info = npa.estimate_motion(rec, preset="kilosort_like", **job_kwargs)99 npa.plot_drift(rec, motion_info, output="drift_map.png")100101 # Example threshold: correct if max drift exceeds 10 µm102 if float(motion_info["motion"].max()) > 10.0:103 rec = npa.correct_motion(rec, preset="nonrigid_accurate", **job_kwargs)104105 # 4) Spike sorting (Kilosort4 recommended; requires GPU)106 sorting = si.run_sorter("kilosort4", rec, folder="ks4_output", **job_kwargs)107108 # 5) Post-processing + metrics109 analyzer = si.create_sorting_analyzer(sorting, rec, sparse=True)110111 analyzer.compute("random_spikes", max_spikes_per_unit=500, **job_kwargs)112 analyzer.compute("waveforms", ms_before=1.0, ms_after=2.0, **job_kwargs)113 analyzer.compute("templates", operators=["average", "std"], **job_kwargs)114 analyzer.compute("spike_amplitudes", **job_kwargs)115 analyzer.compute("correlograms", window_ms=50.0, bin_ms=1.0, **job_kwargs)116 analyzer.compute("unit_locations", method="monopolar_triangulation", **job_kwargs)117 analyzer.compute("quality_metrics", **job_kwargs)118119 metrics = analyzer.get_extension("quality_metrics").get_data()120 metrics.to_csv("quality_metrics.csv")121122 # 6) Automated curation (Allen/IBL-style)123 labels = npa.curate(metrics, method="allen") # e.g., "allen", "ibl", "strict"124125 # 7) Report126 results = {"sorting": sorting, "metrics": metrics, "labels": labels, "analyzer": analyzer}127 npa.generate_analysis_report(results, "output_report/")128 npa.print_analysis_summary(results)129130 # 8) Export131 si.export_to_phy(132 analyzer,133 output_folder="phy_export/",134 compute_pc_features=True,135 compute_amplitudes=True,136 )137138 from spikeinterface.exporters import export_to_nwb139 export_to_nwb(rec, sorting, "output.nwb")140141if __name__ == "__main__":142 main()143```144145## Implementation Details146147### 1) Data I/O and supported formats148149- **SpikeGLX**: `si.read_spikeglx(path, stream_id="imec0.ap")`150- **Open Ephys**: `si.read_openephys(path)`151- **NWB**: `si.read_nwb(path)`152153Neuropixels probe types commonly encountered:154- **Neuropixels 1.0**: requires **phase shift correction** to align channels.155- **Neuropixels 2.0**: denser geometries; phase shift correction typically not required.156157### 2) Preprocessing chain (typical)158159A standard spike-band preprocessing sequence is:1601611. **High-pass filter** (commonly 300–400 Hz) to isolate spikes.1622. **Phase shift correction** (`si.phase_shift`) for **NP1.0**.1633. **Bad channel detection** (`si.detect_bad_channels`) and removal.1644. **Common reference** (often median CAR) to reduce shared noise.165166Key parameters:167- `freq_min` (high-pass cutoff): typical **300–400 Hz**168- bad channel detection sensitivity (implementation-dependent; often exposed as thresholds/presets)169170### 3) Motion estimation and correction171172- Motion/drift can strongly degrade sorting quality; a practical rule is to **inspect drift before sorting**.173- Presets:174 - `preset="kilosort_like"`: faster estimation aligned with common sorter assumptions175 - `preset="nonrigid_accurate"`: more robust correction for severe drift176177Operational threshold often used in practice:178- If estimated drift exceeds **~10 µm**, apply correction before sorting.179180### 4) Spike sorting181182- **Kilosort4** is recommended for Neuropixels due to speed and quality, but requires a GPU.183- CPU alternatives can be used when GPU is unavailable (at the cost of runtime and sometimes quality).184185Sorter parameters to tune (Kilosort4 examples):186- `batch_size`: samples per batch (often ~30000 by default)187- `nblocks`: number of drift blocks (increase for long recordings)188- `Th_learned`: detection threshold (lower → more spikes, potentially more false positives)189190### 5) Post-processing and quality metrics191192Using `SortingAnalyzer`, the pipeline typically computes:193- waveforms (window: `ms_before`, `ms_after`)194- templates (average/std)195- spike amplitudes196- correlograms (e.g., `window_ms=50`, `bin_ms=1`)197- unit locations (e.g., `monopolar_triangulation`)198- quality metrics (e.g., SNR, ISI violations, presence ratio, amplitude cutoff)199200Common QC thresholds (dataset-dependent; document your choices):201- `snr_threshold`: often **3–5**202- `isi_violations_ratio`: often **0.01–0.5**203- `presence_ratio`: often **0.5–0.95**204205### 6) Curation logic (Allen/IBL-style)206207A conservative “good unit” selection often combines:208- high presence ratio (stable across recording)209- low ISI violations (refractory period respected)210- low amplitude cutoff (less truncation / missed spikes)211212Example rule (illustrative):213- `presence_ratio > 0.9`214- `isi_violations_ratio < 0.5`215- `amplitude_cutoff < 0.1`216217### 7) AI-assisted visual analysis (optional)218219For borderline units (e.g., moderate SNR), AI-assisted review can be used to interpret:220- waveform shape consistency221- refractory period evidence in autocorrelograms222- amplitude stability and drift effects223- multi-unit contamination indicators224225If your repository provides `npa.analyze_unit_visually(...)`, it can be integrated with an API client (e.g., `anthropic`) to generate structured curation suggestions.